RU2641461C2 - Audio encoder, audio decoder, method of providing coded audio information, method of providing decoded audio information, computer program and coded presentation using signal-adaptive bandwidth extension - Google Patents

Abstract

FIELD: physics.SUBSTANCE: audio encoder comprises a low-frequency encoder for obtaining an encoded representation of the low-frequency portion and a unit for providing bandwidth extension information based on the input audio information. The audio encoder is also configured to selectively include the bandwidth extension information in the encoded audio information. The audio decoder comprises a low-frequency decoder to obtain a decoded representation of the low-frequency portion and an extension of the bandwidth to obtain a blind-bandwidth expansion signal for portions of audio content, for which the bandwidth expansion parameters are not included in the coded audio information and to obtain a bandwidth extension signal for portions of audio content, for which the parameters of the bandwidth extension are included in the coded audio information.EFFECT: providing bandwidth expansion to improve the tradeoff between the bit rate and the sound quality.38 cl, 8 dwg

Description

FIELD OF THE INVENTION

Embodiments according to the invention relate to an audio encoder for providing encoded audio information based on the input audio information.

Additional embodiments of the invention relate to an audio decoder for providing decoded audio information based on encoded audio information.

Additional embodiments of the invention relate to a method for providing encoded audio information based on the input audio information.

Additional embodiments of the invention relate to a method for providing decoded audio information based on encoded audio information.

Additional embodiments of the invention relate to a computer program for implementing one of the above methods.

Further embodiments according to the invention relate to an encoded audio presentation representing audio information.

Some embodiments of the invention relate to the general extension of the bandwidth of an audio signal with a signal rate of adaptive auxiliary information for encoding audio with a very low bit rate.

BACKGROUND

In recent years, the need for encoding and decoding audio content has been constantly growing. Although the available bit rates and capacities of data storage devices for transmitting and storing encoded audio content are substantially increasing, there is still a need for efficient bit rate encoding, transmission, storage and decoding of audio content with reasonable quality, in particular, speech signals in communication scenarios .

Modern speech coding systems allow encoding of wideband (WB) digital audio content, i.e. signals with frequencies up to 7-8 kHz, at bit rates of only 6 Kbit / s. The most widely discussed examples are ITU-T G.722.2 recommendations (see, for example, reference material [1]), as well as the later developed G.718 (see, for example, reference materials [4] and [10]) and xHE-AAC unified speech and audio-MPEG codec (see, for example, reference material [8]). Both G.722.2, also known as AMR-WB, and G.718 use 6.4-7 kHz bandwidth extension (BWE) technologies to enable the underlying ACELP core encoder to “focus” on more relevant the point of view of perception of lower frequencies (in particular, at frequencies at which the human auditory system is phase-sensitive) and due to this, to achieve sufficient quality, in particular, at very low bit rates. In xHE-AAC, Advanced Spectrum Band Replication (eSBR) is used for bandwidth extension (BWE). The process of expanding bandwidth can, in general, be divided into two conceptual approaches.

A BWE is “blind” (or “artificial” BWE) in which the high frequency (HF) components are reconstructed only from the decoded low frequency (LF) signal of the base encoder, i.e. without the need to transmit auxiliary information from the encoder. This scheme is used by AMR-WB and G.718 at 16 Kbps and below, as well as through some backward compatible post-processing systems with bandwidth expansion that control traditional narrow-band telephone speech (see, for example, reference materials [5] and [ 9]).

A “guided” BWE that differs from blind bandwidth expansion in that some parameters used to recover high-frequency (HF) content are transmitted to the decoder as auxiliary information instead of being estimated from the decoded base signal. AMR-WB, G.718, xHE-AAC, as well as some other codecs (see, for example, reference materials [2], [7] and [11]) use this approach, but not at very low bit rates.

However, it was found that it is difficult to ensure proper bandwidth expansion at low bit rates, which provides good enough quality when restoring audio content.

Thus, there is a need for a bandwidth extension principle that promotes an improved trade-off between bit rate and sound quality.

SUMMARY OF THE INVENTION

An embodiment of the invention creates an audio encoder for providing encoded audio information based on the input audio information. The audio encoder comprises a low-frequency encoder, configured to encode the low-frequency part of the input audio information in order to obtain an encoded representation of the low-frequency part. The audio encoder also comprises a bandwidth extension information providing unit adapted to provide bandwidth extension information based on the input audio information. The audio encoder is configured to selectively include bandwidth extension information in encoded audio information in a signal-adaptive manner.

This embodiment according to the invention is based on such identified information that for some types of audio content and even for some parts of an adjacent piece of audio content, good quality bandwidth expansion can be achieved based on the encoded representation of the low frequency part without any additional bandwidth extension information or with only a small amount the amount of auxiliary bandwidth extension information (for example, with a small number of bandwidth extension parameters ropuskaniya which are included in the encoded audio information). However, the principle is also based on such identified information that for other types of audio content and even for other parts of an adjacent piece of audio content, it may be necessary (or at least very desirable) to include additional bandwidth extension information (e.g., highlighted bandwidth expansion parameters bandwidth) or an increased amount of auxiliary information for expanding the bandwidth (for example, compared with the above case) into encoded audio information, since otherwise However, bandwidth expansion on the decoder side does not provide satisfactory sound quality.

By selectively including bandwidth extension information in encoded audio information (for example, by selectively varying the amount of bandwidth extension information or bandwidth extension parameters included in encoded audio information, or by selectively switching between including bandwidth extension information in encoded audio information and excluding said inclusion of information audio encoded bandwidth extensions information), it is possible to prevent the “optional” bandwidth extension information from using valuable bit rates for the case where the bandwidth extension on the decoder side does not really require bandwidth extension information, and yet it can be ensured that bandwidth extension information (or an increased amount of bandwidth extension information) is included in the encoded audio information if the bandwidth extension information is actually required to expand rhenium bandwidth on the decoder side, i.e. to restore audio content on the side of the decoder.

Thus, by selectively including bandwidth extension information in the encoded audio information in a signal-adaptive manner, i.e. when bandwidth extension information is actually necessary to achieve a sufficiently good quality of the decoded representation of the audio signal, the average bit rate may decrease while maintaining the ability to obtain high sound quality.

In other words, the audio encoder, for example, can switch between providing bandwidth extension information that provides parameter-driven bandwidth expansion on the side of the audio decoder, and eliminating providing bandwidth extension information that requires the use of blindband expansion on the side of the audio decoder.

Accordingly, a very good compromise between bit rate and sound quality can be obtained using the above principle.

In a preferred embodiment, the audio encoder comprises a detector configured to identify portions of the input audio information that cannot be decoded with sufficient or desired quality (for example, in terms of a given quality score) based on the encoded representation of the low frequency portion and using the blind bandwidth extension . In this case, the audio encoder is configured to selectively include bandwidth extension information in encoded audio information for portions of the input audio information identified by the detector. By determining or evaluating (for example, based on the properties of the input audio information or based on partial or complete restoration of the audio information on the side of the audio encoder) which parts of the input audio information cannot be decoded with sufficient (or desired) quality, based on the encoded representation of the low-frequency part and using blind bandwidth expansion, a significant criterion is obtained for determining whether to include bandwidth extension information in coded a audio information, for parts (for example, frames) of the input audio information (or equivalently, for frames or parts of encoded audio information). In other words, the above criterion, which is evaluated by the detector, provides a good compromise between the listening experience that can be achieved by decoding the encoded audio information and the bit rate for the encoded audio information.

In a preferred embodiment, the audio encoder comprises a detector configured to identify portions of the input audio information for which bandwidth expansion parameters cannot be estimated based on the low frequency portion with sufficient or desired accuracy. In this case, the audio encoder is configured to selectively include bandwidth extension information in encoded audio information for portions of the input audio information identified by the detector. This embodiment according to the invention is based on such identified information that determining whether bandwidth expansion parameters can be estimated based on the low-frequency part with sufficient or desired accuracy is a criterion that can be estimated at low computational cost, and which however, constitutes a good criterion for determining whether to include bandwidth extension information in encoded audio information.

In a preferred embodiment, the audio encoder comprises a detector configured to identify parts of the input audio information depending on whether the parts are parts that are motionless in time and depending on whether the parts are of a low frequency nature. In addition, the audio encoder is configured to selectively exclude the inclusion of bandwidth extension information in encoded audio information for parts of the input audio information identified by the detector as stationary in time parts having a low frequency character.

This embodiment according to the invention is based on such identified information that it is usually not necessary to include bandwidth extension information in encoded audio information for parts of the input audio information that are motionless in time and have a low frequency character, since the bandwidth extension is blind (which is not based on the extension information bandwidth or parameters from the bit stream) usually provides the possibility of a sufficiently good recovery of such parts signal. Accordingly, a criterion is provided that can be evaluated in a computationally efficient manner, and which nonetheless provides good results (in terms of a compromise between bit rate and sound quality).

In a preferred embodiment, the detector is configured to identify parts of the input audio information depending on whether the parts contain voiced speech, and / or depending on whether the parts contain ambient noise (e.g., cars), and / or depending on whether parts contain music without percussion instruments. It has been found that those parts that contain voiced speech, or that contain ambient noise, or that contain music without percussion instruments, can usually be restored using blindfold bandwidth extension with sufficient sound quality, so it is recommended that bandwidth extension information be included in the encoded audio information for such parts.

In a preferred embodiment, the audio encoder comprises a detector configured to identify portions of the input audio information depending on whether the difference between the spectral envelope of the low frequency part and the spectral envelope of the high frequency part exceeds or is equal to or equal to a predetermined difference index. In this case, the audio encoder is configured to selectively include bandwidth extension information in encoded audio information for portions of the input audio information identified by the detector.

It has been found that parts of the input audio information that contain a large difference between the spectral envelope of the low-frequency part and the spectral envelope of the high-frequency part cannot usually be reconstructed well using blind bandwidth expansion, since blind bandwidth expansion often provides similar spectral envelopes in the high-frequency part (i.e. e. in the bandwidth extension signal) compared to the corresponding low-frequency part. Accordingly, it has been found that estimating the difference between the spectral envelope of the low-frequency part and the spectral envelope of the high-frequency part is a good criterion for determining whether to include bandwidth extension information in encoded audio information.

In a preferred embodiment, the detector is configured to identify parts of the input audio information depending on whether the parts contain unvoiced speech and / or depending on whether the parts contain sounds of percussion instruments. It has been found that parts containing unvoiced speech and parts containing percussion instrument sounds typically contain spectra in which the spectral envelope of the low-frequency part differs significantly from the spectral envelope of the high-frequency part. Accordingly, it has been found that the detection of unvoiced speech and / or percussion sounds is a good criterion for determining whether to include bandwidth extension information in encoded audio information.

In a preferred embodiment, the audio encoder comprises a detector configured to determine a spectral tilt of parts of the input audio information and to identify parts of the input audio information depending on whether the specific spectral tilt is greater than or equal to a fixed or variable threshold tilt value. In this case, the audio encoder is configured to selectively include bandwidth extension information in encoded audio information for portions of the input audio information identified by the detector. It has been found that the spectral tilt can be extracted at a low computational cost, while providing a good determination criterion as to whether bandwidth extension information should be included in the encoded audio information. For example, if the spectral tilt reaches or exceeds the threshold tilt value, it can be concluded that the spectrum has a high-frequency character and cannot be well restored by expanding the blind bandwidth. In particular, blind bandwidth expansion usually cannot restore spectra containing a positive slope (with the high-frequency part being emphasized with respect to the low-frequency part), with good accuracy. In addition, since the high-frequency part has a specific perceptual relevance in the case of a positive spectral tilt, it is recommended in such cases to include bandwidth extension information in an encoded audio presentation.

In a preferred embodiment, the detector is further configured to determine the frequency of zero transitions of parts of the input audio information and to identify parts of the input audio information also depending on whether or not a certain frequency of zero transitions exceeds a fixed or variable threshold value of the zero transition frequency. It has been found that the zero-crossing rate is also a good criterion for detecting portions of the input audio information that cannot be well reconstructed using blind bandwidth expansion, so it is advisable (in terms of achieving a good compromise between bit rate and quality audio) include bandwidth extension information in encoded audio information.

In a preferred embodiment, the detector is configured to use hysteresis to identify parts of the signal of the input audio information in order to reduce the number of transitions between the identified parts of the signal (for which the bandwidth extension information is included in the encoded audio presentation) and the unidentified parts of the signal (for which the bandwidth extension information not included in encoded audio presentation). It has been found that it is preferable not to allow excessive switching between including bandwidth extension information in encoded audio information and excluding inclusion of bandwidth extension information in encoded audio presentation, since such transitions may contribute to some artifacts, in particular if the number of transitions is very large. Accordingly, using hysteresis, which, for example, can be applied to a slope threshold value (which in this case is a variable slope threshold value) or to a threshold value of zero transitions (which in this case is a variable threshold value of zero transitions), this goal can be achieved.

In a preferred embodiment, the audio encoder is configured to selectively include parameters representing the spectral envelope of the high-frequency part of the input audio information in the encoded audio information in a signal-adaptive manner as bandwidth extension information. This embodiment is based on the idea that the parameters representing the spectral envelope of the high-frequency part are particularly important in the parameter-guided expansion of the passband, so that the inclusion of the above parameters representing the spectral envelope of the high-frequency part of the input audio information allows achieving a bandwidth extension with good quality without resulting high bit rate.

In a preferred embodiment, the low-frequency encoder is configured to encode the low-frequency part of the input audio information containing frequencies up to the maximum frequency, which is in the range of 6-7 kHz. In addition, the audio encoder is configured to selectively include from three to five parameters describing the intensities of the high-frequency parts or sub-parts of the signal (e.g., parts of the signal having frequencies above about 6-7 kHz) having a passband of 300-500 Hz. It was found that such a principle leads to high sound quality without a significant increase in costs in the form of bit rate.

In a preferred embodiment, the audio encoder is configured to selectively include 3-5 scalar quantized parameters describing the intensities of the four high-frequency parts (or sub-parts) of the signal in the encoded audio presentation, the high-frequency parts (or sub-parts) of the signal covering frequency ranges above the low-frequency part. It has been found that the use of 3-5 scalar quantized parameters describing the intensities of the four high-frequency parts of the signal is usually sufficient to achieve a parameter-guided bandwidth extension that exceeds the relatively low sound quality obtained by blindly extending the bandwidth on an identical part of the signal. Accordingly, there are no large differences in quality between the reconstructed parts of the audio signal, regardless of whether the reconstructed parts of the audio signal using the blind bandwidth extension or the directional bandwidth extension are restored. Thus, the above principle is well adapted to the principle that provides the ability to switch between blind bandwidth expansion and parameter-guided bandwidth expansion.

In a preferred embodiment, the audio encoder is configured to selectively include in the encoded audio presentation a plurality of parameters describing the relationship between the energies of the spectrally adjacent frequency parts, one of the parameters describing the relationship between the energy of the first high-frequency part and the low-frequency part of the bandwidth extension, and other parameters describe the relationship between the energies (pairs) of other high-frequency parts of the expansion of the bandwidth. It has been found that such a principle describing the relationships (or differences) between the energies (or, equivalently, intensities) of different (preferably adjacent) frequency parts provides the ability to efficiently encode bandwidth extension information. It has also been found that such parameters describing the relationship between the energies of spectrally adjacent frequency parts can usually be quantized with only a small number of bits without significantly degrading the sound quality obtained by expanding the bandwidth.

Another embodiment according to the invention provides an audio decoder for providing decoded audio information based on encoded audio information. The audio decoder comprises a low-frequency decoder, configured to decode the encoded representation of the low-frequency part (audio content) in order to obtain a decoded representation of the low-frequency part. The audio decoder also comprises a bandwidth extension configured to receive a bandwidth extension signal using blindly bandwidth extension for parts of audio content for which the bandwidth extension parameters are not included in the encoded audio information, and to obtain a bandwidth extension signal using the bandwidth extension parameter guided for parts of audio content for which bandwidth extension parameters are included in the encoded audio information.

This audio encoder is based on the idea that a good compromise between sound quality and bit rate can be obtained if you can switch between blind bandwidth expansion and parameter-guided bandwidth expansion even within an adjacent piece of audio content, since it has been found that many characteristic pieces of audio content contain both sections for which high sound quality can be obtained using blind bandwidth expansion, and sections for which ebuetsya guided bandwidth extension parameters so as to achieve sufficient sound quality. In addition, it should be obvious that the identical considerations explained above with respect to the audio encoder also apply to the audio decoder.

In a preferred embodiment, the audio decoder is configured to determine whether to receive a bandwidth extension signal using a blindband extension or using a parameterized bandwidth extension on a frame-by-frame basis. It has been found that such a fine-tuning (frame-by-frame) switching between blind bandwidth extension and bandwidth-guided parameters helps keep the bit rate reasonably low even if some frames regularly appear in which parameter-guided bandwidth expansion is required in order to avoid excessive deterioration in audio content.

In a preferred embodiment, the audio decoder is configured to switch between using blind bandwidth expansion and parameter-guided bandwidth expansion within an adjacent piece of audio content. This embodiment is based on such revealed information that even one (adjacent) fragment of audio content often contains passages (or parts or frames) of various types, some of which must be encoded (and therefore decoded) using the bandwidth extension while other passages or frames can be decoded using blind bandwidth extension without significant degradation of sound quality.

In a preferred embodiment, the audio decoder is configured to evaluate flags included in the encoded audio information for different parts (e.g., frames) of audio content to determine whether to use blind bandwidth extension or parameter-guided bandwidth extension (e.g., for the frame with which it is associated flag). Accordingly, determining whether a bandwidth extension is used blindly or parameter-guided bandwidth expansion is supported is simple, and the audio decoder should not have significant intelligence to determine whether to use a bandwidth expansion blindly or parameter-guided bandwidth extension.

However, in another preferred embodiment, the audio decoder is configured to determine whether to use blind bandwidth extension or parameter-driven bandwidth extension based on the encoded representation of the low frequency portion without evaluating the bandwidth extension mode signaling flag. Thus, by providing intelligence in the audio decoder, the bandwidth extension mode signaling flag can be eliminated, which reduces the bit rate.

In a preferred embodiment, the audio decoder is configured to determine whether to use blind bandwidth expansion or parameter-guided bandwidth expansion based on one or more properties of the decoded representation of the low frequency portion (audio content). It has been found that the properties of the decoded representation of the low-frequency part are values that can be used with good accuracy in order to determine whether to use the bandwidth extension blindly or parameter-guided bandwidth extension. This is particularly true if the same properties are used on the audio encoder side. Accordingly, it is no longer necessary to evaluate the signaling flag of the bandwidth extension mode, which, in turn, makes it possible to reduce the bit rate, since it is not necessary to include the signaling flag of the bandwidth expansion mode in the encoded audio presentation on the audio encoder side.

In a preferred embodiment, the audio decoder is configured to determine whether to use blind bandwidth expansion or parameter-guided bandwidth expansion based on quantized linear prediction coefficients and / or statistics in the time domain of the decoded representation of the low frequency part (audio content). It has been found that quantized linear prediction coefficients can be easily obtained on the side of the audio decoder and by providing the ability to extract the spectral tilt, as a result, they can serve as a good indicator of whether to use the bandwidth extension blindly or parameter-guided bandwidth expansion. In addition, the quantized linear prediction coefficients are also readily available on the audio encoder side, so that switching between the blind bandwidth extension and the parameter-guided bandwidth extension on the audio encoder side and the audio decoder side can be easily coordinated. Similarly, it has been found that time-domain statistics of the decoded representation of the low-frequency part, such as the zero-crossing frequency, are a reliable value for determining whether to use blind bandwidth extension or parameter-guided bandwidth extension on the side of the audio decoder.

In a preferred embodiment, the bandwidth extension is configured to receive a bandwidth extension signal using one or more decoded representations of the low frequency part and / or using one or more low frequency decoder parameters for temporary parts of the input audio information (or content) for which the expansion parameters bandwidths are not included in encoded audio information. It has been found that such blind bandwidth expansion results in high sound quality.

In a preferred embodiment, the bandwidth extension is configured to receive a bandwidth extension signal using spectral centroid information and / or using energy information and / or using tilt information (spectral) and / or using encoded filter coefficients for the time parts input audio information (or content) for which bandwidth extension parameters are not included in the encoded audio information. It has been found that using these values results in an effective way to obtain bandwidth expansion with good quality.

In a preferred embodiment, the bandwidth extension is configured to receive a bandwidth extension signal using bitstream parameters describing the spectral envelope of the high frequency portion for the time portions of the audio content for which the bandwidth expansion parameters are included in the encoded audio information. It was found that the use of bit stream parameters describing the spectral envelope of the high-frequency part provides the possibility of efficient bit rate directed by the bandwidth expansion parameters with good quality, while the bit stream parameters describing the spectral envelope usually do not require a high bit rate, and can only encoded with a relatively small number of bits per audio frame. Therefore, even switching to parameter-driven bandwidth expansion does not significantly increase the bit rate.

In a preferred embodiment, the bandwidth extension is configured to evaluate from three to five bitstream parameters describing the intensities of the high frequency parts of the signal having a bandwidth of 300-500 Hz to obtain a bandwidth extension signal. It has been found that a relatively small number of bitstream parameters is sufficient to obtain bandwidth expansion in a perceptually important range, so that high sound quality can be obtained with a slight increase in bit rate.

In a preferred embodiment, from three to five bitstream parameters describing the intensity of the high frequency parts of the signal having a passband of 300-500 Hz are scalarly quantized with a resolution of 2 or 3 bits, so that 6-15 bits are provided for the parameters of forming the band extension spectrum bandwidth per audio frame. It was found that this choice provides a very high bit rate efficiency guided by the parameters of the bandwidth extension, while the quality of the bandwidth expansion is usually comparable to the quality of the bandwidth obtained using blind bandwidth expansion for "non-critical" parts of audio content, in which blindfold bandwidth expansion offers good results. Accordingly, a balanced quality is ensured both in the case that blind bandwidth expansion is applied, and in the case where the bandwidth expansion directed by the parameters is applied.

In a preferred embodiment, the bandwidth extension is configured to smooth out the energies of the bandwidth extension signal when switching from a blind bandwidth extension to a parameter-driven bandwidth extension and / or when switching from a parameterized bandwidth extension to a blind-bandwidth extension. Accordingly, clicks or “blocking artifacts” that may be triggered by various blind bandwidth expansion properties and channelized by bandwidth expansion parameters may not be allowed.

In a preferred embodiment, the bandwidth extension is configured to suppress the high frequency portion of the bandwidth extension signal for the portion of audio content to which the parameter-directed bandwidth extension is applied after the portion of the audio content to which the blindband extension is applied. In addition, the bandwidth extension is configured to reduce blanking for the high frequency portion of the bandwidth extension signal for the portion of audio content to which the bandwidth extension is applied blindly after the portion of the audio content to which the parameter-directed bandwidth extension is applied. Accordingly, to some extent, such an effect can be compensated that the expansion of the passband blindly usually shows the characteristic of the lower frequencies, while this does not necessarily occur for the bandwidth extension guided by the parameters. Accordingly, artifacts are reduced during transitions between portions of audio content decoded using blind bandwidth extension and using parameter-driven bandwidth expansion.

Another embodiment according to the invention provides a method for providing encoded audio information based on the input audio information. The method comprises encoding the low-frequency part of the input audio information in order to obtain an encoded representation of the low-frequency part. The method also comprises providing bandwidth extension information based on input audio information. Bandwidth extension information is selectively included in the encoded audio information in a signal adaptive manner. This method is based on considerations identical to those for the above audio encoder.

Another embodiment according to the invention provides a method for providing decoded audio information based on encoded audio information. The method comprises decoding the encoded representation of the low frequency part in order to obtain a decoded representation of the low frequency part. The method further comprises receiving a bandwidth extension signal using blindly bandwidth extension for parts of audio content for which bandwidth extension parameters are not included in the encoded audio information. The method further comprises receiving a bandwidth extension signal using the bandwidth expansion-guided parameters for parts of audio content for which the bandwidth expansion parameters are included in the encoded audio information. This method is based on considerations identical to those for the above audio decoder.

Another embodiment according to the invention creates a computer program for implementing one of the above methods when the computer program is running on a computer.

Another embodiment according to the invention creates an encoded audio presentation representing audio information. The encoded audio presentation comprises an encoded representation of the low frequency portion of the audio information and the bandwidth extension information. Bandwidth extension information is included in the encoded audio presentation in a signal-adaptive manner for some, but not all parts of the audio information. This encoded audio information is provided by the audio encoder described above, and can be estimated by the audio decoder described above.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention are described below with reference to the accompanying drawings, in which:

FIG. 1 shows a schematic block diagram of an audio encoder according to an embodiment of the present invention;

FIG. 2 shows a schematic block diagram of an audio encoder according to another embodiment of the present invention;

FIG. 3 shows a graphical representation of the frequency parts and encoded audio information associated with them;

FIG. 4 shows a schematic block diagram of an audio decoder according to an embodiment of the present invention;

FIG. 5 shows a schematic block diagram of an audio decoder according to another embodiment of the present invention;

FIG. 6 shows a flowchart of a method for providing an encoded audio presentation according to an embodiment of the present invention;

FIG. 7 shows a flowchart of a method for providing a decoded audio presentation according to an embodiment of the present invention;

FIG. 8 shows a schematic illustration of an encoded audio presentation according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

1. The audio encoder of FIG. one

FIG. 1 shows a schematic block diagram of an audio encoder according to an embodiment of the present invention.

The audio encoder 100 of FIG. 1 receives the input audio information 110 and provides, based on its encoded audio information 112. The audio encoder 100 comprises a low-frequency encoder 120, which is configured to encode the low-frequency part of the input audio information 110 so as to obtain an encoded representation of the low-frequency part 122. The audio encoder 100 also includes a bandwidth extension information providing unit 130 adapted to provide bandwidth extension information 132 based on the input audio information 110. The audio encoder 100 is configured to selectively include the bandwidth extension information 132 in the signal-adaptive encoded audio information 112.

Regarding the functionality of the audio encoder 100, it can be said that the audio encoder 100 provides bit-rate coding of the input audio information 110. The low-frequency part, for example, in the frequency range up to about 6 or 7 kHz, is encoded using a low-frequency encoder 120, in which any of well-known principles of audio coding. For example, the low-frequency encoder 120 may be a “common audio encoder” (such as, for example, an AAC audio encoder) or a speech audio encoder (such as, for example, an audio encoder based on linear prediction, CELP audio encoder, ACELP audio encoder, etc.) . Accordingly, the low-frequency part of the input audio information is encoded using any of the traditional principles. However, the bit rate of the encoded representation 122 of the low-frequency part is reasonably low, since only the frequency components are encoded up to about 6-7 kHz. In addition, the audio encoder 100 allows the provision of bandwidth extension information, for example, in the form of bandwidth extension parameters describing the high-frequency portion of the input audio information 110, such as, for example, a frequency domain containing higher frequencies than the frequency domain encoded by a low-frequency encoder 120. Thus, the bandwidth extension information providing unit 130 allows for providing auxiliary information of the encoded audio information 112, which can control the bandwidth extension performed on the side of an audio decoder not shown in FIG. 1. The bandwidth extension information (or auxiliary bandwidth extension information), for example, can represent the spectral shape (or spectral envelope) of the high-frequency part of the input audio information, i.e. the frequency range of the input audio information that is not covered by the low-frequency encoder 120.

However, the audio encoder 100 is configured to determine, in a signal-adaptive manner, whether bandwidth extension information should be included in the encoded audio information 112. Accordingly, the audio encoder 100 allows the inclusion of bandwidth extension information in the encoded audio information 112 only if if bandwidth extension information is required (or at least desirable) to restore audio information on the side of the audio decoder. In this context, the audio encoder can also control whether to provide bandwidth extension information 132 through the bandwidth extension information providing unit 130 for a portion of the input audio information (or, equivalently, for a portion of the encoded audio information), since, of course, it is not necessary to provide bandwidth information bandwidth for part of the input audio information (or encoded audio information) if bandwidth extension information should not be included in the encoded th audio information. Accordingly, the audio encoder 100 allows the bit rate for the encoded audio information 112 to be kept as low as possible by not allowing the inclusion of bandwidth extension information 132 in the encoded audio information 112, if detected, based on some analysis process and / or decision process performed by the audio encoder 100, then that bandwidth extension information is not required to obtain a certain sound quality when restoring the corresponding part of the audio content n on the side of the audio decoder.

Thus, the audio encoder 100 includes bandwidth extension information in the encoded audio information only if it is required (in order to obtain a certain sound quality) on the side of the audio decoder, which, on the one hand, helps to reduce the bit rate for the encoded audio information 112, and on the other hand, ensures that the proper bandwidth extension information 132 is included in the encoded audio information 112 if it is required in order to prevent poor sound quality When the decoding of the encoded audio data to the audio decoder side. Thus, an improved trade-off between bit rate and sound quality is achieved by the audio encoder 100 compared to traditional solutions.

For example, an audio decoder may determine, based on an audio frame, whether bandwidth extension information should be included in encoded audio information 112 (or even whether bandwidth extension information should be determined). However, as an alternative, the audio decoder may determine, based on “input” (for example, based on an audio file or based on an audio stream), whether bandwidth extension information should be included in encoded audio information 112. To this end, input can be analyzed (for example, before coding), so that the decision is made in a signal-adaptive way.

2. The audio encoder of FIG. 2

FIG. 2 shows a schematic block diagram of an audio encoder according to an embodiment of the present invention. The audio encoder 200 receives the input audio information 210 and provides, based on its encoded audio information 212. The audio encoder 200 comprises a low-frequency encoder 220, which may be substantially identical to the low-frequency encoder 120 described above. The low-frequency encoder 220 provides an encoded representation 222 of the low-frequency part of the input audio information (or, equivalently, the audio content represented by the input audio information 210). The audio encoder 200 also includes a bandwidth extension information providing unit 230, which may be substantially identical to the bandwidth extension information providing unit 130 described above. The bandwidth extension information providing unit 230 typically receives input audio information 210. However, the bandwidth extension information providing unit 230 can also receive control information (or intermediate information) from the low-frequency encoder 220, wherein said control information (or intermediate information), for example, it may contain information regarding the spectrum (spectral shape or spectral envelope) of the low-frequency part of the input audio information 210. However, I control Single information (or intermediate information) may also include encoding parameters (e.g., LPC-filter coefficients either in the transform domain values, such as coefficients or MDCT-QMF-factors), etc. In addition, the bandwidth extension information providing unit 230 may optionally receive an encoded representation 222 of the low frequency part or at least a part thereof. In addition, the audio encoder 200 includes a detector 240, which is configured to determine whether bandwidth extension information is included in the encoded audio information 212 for a given part of the input audio information 210 (or for a given part of the encoded audio information 212). If necessary, the detector 240 may also determine whether said bandwidth extension information is determined by the bandwidth extension information unit 230 for said given portion of the input audio information 210 (or encoded audio information 212). Therefore, the detector 240 may receive input audio information 210 and / or control information or intermediate information 224 from the low-frequency encoder 220 (for example, as described above) and / or the encoded representation 222 of the low-frequency part. In addition, the detector 240 is configured to provide a control signal 242 that controls the selective provision of bandwidth extension information and / or the selective inclusion of bandwidth extension information in encoded audio information 212.

Regarding the functionality of the audio encoder 200, refer to the above explanations given with respect to the audio encoder 100.

In addition, it should be noted that the detector 240 plays a central role, because the detector 240 determines whether the bandwidth extension information is included in the encoded audio information 212, and therefore determines whether the audio decoder that receives the encoded audio information 212 restores the audio content that is described through the input audio information 210, using blindly expanding the bandwidth or using the parameterized bandwidth extension ( bandwidth broadening represents parameters directing the parameter-guided bandwidth expansion).

Generally speaking, the detector identifies portions of the input audio information that cannot be decoded with sufficient or desired quality based on the encoded representation of the low-frequency portion 222 using the blind bandwidth extension. In other words, the detector 240 must recognize when the encoded representation of the low-frequency part 222 alone does not provide the possibility of expanding the bandwidth blindly with sufficient quality. In other words, the detector 240 preferably identifies portions of the input audio information for which bandwidth extension parameters cannot be estimated based on the low frequency portion with sufficient (or desired) accuracy to achieve acceptable (or desired) sound quality. Therefore, the detector 240 can determine, using the control signal 242, that the bandwidth extension information should be included in the encoded audio information for parts of the input audio information, which cannot be decoded with sufficient or desired quality based on the encoded representation of the low-frequency part 222 using the extension blind bandwidth (i.e., without receiving bandwidth extension information from an encoder). Equivalently, the detector can determine, using the control signal 242, that the bandwidth extension information should be included in the encoded audio information for parts of the input audio information for which the bandwidth expansion parameters cannot be estimated based on the low frequency part (or, equivalently, the encoded representation 222 low-frequency parts) with sufficient or desired accuracy.

To identify portions for which bandwidth extension information should be included in the encoded audio information (or, equivalently, to identify portions of the input audio information for which bandwidth extension information may be included in the encoded audio information 212 if necessary), the detector 240 may use various strategies . As mentioned above, detector 240 may receive various types of input information. In some cases, determining by the detector whether bandwidth extension information should be included in the encoded audio information 212 may be based solely on the input audio information 210. In other words, the detector 240, for example, may be configured to analyze the input audio information 210, s in order to identify for which part of the input audio information (which corresponds to parts of the encoded audio information 212) it is necessary to include information 232 bandwidth extension in the code encoded audio information 212 in order to achieve acceptable (or desired) sound quality. However, the determination by the detector 240 may alternatively be based on some control information or intermediate information 224 provided by the low-frequency encoder 200. Alternatively, or in addition, the determination by the detector 240 may be based on the encoded representation 222 of the low-frequency portion of the input audio information 210. Thus, the detector can evaluate various quantities to determine (or evaluate) whether the bandwidth extension should be blindly the audio decoder and the side lead to sufficient sound quality (or is likely to lead to a sufficient audio quality, or suspected to result in adequate quality of sound).

For example, the detector may determine whether the parts of the input audio information 210 are parts that are motionless in time, and whether the parts of the input audio information 210 are of a low frequency nature. For example, detector 240 may conclude that it is not necessary to include bandwidth extension information in encoded audio information 212 for parts that are found to be parts that are motionless in time and that are of a low frequency nature, as it should be recognized that such parts of the input audio information 210 can usually be reproduced with sufficiently high sound quality on the side of the audio decoder even when using the blind bandwidth extension. This is due to this fact that blind bandwidth expansion usually works well for parts of input audio information (or content) that do not contain strong changes in audio content (or that do not contain transition parts or other strong variations in audio content) and therefore can be considered fixed over time . In addition, it was found that blind bandwidth expansion works well for parts of audio content that are of a low frequency nature, i.e. for the part of audio content for which the intensity of the low-frequency part is higher than the intensity of the high-frequency part, as this is a fundamental assumption with respect to most principles of blind bandwidth expansion. Accordingly, the detector 240 can signal using the control signal 242 the need to selectively exclude the inclusion of bandwidth extension information in the encoded audio information 212 for such time-stationary parts having a low frequency character.

For example, detector 240 may be configured to identify portions of input audio information that contain voiced speech, and / or portions of input audio information that contain environmental noise, and / or portions of input audio information that contain music without percussion instruments. Such parts of the input audio information are usually time-constant and have a low-frequency character, so that the detector 240 usually signals the need to exclude the inclusion of bandwidth extension information in the encoded audio information for such parts.

As an alternative or addition, detector 240 can analyze whether the spectral shape in the high-frequency part of the input audio information can be predicted with reasonable accuracy (for example, using principles applied by expanding the blind bandwidth) based on the spectral envelope of the low-frequency part. Accordingly, the detector, for example, can be configured to determine whether the difference between the spectral envelope of the low-frequency part (which can be described, for example, by intermediate information 224 or the encoded representation of the low-frequency part 222) and the spectral envelope of the high-frequency part ( which, for example, can be determined by a detector 240 based on the input audio information 210) to a predetermined difference index. For example, the detector 240 may determine the difference in terms of the difference in intensity or in terms of the difference in shape, or from the point of view of variation in frequency, or from the point of view of any other distinguishing features of the spectral envelopes. Accordingly, the detector 240 can determine (and signal) the need to include bandwidth extension information 232 in the input audio information, in response to such detected information that the difference between the spectral envelope of the low-frequency part and the spectral envelope of the high-frequency part is greater than or equal to a predetermined difference index. In other words, the detector 240 can determine how well the spectral envelope of the high-frequency part can be predicted based on the spectral envelope of the low-frequency part, and if prediction is impossible with good results (which, for example, occurs if the predicted spectral envelope of the high-frequency part is too significant differs from the actual spectral envelope of the high-frequency part), we can conclude that the bandwidth extension information 232 should require atsya on the side of the audio decoder. However, instead of comparing the predicted spectral envelope of the high-frequency part with the actual spectral envelope of the high-frequency part, the detector 240, as an alternative, can compare the spectral envelope of the low-frequency part with the spectral envelope of the high-frequency part. This is useful if it is assumed that the spectral envelope of the high-frequency part is usually the same as the spectral envelope of the low-frequency part when applying the blind bandwidth estimate.

As an alternative or addition, detector 240 may identify parts containing unvoiced speech and / or parts containing percussion instrument sounds. Since the spectral envelope of the high-frequency part is usually very different from the spectral envelope of the low-frequency part in such cases, the detector may signal the need to include bandwidth extension information in the encoded audio representation for such parts of the input audio information (or encoded audio information) containing unvoiced speech or containing the sounds of percussion instruments.

However, as an alternative or addition, the detector 240 can analyze the spectral tilt of the parts of the input audio information 210. In addition, the detector 240 can use the information regarding the spectral tilt of the parts of the input audio information to determine whether bandwidth extension information 232 should be turned on encoded audio information 212. This principle is based on the idea that blind bandwidth expansion works well for parts of audio content for which there is more I have the energy (or, in general, the intensity) in the low-frequency range compared to the high frequency band. On the contrary, if the high-frequency part (also referred to as the high-frequency range) is “dominant”, i.e. contains a significant amount of energy, blindband expansion usually cannot reproduce audio content well, so bandwidth extension information should be included in encoded audio information. Accordingly, in some embodiments, the detector determines whether the spectral tilt (which describes the distribution of energies or, in general, frequency intensity) is greater than or equal to a fixed or variable threshold tilt value. If the spectral tilt is greater than or equal to a fixed or variable threshold tilt (which means that there is a relatively high energy or intensity in the high-frequency part of the audio content, at least compared to the “normal” case, in which the energy or intensity decreases with increasing frequency), the detector may determine the need to include bandwidth extension information in encoded audio information.

In addition to some or all of the above features, the detector can also estimate the frequency of transitions through zero parts of the input audio information. In addition, determining by a detector whether to include bandwidth extension information can also be based on whether a particular zero-crossing frequency is greater than or equal to a fixed or variable threshold zero-crossing frequency. This principle is based on the idea that a high zero-crossover frequency usually indicates that high frequencies play an important role in the input audio information, which in turn indicates that the parameter-driven bandwidth extension should be used on the audio decoder side.

In addition, it should be noted that the detector 240 can preferably use some hysteresis in order to prevent excessive switching between including bandwidth extension information 232 in the encoded audio information and excluding said inclusion. For example, hysteresis can be applied to a variable threshold slope threshold, to a variable threshold threshold value of zero transitions, or to any other threshold value that is used to determine the transition from enabling bandwidth extension information to avoiding said inclusion, or vice versa. Thus, the hysteresis can vary the threshold value in order to reduce the likelihood of switching to exclude the inclusion of bandwidth extension information when the bandwidth extension information is turned on for the current part of the input audio information. Similarly, the threshold value may be varied to reduce the likelihood of switching to turning on bandwidth extension information when bandwidth extension information is not allowed for the current part of the input audio information. Thus, artifacts that can be triggered by transitions between different modes can be reduced.

Next, some details are explained regarding the bandwidth extension information providing unit 230. In particular, what information is included in the encoded audio information 212 in response to a signaling by the detector that bandwidth extension information 232 should be included in the encoded audio information is explained. For purposes of explanation, reference should also be made to FIG. 3, which shows a schematic representation of the frequency parts of the input audio information and the parameters included in the encoded audio presentation. Abscissa 310 describes the frequency, and ordinate 312 describes the intensity (for example, intensity, for example, amplitude or energy) of various spectral elements of the sample (such as, for example, MDCT coefficients, QMF coefficients, FFT coefficients, etc.). As you can see, the low-frequency part of the input audio information, for example, can cover the frequency range from the lower-frequency boundary (for example, 0 or 50 Hz, or 300 Hz, or any other reasonable lower-frequency boundary) up to a frequency of approximately 6.4 kHz. As you can see, encoded representation 222 can be provided for this low-frequency part (for example, from 300 Hz to 6.4 kHz, etc.). In addition, a high-frequency part is provided, which, for example, ranges from 6.4 kHz to 8 kHz. However, the high-frequency part, of course, can cover a different frequency range, which is usually limited by the frequency range perceived by the human listener. However, in FIG. 3, it can be seen that, by way of example, the spectral envelope shown by reference numeral 320 contains an irregular shape in the high frequency portion. In addition, it can be seen that the spectral envelope 320 contains a relatively large energy in the high-frequency part and even a relatively high energy of 7.2-7.6 kHz. For comparison, the second spectral envelope 330 is also shown in FIG. 3, the second spectral envelope 330 shows the attenuation of the intensity or energy (for example, based on the unit frequency) in the high-frequency part. Accordingly, spectral envelope 320 typically should cause the detector to determine whether to include bandwidth extension information in the encoded audio representation for the portion containing spectral envelope 320, while spectral envelope 330 should usually cause the detector to determine whether to exclude bandwidth information for a portion of audio content containing a spectral envelope 330.

As can also be seen, for the portion of the audio content containing the spectral envelope 320, four scalar parameters should be included in the encoded audio presentation as bandwidth extension information. The first scalar parameter, for example, can describe the spectral envelope (or average spectral envelope) for the frequency domain of 6.4-6.8 kHz, the second scalar parameter can describe the spectral envelope (or its average) for the frequency domain of 6.8- 7.2 kHz, the third scalar parameter can describe the spectral envelope 320 (or its average) for the frequency domain of 7.2-7.6 kHz, and the fourth scalar parameter can describe the spectral envelope (or its average) for the frequency domain of 7, 6-8 kHz. Scalar parameters can describe the spectral envelope in an absolute or relative way, for example, with respect to the spectrally previous frequency range (or region). For example, the first scalar parameter can describe the intensity ratio (which, for example, can be normalized as a value) between the spectral envelope in the frequency domain of 6.4-6.8 kHz and the spectral envelope in the lower frequency region (for example, below 6, 4 kHz). The second, third and fourth scalar parameters, for example, can describe the difference (or ratio) between the (intensities) of the spectral envelope in adjacent frequency ranges, so that, for example, the second scalar parameter can describe the relationship between the (average value) of the spectral envelope in the frequency range in 6.8-7.2 kHz and a spectral envelope in the frequency range of 6.4-6.8 kHz.

In addition, it should be noted that the encoded representation of the low-frequency part, i.e. the frequency part below 6.4 kHz can be turned on anyway. The frequency part below 6.4 kHz (the low frequency part) can be encoded using any of the known encoding principles, for example, using “common” encoding of audio data, such as AAC (or its derivative), or speech encoding (such as, for example, CELP ACELP or its derivative). Accordingly, for the part of the audio content containing the spectral envelope 320, both the encoded representation of the low-frequency part and the four scalar bandwidth expansion parameters (which can be quantized using a relatively small number of bits) should be included in the encoded audio representation. In contrast, for a portion of audio content containing a spectral envelope 330, only the encoded representation of the low-frequency portion should be included in the encoded audio presentation, and the (scalar) bandwidth expansion parameters should not be included in the encoded audio presentation (which, however, does not cause serious problems, since the spectral envelope 330 exhibits a regular damped (low-frequency) response that can be well reproduced using the bandwidth extension blindly).

As a conclusion, the audio encoder 200 is configured to selectively include parameters representing the spectral envelope of the high-frequency part of the input audio information in the encoded audio information in a signal-adaptive manner as bandwidth extension information. For example, scalar bandwidth expansion parameters mentioned with reference to FIG. 3 may be included in the encoded audio information in a signal adaptive manner. Generally speaking, the low-frequency encoder 220 may be configured to encode the low-frequency part of the input audio information 210 containing frequencies up to a maximum frequency that is in the range of 6-7 kHz (the 6.4 kHz boundary used in the example of Fig. 3) . In addition, the audio encoder may be configured to selectively include from three to five parameters describing the intensity of the high-frequency parts of the signal having a passband of 300-500 Hz in the encoded audio presentation. In the example of FIG. 3, four scalar parameters are shown describing the intensity of the high-frequency parts of the signal having passbands of approximately 400 Hz. In other words, the audio encoder can be configured to include four scalar-quantized parameters describing the intensity of the four high-frequency parts of the signal in the encoded audio representation, the high-frequency parts of the signal covering the frequency ranges (for example, as shown in Fig. 3) above the low-frequency part (for example, as explained with reference to Fig. 3). For example, an audio encoder may be configured to selectively include in a coded audio presentation a plurality of parameters describing the relationship between the energies or intensities of spectrally adjacent frequency parts, one of the parameters describing the relationship between the energy or intensity of the first high-frequency part of the bandwidth extension and the energy or intensity of the low-frequency part , while other parameters describe the relationship between the energies or intensities of other high frequencies parts of the expansion of the bandwidth (in this case, the high-frequency parts of the expansion of the bandwidth can be the frequency parts of 6.4-6.8 kHz, 6.8-7.2 kHz, 7.2-7.6 kHz and 7 , 6-8 kHz). Alternatively, three to five envelope shape parameters (describing the intensities of the high-frequency parts of the signal) can be vector quantized. Vector quantization is usually a little more efficient than scalar quantization. Vector quantization, on the other hand, is more complex than scalar quantization. In other words, the quantization of four values of the bandwidth extension energy can alternatively be performed using vector quantization (instead of using scalar quantization).

As a conclusion, the audio encoder can be configured to include relatively simple bandwidth extension information in the encoded audio presentation, so that the bit rate of the encoded audio presentation only increases slightly for parts of the input audio information (or encoded audio presentation) for which what is desired is detected by the detector bandwidth-driven parameter expansion.

3. The audio decoder according to FIG. four

FIG. 4 shows a schematic block diagram of an audio decoder according to an embodiment of the present invention. The audio decoder 400 of FIG. 4 receives encoded audio information 410 (which, for example, can be provided by an audio encoder 100 or by an audio encoder 200) and provides based on its decoded audio information 412.

The audio decoder 400 comprises a low-frequency decoder 420, which receives encoded audio information 410 (or at least an encoded representation of the low-frequency part included therein), decodes the encoded representation of the low-frequency part, and obtains the decoded representation of the low-frequency part 422. The audio decoder 400 also comprises a bandwidth extension 430 that is configured to receive a bandwidth extension signal 432 using blind bandwidth expansion for portions of (encoded) audio content (represented by encoded audio information 410) for which bandwidth expansion parameters are not included in the encoded audio information 410, and receives a bandwidth extension signal 432 using the parameterized bandwidth extension (with general bandwidth extension information or bandwidth expansion parameters included in encoded audio information 410) for parts of audio content for which bandwidth expansion parameters are included in encoded audio information 410 (or encoded audio presentation).

Accordingly, the audio decoder 400 allows bandwidth expansion to be performed regardless of whether the bandwidth expansion parameters are included in the encoded audio information 410. Thus, the audio decoder can adapt to the encoded audio information 410 and provides a principle in which switching between blind bandwidth extension and guided parameters occurs bandwidth extension. Therefore, the audio decoder 400 allows processing of encoded audio information 410, in which bandwidth extension parameters are included only for portions (eg, frames) of audio content that cannot be reconstructed with sufficient quality using the blindband extension. Thus, decoded audio information 412 can be provided that contains both a decoded representation of the low-frequency part and a bandwidth extension signal (in this case, a signal, for example, can be added to the decoded representation of the low-frequency part 422 to thereby obtain decoded audio information 412).

Thus, the audio decoder 400 helps to obtain a good compromise between sound quality and bit rate.

An additional optional enhancement to the audio decoder 400 is described below, for example with reference to FIG. 5.

4. The audio decoder of FIG. 5

FIG. 5 shows a schematic block diagram of an audio decoder 500, according to another embodiment of the present invention. The audio decoder 500 receives the encoded audio information 510 (also referred to as an encoded audio presentation) and provides based on its decoded audio information 512 (also referred to as a decoded audio presentation). The audio decoder 500 comprises a low-frequency decoder 520, which may be identical to the low-frequency decoder 420 and may provide comparable functionality. Thus, the low-frequency decoder 500 provides a decoded representation 522 of the low-frequency portion of the audio content represented by the encoded audio information 510. The audio decoder 500 also includes a bandwidth extension 530, which can implement functionality identical to that of the bandwidth extension 430.

Therefore, the bandwidth extension 530 may provide a bandwidth extension signal 532, which is usually combined (eg, added) with the decoded representation of the low-frequency part 522 to thereby obtain decoded audio information 512. The bandwidth extension 530, for example, may receive the decoded representation 522 low-frequency part 522. However, as an alternative, the expansion of bandwidth 532 may receive control information 524 (which also considers I, as auxiliary information or intermediate information), which is provided by the low-frequency decoder 520. The auxiliary information or control information or intermediate information 524, for example, can represent the spectral shape of the low-frequency part of the audio content, the frequency of zero transitions of the decoded representation of the low-frequency part, or any other intermediate value used by low-frequency decoder 520, which is useful in the process of expanding the bandwidth lowering. In addition, the audio decoder comprises a controller 540, which is configured to provide control information 542 indicating whether blind bandwidth expansion or parameter-guided bandwidth expansion should be performed by bandwidth extension 530. Controller 540 may use various types of information to provide control information 542. For example, controller 540 may receive a bandwidth extension bitstream flag, which may be included in encoded audio information 510. For example, one bandwidth expansion bitstream flag may be provided for each part (for example, a frame) of encoded audio information that can be extracted from encoded audio information by the controller 540 and which can be used to th to extract management information 542 (or which can directly be control information 542). However, as an alternative, the controller 540 may receive information that represents the low frequency part and / or which describes how to decode the low frequency part (and which is therefore also referred to as “low frequency decoding information”). Alternatively or in addition, the controller 540 may receive control information or auxiliary information, or intermediate information 524 from a low-frequency decoder, which, for example, can carry information about the spectral envelope of the low-frequency part and / or information regarding the frequency of transitions through zero of the decoded representation of the low-frequency part. However, the control information or auxiliary information or intermediate information 524 may also carry information regarding the statistics of the decoded representation of the low-frequency part 522 or may represent any other intermediate information that is extracted by the low-frequency decoder 520 from the encoded representation of the low-frequency part (also referred to as decoding information low frequency part).

As an alternative or addition, the controller 540 may receive the decoded representation of the low-frequency part 522 and can directly extract property values (e.g., zero-crossing frequency information, spectral envelope information, spectral tilt information, etc.) from the decoded low-frequency part representation 522.

Accordingly, the controller 540 may evaluate the bitstream flag to provide control information 542 regarding the blind / guided mode parameters if such a bitstream flag (indicating whether blind bandwidth expansion or parameter-guided bandwidth extension is to be used) is included in the encoded audio information 510 However, if such a bitstream flag is not included in the encoded audio information 510 (for example, to reduce the bit rate), the controller 540 o Usually determines whether to use bandwidth expansion blindly or parameter-guided bandwidth expansion, based on other information. To this end, the decoding information of the low-frequency part (which may be identical to the encoded representation of the low-frequency part or its subset) can be evaluated by the controller 540. As an alternative or addition, the controller may consider the decoded representation 522 of the low-frequency part to decide whether to use bandwidth expansion blindly or parameter-guided bandwidth expansion, i.e. to provide control information 542. In addition, the controller 540 may optionally use control information or auxiliary information or intermediate information 524 provided by the low-frequency decoder 520, provided that the low-frequency decoder 520 provides any intermediate values that are applicable by the controller 540.

Accordingly, the controller 540 may switch the bandwidth extension between the blind bandwidth extension and the parameter-guided bandwidth extension.

In the case of a blindfold bandwidth extension, the bandwidth extension 530 may provide a bandwidth extension signal 532 based on the decoded representation of the low frequency part 522 without evaluating additional bitstream parameters. In contrast, in the case of bandwidth expansion-driven parameters, bandwidth extension 530 may provide a bandwidth extension signal 532, taking into account additional (allocated) bandwidth expansion bitstream parameters that help determine the characteristics of the high-frequency portion of the audio content (i.e., the characteristics of the bandwidth extension signal transmission). However, the bandwidth extension 530 can also use the decoded representation 522 of the low-frequency part and / or control information or auxiliary information or intermediate information 524 provided by the low-frequency decoder 520 to provide a bandwidth extension signal 532.

Thus, the decision between using the blind bandwidth extension and the parameter-guided bandwidth extension effectively determines whether the selected bandwidth extension parameters (which are not typically used by the low-frequency decoder 520 to provide a decoded representation of the low-frequency part) are used to receive a bandwidth extension signal (which typically describes the high-frequency portion of the audio content presented by m coded audio information).

To summarize the above, the audio decoder 500 can be configured to determine whether to receive a bandwidth extension signal 532 using a blindband extension or using a parameterized bandwidth extension, on a frame-by-frame basis (the “frame” is an example of part of the audio content, and the frame, for example, may contain a duration between 10 ms and 40 ms and preferably may have a duration of approximately 20 ms ± 2 ms). Thus, the audio decoder can be configured to switch between blindly expanding bandwidth and parameter-guided bandwidth expansion with very accurate time detail.

In addition, it should be noted that the audio decoder 500 typically allows switching between using blind bandwidth extension and parameter-guided bandwidth expansion within an adjacent piece of audio content. Thus, switching between blindly expanding bandwidth and parameterized bandwidth expansion can be performed at almost any time (of course, taking into account framing) within an adjacent piece of audio content in order to adapt the bandwidth extension to the (changing) characteristics of different parts of one fragment of audio content.

As mentioned above, an audio decoder (preferably, controller 540) may be configured to evaluate flags (e.g., one single-bit flag per frame) included in encoded audio information 510 for different parts (e.g., frames) of audio content to determine if Use bandwidth extension blindly or parameter-guided bandwidth extension. In this case, the controller 540 can be kept very simple due to the fact that the alarm flag must be included in the encoded audio information for each piece of audio content. However, as an alternative, the controller 540 may be configured to determine whether to use a blindband extension or parameter-guided bandwidth extension based on an encoded representation of the low frequency portion (which may include the use of control information or auxiliary information, or intermediate information 524 extracted by a low-frequency decoder 520 from said encoded representation of the low-frequency part, and which may also include using the decoded representation 522, which is extracted from the encoded representation of the low frequency part by the low frequency decoder 520), without evaluating the (allocated) bandwidth extension mode signaling flag. Thus, switching between blindly bandwidth extension and parameter-guided bandwidth extension can be performed even without additional overhead in the bitstream.

An audio decoder (or controller 540) may be configured to determine whether to use blind bandwidth expansion or parameter-guided bandwidth expansion based on one or more properties of the decoded representation of the low frequency portion. Properties such as, for example, spectral tilt information, zero-crossing frequency information, and the like, can either be extracted from the decoded representation 522 of the low-frequency part, or can be signaled by control information / auxiliary information / intermediate information 524. For example, an audio decoder ( or controller 540) may be configured to determine whether to use blind bandwidth expansion or parameter-guided bandwidth expansion based on quantized linear prediction coefficients (which, for example, can be included in the control information / auxiliary information / intermediate information 524) and / or depending on the statistics in the time domain of the decoded representation 522 of the low-frequency part.

The following describes some principles regarding how to achieve bandwidth expansion. For example, bandwidth extension may be configured to receive a bandwidth extension signal 532 using one or more properties of the decoded representation of the low frequency part 522 and / or one or more parameters of the low frequency decoder 520 (which may be signaled by control information / auxiliary information / intermediate information 524) for temporary parts of (input) audio content for which bandwidth extension parameters are not included in the encoded au ioinformatsiyu. Thus, the bandwidth extension 530 can perform blind bandwidth expansion, which is based on the idea of drawing conclusions from a decoded representation of the low-frequency part in the high-frequency part of the audio content represented by the encoded audio information. For example, bandwidth extension 530 may be configured to receive a bandwidth extension signal 532 using spectral centroid information and / or using energy information and / or using (eg, coded) filtering coefficients for the time portions of the input audio content for which bandwidth expansion parameters are not included in encoded audio information 510. Accordingly, good blindband expansion can be achieved.

However, of course, other principles for blindly expanding bandwidth may also apply.

However, the bandwidth extension may be configured to receive a bandwidth extension signal 532 using bitstream parameters describing the spectral envelope of the high frequency portion for the time portions of the audio content for which the bandwidth expansion parameters are included in the encoded audio information. In other words, the parameter-driven bandwidth extension can be performed using bitstream parameters describing the spectral envelope of the high-frequency part. The bitstream parameters describing the spectral envelope of the high-frequency part can support parameter-guided bandwidth expansion (which, however, can additionally be based on some or all of the values used by blindly extending the bandwidth).

For example, it was found that the bandwidth extension should preferably be configured to evaluate from three to five bitstream parameters describing the intensity of the high frequency parts of the signal having a bandwidth of 300-500 Hz to obtain a bandwidth extension signal. The use of such a relatively small number of bitstream parameters does not significantly increase the bit rate, but it contributes to a sufficient improvement in terms of bandwidth expansion in the case of “hard” parts of the signal, so that the quality obtained by such directional bandwidth expansion for “hard” parts of the signal is comparable to the quality obtained for the “simple” parts of the signal using the blind bandwidth extension (while the “difficult” parts of the signal are parts of a signal for which blind bandwidth expansion does not lead to good or acceptable sound quality, while “simple” signal parts are parts of a signal for which blind bandwidth expansion provides sufficient results).

Accordingly, it is preferable that from three to five parameters of the bitstream describing the intensity of the high-frequency parts of the signal having a bandwidth of 300-500 Hz are scalarly quantized with a resolution of two or three bits, so that 6-15 bits are provided for the parameters of the formation of the expansion spectrum bandwidth per frame. It was found that such a low bit rate of the bandwidth extension information is already sufficient to obtain a sufficiently good bandwidth extension in the case of “difficult” parts of the audio content.

If necessary, the bandwidth extension 530 can be configured to smooth out the energies of the bandwidth extension signal when switching from blindly expanding the bandwidth to the parameter-driven bandwidth extension and / or when switching from the bandwidth-expanding parameter to the blindly bandwidth extension. Accordingly, continuity disturbances in spectral form are reduced when switching between blind bandwidth expansion and parameterized bandwidth expansion. For example, the bandwidth extension may be configured to suppress the high frequency portion of the bandwidth extension signal for the portion of audio content to which the parameter-directed bandwidth extension is applied after the portion of the audio content to which the blindband extension is applied. In addition, the bandwidth extension can be configured to reduce blanking for the high-frequency part of the bandwidth extension signal (i.e., to emphasize the high-frequency part of the bandwidth extension signal to a certain extent) for the part of the audio content to which the bandwidth expansion is applied blindly, after the part audio content to which the parameter-driven bandwidth extension is applied. However, smoothing can also be performed by any other operation that reduces the disruption of the spectral shape of the high-frequency part when switching between bandwidth extension modes. Thus, sound quality is improved by reducing artifacts.

As a conclusion, the audio decoder 500 provides the ability to decode audio content with good quality both in case bandwidth extension information is provided in the encoded audio information, and in the case where bandwidth extension information is not provided in the encoded audio information. An audio decoder can switch between blindfold bandwidth extension and parameter-guided bandwidth extension with precise time detail (for example, frame-by-frame), while artifacts are kept small.

5. A method for providing encoded audio information based on input audio information according to FIG. 6

FIG. 6 shows a flow diagram of a method 600 for providing encoded audio information based on input audio information. The method 600 comprises encoding 610 the low-frequency part of the input audio information in order to obtain an encoded representation of the low-frequency part. The method 600 also comprises providing 620 bandwidth extension information based on input audio information, wherein the bandwidth extension information is selectively included in the encoded audio information in a signal-adaptive manner.

It should be noted that the method 600 of FIG. 6 may be supplemented by any of the features and functionalities described herein with respect to the audio encoder (as well as with respect to the audio decoder).

6. A method for providing decoded audio information according to FIG. 7

FIG. 7 shows a flowchart of a method for providing decoded audio information according to an embodiment of the invention. Method 700 comprises decoding 710 an encoded representation of a low frequency part in order to obtain a decoded representation of a low frequency part. The method 700 also comprises receiving a 720 bandwidth extension signal using blindly bandwidth extension for parts of audio content for which bandwidth extension parameters are not included in the encoded audio information. In addition, method 700 comprises obtaining 730 a bandwidth extension signal using bandwidth expansion-guided parameters for parts of audio content for which bandwidth expansion parameters are included in encoded audio information.

It should be noted that the method 700 of FIG. 7 may be supplemented by any of the features and functionalities described herein with respect to an audio decoder (as well as with respect to an audio encoder).

7. The encoded audio presentation of FIG. 8

FIG. 8 shows a schematic illustration of an encoded audio presentation 800 representing audio information.

The encoded audio presentation (also referred to as encoded audio information) contains an encoded representation of the low-frequency portion of the audio information. For example, an encoded representation 810 of the low-frequency part of the audio information is provided for the first part of the audio information, for example, for the first frame of the audio information. In addition, an encoded representation of the low-frequency part of the audio information is also provided for the second part (for example, the second frame) of the audio information. However, the encoded audio presentation 800 also contains bandwidth extension information, and bandwidth extension information is included in the encoded audio presentation in a signal-adaptive manner for some, but not all parts of the audio information. For example, bandwidth extension information 812 is included for a first piece of audio information. In contrast, bandwidth extension information is not provided for the second part of the audio information.

As an output, encoded audio presentation 800 is typically provided by the audio encoders described herein, and evaluated by the audio decoders described herein. Of course, the encoded audio presentation can be stored on non-volatile computer-readable media, etc. In addition, it should be noted that the encoded audio presentation 800 may be supplemented by any of the features, information elements, etc. described with respect to the audio encoder and audio decoder.

8. Conclusions and additional aspects

Embodiments of the present invention solve the problems of traditional bandwidth expansion when encoding audio at very low bit rates and the disadvantages of existing, traditional bandwidth expansion technologies by offering a “minimally directional” bandwidth extension as a signal adaptive combination of blind bandwidth and directional bandwidth bandwidth expansion options, which:

- uses directional bandwidth expansion, i.e. transmits several bits of auxiliary information every 20 ms (for example, per audio frame) only if the high-frequency content (for example, the high-frequency part) of the input audio cannot be restored quite well from the low-frequency audio (for example, the low-frequency part of the audio content),

- Uses blind bandwidth extension, i.e. classical restoration of high-frequency components (e.g., high-frequency parts) from low-frequency basic properties (e.g., properties of the restored low-frequency parts), such as spectral centroid, energy, tilt, coded filter coefficients, otherwise,

- demonstrates very low computational complexity by using scalar instead of vector quantization of auxiliary information and by avoiding operations involving large numbers of data points, such as Fourier transforms and autocorrelation and / or filter calculations,

- is reliable with respect to the characteristics of the input signal, i.e. It is not optimized for specific input signals, such as adult speech in quiet environments, so it is well suited for all types of speech as well as music.

The question is what parameters should be transmitted as supporting information regarding the directed bandwidth expansion according to the embodiments of the present invention, and also when the parameters should be transmitted, should still be resolved.

It has been found that in broadband codecs such as AMR-WB, the spectral envelope of the high frequency region above the region of the base encoder represents the most critical data necessary (or desired) to perform bandwidth expansion with appropriate quality. All other parameters, for example, the exact spectral structure and time envelope, can be extracted from the decoded base signal quite accurately or have little perceptual importance. Therefore, the guided portion of the minimally guided bandwidth extension described herein transmits only the high frequency spectral envelope as auxiliary information (e.g., as bandwidth extension information). This helps in keeping the bandwidth extension auxiliary information low. In addition, it has been experimentally discovered that blind bandwidth extensions provide sufficient, i.e. at least acceptable quality for passages of a motionless in time signal with more or less obvious low-frequency character. Voiced speech, environmental noise, and music sections without percussion instruments are typical examples. In fact, most input to a broadband speech and audio coding system usually falls into this category.

However, signal segments whose instantaneous spectra exhibit a significantly different envelope in the high-frequency region (e.g., in the high-frequency part) compared to the low-frequency region (or low-frequency part) (base encoder) should preferably be encoded through a directional bandwidth extension transmitting the quantized presentation of the high-frequency spectral envelope as auxiliary information (for example, as bandwidth extension information). The reason is that in such spectral structures, blind bandwidth extensions are generally incapable of predicting the development of a high-frequency spectral envelope from the envelope of the base signal, as represented by coded filter coefficients or a spectral-shaped residual signal (which is also known as speech excitation encoders). Well-known examples are unvoiced speech, in particular strong fricative sounds and afflicative sounds such as "s" or German "z", as well as certain percussion instrument sounds mainly in contemporary music. In the embodiments of the present invention, directional bandwidth expansion is therefore only activated for such “unpredictable” high-frequency spectra.

The minimum directional bandwidth extension of the present invention is implemented in the context of LD-USAC, the xHE-AAC low latency version, in order to expand the bandwidth of a broadband encoded (WB encoded) signal from 13.2 Kbps from 6.4 to 8 , 0 kHz. On the encoder side, the blind / guided mode decision is calculated per 20 ms codec frame from the spectral slope of the input signal on the perceptual frequency scale (existing properties also used in the ACELP coding path), as well as time-domain properties, such as changing the frequency of transitions through zero of the input signal provided by the existing transient event detector (which is also used for other decisions on choosing the encoding mode). More specifically, if the spectral tilt is positive, which means that the spectral energy tends to increase with increasing frequency and above a specified threshold value, and at the same time the frequency of zero transitions is increased by a certain ratio or exceeds a certain threshold value, which means that the current If the frame represents the beginning or is located inside the noisy passage of the waveform, then the directional bandwidth extension is selected and signaled. Otherwise, blindband expansion is selected. Regarding the above thresholds, a simple hysteresis is additionally applied to reduce the likelihood of mutual switching between directional bandwidth expansion and blind bandwidth expansion. When the directed bandwidth extension mode is adapted for the frame, the decision thresholds to be used in subsequent frames are slightly reduced, so that the codec is more likely to remain in the guided mode. When it is determined to switch back to blind mode, the initial threshold values are restored, making it less likely that the decision to expand the bandwidth in the guided mode will immediately return.

The remaining procedure for expanding the bandwidth per frame is summarized as follows:

1. If the bandwidth extension is in blind mode, transmit 0 using one bit in the bitstream to signal this mode to the decoder. If necessary, failure to transmit the bit and allowing the decoder to identify the frame as using the blind bandwidth extension mode by analyzing the base signal on the side of the decoder.

2. If the bandwidth extension is in steered mode, transmit 1 using one bit in the bitstream. The encoder then calculates four frequency gain indexes, each of which covers 400 Hz of the input signal, to enable accurate spectrum shaping in the range of 6.4-8 kHz bandwidth expansion in the decoder. In the low latency USAC implementation, each of the four indices is the result of scalar quantization of one of the four QMF energies of the bandwidth extension region relative to the previous QMF energy (or the energy of the QMF spectrum of 4.8-6.4 kHz, in the case of the first bandwidth expansion gain). Since a 2-bit quantizer with a non-zero step with a step size of 2 dB is used, the amplifications cover a range of values of -3 ... 3 dB and use 8 bits per frame. This results in complete auxiliary information of 9 bits per frame for the directional bandwidth extension or, if necessary, 8 bits when signaling is excluded, as shown in step 1.

3. The first bandwidth extension bit is read in the corresponding decoder. If it is 0, blindband expansion is used, otherwise another 8 bits are read, and directional bandwidth expansion is used. If necessary, the reading of the first bit of the bandwidth extension is skipped (since this bit is not present in the bitstream), and the decision on the blind / guided mode is performed locally by analyzing the basic signals, as mentioned in step 1.

4. If the blind bandwidth extension mode is defined in the decoder, bandwidth expansion is performed using only the properties of the decoded base signal. This bandwidth extension essentially corresponds to the principle of bandwidth expansion described in one of the reference materials [2], [3], [6] and [9], but in QMF instead of the DFT region and only with properties with low complexity, retrieved from the base QMF spectrum, for example, spectral centroid / tilt.

5. If the guided bandwidth extension mode is selected in the decoder, four 2-bit gain indexes are inverse quantized into QMF energy amplifications and are used to form the spectrum of the QMF frequency bands of the bandwidth extension region, which are restored, as shown in step 4. In other words , blind bandwidth expansion is also used here, except that spectrum shaping is done through scaling factors transmitted in the bitstream, and not through scaling, extrapolated th of the base signal (which, as a result, is guided bandwidth extension parameters).

6. When switching between blindly expanding bandwidth and guided bandwidth expansion from one frame to the next, a simple smoothing of high-frequency energies is performed in order to minimize switching artifacts (high frequency energy continuity disruptions) caused by behavior based on lower bandwidth-expansion frequencies blindly. Anti-aliasing essentially acts as a smooth transition regulator between blind bandwidth extensions and guided bandwidth extensions: the first frame of guided bandwidth expansion after some blindband bandwidth extension (s) is blanked out a little in the high-frequency region, while the high-frequency blanking of the first bandwidth extension frame blind bandwidth after some directional bandwidth expansion (s) is slightly reduced.

For regular telephone voice content and popular music, experiments show that approximately 13% of all 20 ms frames use the directed bandwidth extension in LD-USAC. Therefore, the average transmission rate of the auxiliary bandwidth extension information is about 2 bits per frame or 0.1 Kbps. This is much less than the (e) SBR speeds (see, for example, reference [8]) or any of the routed speech encoder bandwidth extensions mentioned in this document.

In addition, it should be noted that, as suggested as an optional method in the step-by-step description earlier in this section, 1-bit signaling of the decision to select the bandwidth extension mode to the decoder may be excluded if both the encoder and the decoder can extract this solution from the base coded signal in a bit-accurate manner. This can be achieved if the encoder selects a bandwidth extension mode based on some properties extracted from the locally decoded base signal, since it is the only signal available in the decoder. Provided that transmission errors do not occur in a particular frame, both the encoder and decoder determine the bandwidth extension mode from completely identical properties of the base signal (such as quantized LPC coefficients or time-domain statistics from a decoded residual signal, for example the frequency of transitions through zero, as noted above), the decision on the choice of mode is identical in the encoder and decoder.

Embodiments according to the invention solve a certain quality dilemma in broadband codecs that can be observed at bit rates of 9-13 kbit / s. It has been found that, on the one hand, such speeds are already too low to justify the transfer of even medium volumes of bandwidth extension data, which excludes conventional guided bandwidth expansion systems with 1 Kbps or more of auxiliary information. On the other hand, it has been found that feasible blind bandwidth expansion has been found to sound significantly worse for at least some types of speech or musical material due to the inability to properly predict the parameter from the base signal. It was found that, because of this, it is desirable to reduce the transmission rate of auxiliary information of the guided bandwidth expansion scheme to a level much lower than 1 Kbit / s, which makes it possible to adapt it even when encoding with a very low bit rate. The approach used in the embodiments according to the invention is to identify segments of conventional input signals that are poorly or suboptimally restored by expanding the bandwidth blindly, and to transmit only for these segments auxiliary information necessary in order to improve the quality of high-frequency recovery to an acceptable level (or at least to a level that is in the range of medium quality blindband expansion for this signal). In other words, portions of the high-frequency input signal that are recreated quite well by expanding the blind bandwidth should be encoded with very little or no auxiliary bandwidth extension information, and only passages in which the blind bandwidth extension worsens the overall impression of the quality of the codec, must have high frequency components reproducible through directional bandwidth expansion. Such a bandwidth extension scheme that controls the transmission rate of auxiliary information in a signal-adaptive manner is the subject of the present invention and is called a “minimally directional bandwidth extension”.

Embodiments according to the invention are superior to several approaches to bandwidth expansion, which are given in the relevant literature in recent years (see, for example, sources [1], [2], [3], [4], [5], [6] , [7], [8], [9] and [10]). In general, all of them are either completely blind or completely guided at a given operating point, regardless of the instantaneous characteristics of the input signal. In addition, all implementations of blind bandwidth extensions (see, for example, reference materials [1], [3], [4], [5], [9] and [10]) are optimized exclusively for speech signals, and in fact , they are unlikely to provide satisfactory quality for other input such as music (which is even noted in some publications). In conclusion, most traditional bandwidth extension implementations are relatively complex due to the use of Fourier transforms, LPC filter calculations, or vector quantization of auxiliary information. This can lead to a lack of adaptation of the new coding technology in the mobile markets, given that most mobile devices provide very limited computing power.

As a further conclusion, the embodiments according to the invention create an audio encoder or method for encoding audio or a corresponding computer program, as described above.

Additional embodiments according to the invention create an audio decoder or method for decoding audio or a corresponding computer program, as described above.

Additional embodiments according to the invention create an encoded audio signal or storage medium having a stored encoded audio signal, as described above.

9. Alternative implementation options

Although some aspects are described in the context of the device, it is obvious that these aspects also represent a description of the corresponding method, while the unit or device corresponds to a step of the method or an indication of the step of the method. Similarly, the aspects described in the context of a method step also provide a description of a corresponding unit or element, or feature of a corresponding device. Some or all of the steps of the method may be performed by (or using) a device, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, implementation, some of the one or more most important steps of the method can be performed by this device.

The inventive encoded audio signal may be stored on a digital storage medium or may be transmitted via a transmission medium, such as a wireless transmission medium or a wired transmission medium, for example, the Internet.

Depending on certain implementation requirements, embodiments of the invention may be implemented in hardware or in software. The implementation may be carried out using a digital storage medium, for example, a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM or flash memory, which has stored electronically readable control signals that interact (or allow interaction) with programmable computer system, so that the corresponding method. Therefore, the digital storage medium may be computer readable.

Some embodiments of the invention comprise a storage medium having electronically readable control signals that allow interaction with a programmable computer system in such a way that one of the methods described herein is carried out.

In general, embodiments of the present invention can be implemented as a computer program product with program code, wherein the program code is configured to implement one of the methods when the computer program product is running on a computer. The program code, for example, may be stored on a computer-readable medium.

Other embodiments comprise a computer program for implementing one of the methods described herein stored on a computer-readable medium.

In other words, therefore, an embodiment of the inventive method is a computer program having program code for implementing one of the methods described herein when the computer program is running on a computer.

Therefore, an additional embodiment of the inventive methods is a storage medium (digital storage medium or computer-readable medium) comprising a recorded computer program for implementing one of the methods described herein. A storage medium, a digital storage medium or a medium with recorded data is usually tangible and / or non-volatile.

Therefore, an additional embodiment of the inventive method is a data stream or a sequence of signals representing a computer program for implementing one of the methods described herein. A data stream or signal sequence, for example, may be configured to be transmitted over a data connection, for example, over the Internet.

A further embodiment comprises processing means, for example, a computer or programmable logic device, configured to implement one of the methods described herein.

A further embodiment comprises a computer having an installed computer program for implementing one of the methods described herein.

An additional embodiment according to the invention comprises a device or system configured to transmit (for example, electronically or optically) a computer program for implementing one of the methods described herein to a receiving device. The receiving device, for example, may be a computer, a mobile device, a storage device, or the like. A device or system, for example, may comprise a file server for transmitting a computer program to a receiving device.

In some embodiments, a programmable logic device (eg, a user programmable gate array) may be used to perform part or all of the functionality of the methods described herein. In some embodiments, a user-programmable gate array may interact with a microprocessor to implement one of the methods described herein. In general, the methods are preferably carried out by any device.

The device described herein may be implemented using a hardware device, either using a computer or using a combination of a hardware device and a computer.

The methods described herein may be carried out using a hardware device, either using a computer or using a combination of a hardware device and a computer.

The above embodiments are merely illustrative with respect to the principles of the present invention. It should be understood that modifications and changes to the layouts and details described herein should be apparent to those skilled in the art. Therefore, they are meant to be limited only by the scope of the claims below, and not by way of the specific details presented by describing and explaining the embodiments herein.

BIBLIOGRAPHIC LIST

[1] B. Bessette et al. "The Adaptive Multi-rate Wideband Speech Codec (AMR-WB)", IEEE Trans. on Speech and Audio Processing, edition 10, number 8, November 2002.

[2] B. Geiser et al. "Bandwidth Extension for Hierarchical Speech and Audio Coding in ITU-T Rec. G.729.1", IEEE Trans. on Audio, Speech and Language Processing, Edition 15, Number 8, November 2007.

[3] B. Iser, W. Minker, and G. Schmidt "Bandwidth Extension of Speech Signals", Springer Lecture Notes in Electrical Engineering, vol. 13, New York, 2008.

[4] M. Jelínek and R. Salami "Wideband Speech Coding Advances in VMR-WB Standard", IEEE Trans. on Audio, Speech and Language Processing, Edition 15, Number 4, May 2007.

[5] I. Katsir, I. Cohen and D. Malah "Speech Bandwidth Extension Based on Speech Phonetic Content and Speaker Vocal Tract Shape Estimation", in Proc. EUSIPCO 2011, Barcelona, Spain, September 2011.

[6] E. Larsen and R. M. Aarts, “Audio Bandwidth Extension: Application of Psychoacoustics, Signal Processing and Loudspeaker Design,” Wiley, New York, 2004.

[7] J. Mäkinen et al. "AMR-WB +: A New Audio Coding Standard for 3rd Generation Mobile Audio Services", in Proc. ICASSP 2005, Philadelphia, USA, March 2005.

[8] M. Neuendorf et al. "MPEG Unified Speech and Audio Coding-The ISO / MPEG Standard for High-Efficiency Audio Coding of All Content Types", in Proc. 132nd AES Convention, Budapest, Hungary, April 2012. Also found in the Journal of the AES, 2013.

[9] H. Pulakka and P. Alku "Bandwidth Extension of Telephone Speech Using the Neural Network and the Filter Bank Implementation for Highband Mel Spectrum", IEEE Trans. on Audio, Speech and Language Processing, Edition 19, Number 7, September 2011.

[10] T. Vaillancourt et al. "ITU-T EV-VBR: A Robust 8-32 kbit / s Scalable Coder for Error Prone Telecommunications Channels", in Proc. EUSIPCO 2008, Lausanne, Switzerland, August 2008.

[11] L. Miao et al. "G.711.1 Annex D and G.722 Annex B: New ITU-T Superwideband codecs", in Proc. ICASSP 2011, Prague, Czech Republic, May 2011.

Claims (92)

1. An audio encoder (100; 200) for providing encoded audio information (112; 212) based on the input audio information (110; 210), the audio encoder comprising: - a low-frequency encoder (120; 220), configured to encode the low-frequency part of the input audio information to obtain an encoded representation (122; 222) of the low-frequency part; and - a block (130; 230) for providing bandwidth extension information, configured to provide information (132; 232) for bandwidth expansion based on input audio information; - while the audio encoder is configured to selectively include bandwidth extension information in encoded audio information in a signal-adaptive manner; - wherein the audio encoder comprises a detector (240) configured to identify parts of the input audio information for which bandwidth expansion parameters cannot be estimated based on the low-frequency part with sufficient or desired accuracy; and - while the audio encoder is configured to selectively include bandwidth extension information in encoded audio information for parts of the input audio information identified by the detector. 2. The audio encoder (100; 200) according to claim 1, wherein the audio encoder comprises a detector (240) configured to identify parts of the input audio information that cannot be decoded with sufficient or desired quality based on the encoded representation of the low-frequency part and using the band extension blind transmittance; and - while the audio encoder is configured to selectively include bandwidth extension information in encoded audio information for parts of the input audio information identified by the detector. 3. The audio encoder (100; 200) according to claim 1, wherein the audio encoder comprises a detector (240) configured to identify parts of the input audio information depending on whether the parts are parts that are motionless in time, and depending on whether parts of a low-frequency nature; and - while the audio encoder is configured to selectively exclude the inclusion of bandwidth extension information in encoded audio information for parts of the input audio information identified by the detector as parts that are stationary in time having a low-frequency character. 4. The audio encoder (100; 200) according to claim 3, wherein the detector is configured to identify parts of the input audio information depending on whether the parts contain voiced speech, and / or depending on whether the parts contain environmental noise, and / or depending on whether the parts contain music without percussion instruments. 5. The audio encoder (100; 200) according to claim 1, wherein the audio encoder comprises a detector (240) configured to identify parts of the input audio information depending on whether the difference between the spectral envelope of the low-frequency part and the spectral envelope of the high-frequency part is greater than or equal to the specified index of the difference; and - while the audio encoder is configured to selectively include bandwidth extension information in encoded audio information for parts of the input audio information identified by the detector. 6. The audio encoder (100; 200) according to claim 5, in which the detector is configured to identify parts depending on whether the parts contain unvoiced speech, and / or in which the detector is configured to identify parts, depending on whether parts of the sounds of percussion instruments. 7. The audio encoder (100; 200) according to claim 1, wherein the audio encoder comprises a detector (240) configured to determine a spectral tilt of parts of the input audio information and identify parts of the input audio information depending on whether or not a certain spectral tilt is fixed or equal to a fixed or a variable slope threshold value; and - while the audio encoder is configured to selectively include bandwidth extension information in encoded audio information for parts of the input audio information identified by the detector. 8. The audio encoder (100; 200) according to claim 7, wherein the detector is further configured to determine the frequency of transitions through zero of parts of the input audio information and to identify parts of the input audio information also depending on whether or not a certain frequency of transitions through zero is fixed or a variable threshold value of the zero transition rate, or depending on whether the frequency of zero transitions contains a temporary change that exceeds the threshold value of the transition frequency change Erez zero. 9. The audio encoder (100; 200) according to claim 2, wherein the detector (240) is configured to use hysteresis to identify signal portions of the input audio information to reduce the number of transitions between the identified signal parts and unidentified signal parts. 10. The audio encoder (100; 200) according to claim 1, wherein the audio encoder is configured to selectively include parameters representing the spectral envelope of the high-frequency part of the input audio information in the encoded audio information in a signal-adaptive manner as bandwidth extension information. 11. The audio encoder (100; 200) according to claim 1, wherein the low-frequency encoder is configured to encode the low-frequency part of the input audio information containing frequencies up to a maximum frequency that is in the range of 6-7 kHz, and - while the audio encoder is configured to selectively include in the encoded audio presentation from three to five parameters that describe the intensity of the high-frequency parts of the signal having a passband of 300-500 Hz. 12. The audio encoder (100; 200) according to claim 11, wherein the audio encoder is configured to selectively include 4 scalar quantized parameters describing the intensity of the four high-frequency parts of the signal in the encoded audio presentation, the high-frequency parts of the signal covering frequency ranges above the low-frequency parts. 13. The audio encoder (100; 200) according to claim 11, wherein the audio encoder is configured to selectively include in the encoded audio presentation a plurality of parameters describing the relationship between the energies or intensities of spectrally adjacent frequency parts, one of the parameters describing the ratio or difference between the energy or the intensity of the first high-frequency part and the low-frequency part of the expansion of the passband, while the other parameters describe the relationships or differences between the energies or intensities of each other x high frequency bandwidth extension parts. 14. An audio decoder (400; 500) for providing decoded audio information (412; 512) based on encoded audio information (410; 510), the audio decoder comprising: - a low-frequency decoder (420; 520), configured to decode the encoded representation of the low-frequency part to obtain a decoded representation (422; 522) of the low-frequency part; and - bandwidth extension (430; 530), configured to receive a bandwidth extension signal (432; 532) using blind bandwidth expansion for parts of audio content for which bandwidth expansion parameters are not included in the encoded audio information, and receiving a bandwidth extension signal bandwidth using parameter-guided bandwidth expansion for parts of audio content for which bandwidth expansion parameters are included in the encoded audio information Mats - while the audio decoder is configured to determine whether to use the bandwidth extension blindly or parameter-driven expansion of the bandwidth based on the encoded representation of the low-frequency part without evaluating the signaling flag of the bandwidth extension mode. 15. The audio decoder (400; 500) according to claim 14, wherein the audio decoder is configured to determine whether to receive a bandwidth extension signal using blindly expanding bandwidth or using the bandwidth expansion parameter-guided, on a frame-by-frame basis. 16. The audio decoder (400; 500) according to claim 14, wherein the audio decoder is configured to switch between using blind bandwidth extension and parameter-guided bandwidth extension within an adjacent piece of audio content. 17. The audio decoder (400; 500) according to claim 14, wherein the audio decoder is configured to evaluate flags included in the encoded audio information for various parts of the audio content to determine whether to use the blindband extension or the bandwidth-directed extension of the parameters. 18. The audio decoder (400; 500) according to claim 14, wherein the audio decoder is configured to determine whether to use blind bandwidth expansion or parameter-guided bandwidth expansion based on one or more properties of the decoded representation of the low-frequency part. 19. The audio decoder (400; 500) according to claim 14, wherein the audio decoder is configured to determine whether to use blind bandwidth extension or parameter-guided bandwidth extension based on linear prediction coefficients and / or based on statistics in the time domain of the decoded representations of the low-frequency part. 20. The audio decoder (400; 500) according to claim 14, wherein the bandwidth extension is configured to receive a bandwidth extension signal using one or more properties of the decoded representation of the low-frequency part and / or using one or more parameters of the low-frequency decoder for the time parts input audio content for which bandwidth extension parameters are not included in encoded audio information. 21. The audio decoder (400; 500) according to claim 14, wherein the bandwidth extension is configured to receive a bandwidth extension signal using spectral centroid information and / or using energy information and / or using oblique information and / or using filtering coefficients for the time portions of the input audio content for which bandwidth extension parameters are not included in the encoded audio information. 22. The audio decoder (400; 500) according to claim 14, wherein the bandwidth extension is configured to receive a bandwidth extension signal using bitstream parameters describing the spectral envelope of the high-frequency part for time portions of audio content for which the bandwidth expansion parameters are included in encoded audio information. 23. The audio decoder (400; 500) according to claim 22, wherein the bandwidth extension is configured to estimate from three to five parameters of a bit stream describing the intensity of high-frequency parts of a signal having a bandwidth of 300-500 Hz to obtain a band extension signal transmittance. 24. The audio decoder (400; 500) according to claim 23, wherein three to five parameters of the bit stream describing the intensity of the high-frequency parts of the signal are scalarly quantized with a resolution of 2 or 3 bits, so 6-15 bits are provided for the parameters of the formation of the spectrum bandwidth expansion per audio frame. 25. The audio decoder (400; 500) according to claim 14, wherein the bandwidth expansion is configured to smooth out the energies of the bandwidth expansion signal when switching from blind bandwidth expansion to parameter-controlled bandwidth expansion and / or when switching from directionally expanded bandwidth Blind bandwidth expansion bandwidth. 26. The audio decoder (400; 500) according to claim 25, wherein the bandwidth extension is configured to suppress the high-frequency portion of the bandwidth extension signal for the portion of audio content to which the parameter-directed bandwidth extension is applied after the portion of the audio content to which the bandwidth is applied blind transmittance; and - wherein the bandwidth extension is configured to reduce blanking or increase the level for the high-frequency part of the bandwidth extension signal for the part of the audio content to which the bandwidth expansion is applied blindly, after the part of the audio content to which the bandwidth extension directed by the parameters is applied. 27. The method (600) of providing encoded audio information based on the input audio information, the method comprising the steps of: - encode (610) the low-frequency part of the input audio information to obtain an encoded representation of the low-frequency part; and - provide (620) bandwidth extension information based on input audio information; - wherein the bandwidth extension information is selectively included in the encoded audio information in a signal-adaptive manner; - wherein the method comprises the step of identifying parts of the input audio information for which the bandwidth expansion parameters cannot be estimated based on the low-frequency part with sufficient or desired accuracy; and - wherein the method comprises the step of selectively including bandwidth extension information in encoded audio information for the identified parts of the input audio information. 28. The method (700) of providing decoded audio information based on encoded audio information, the method comprising the steps of: - decode (710) the encoded representation of the low-frequency part to obtain a decoded representation of the low-frequency part; and - receive (720) a bandwidth extension signal using blindly bandwidth extension for parts of audio content for which bandwidth extension parameters are not included in the encoded audio information, and - receive (730) a bandwidth extension signal using the bandwidth extension parameters guided for parts of audio content for which the bandwidth expansion parameters are included in the encoded audio information; - wherein the method comprises the step of determining whether to use the bandwidth extension blindly or parameter-guided bandwidth expansion based on the encoded representation of the low-frequency part without evaluating the signaling flag of the bandwidth extension mode. 29. A computer-readable medium that stores a computer program for implementing the method according to claim 27, when the computer program is executed on a computer. 30. A computer-readable medium on which a computer program is stored for implementing the method of claim 28, when the computer program is executed on a computer. 31. An audio encoder (100; 200) for providing encoded audio information (112; 212) based on input audio information (110; 210), the audio encoder comprising: - a low-frequency encoder (120; 220), configured to encode the low-frequency part of the input audio information to obtain an encoded representation (122; 222) of the low-frequency part; and - a block (130; 230) for providing bandwidth extension information, configured to provide information (132; 232) for bandwidth expansion based on input audio information; - while the audio encoder is configured to selectively include bandwidth extension information in encoded audio information in a signal-adaptive manner; - wherein the audio encoder comprises a detector (240) configured to identify parts of the input audio information depending on whether or not the difference between the spectral envelope of the low-frequency part and the spectral envelope of the high-frequency part exceeds a predetermined difference index; and - while the audio encoder is configured to selectively include bandwidth extension information in encoded audio information for parts of the input audio information identified by the detector. 32. An audio encoder (100; 200) for providing encoded audio information (112; 212) based on input audio information (110; 210), the audio encoder comprising: - a low-frequency encoder (120; 220), configured to encode the low-frequency part of the input audio information to obtain an encoded representation (122; 222) of the low-frequency part; and - a block (130; 230) for providing bandwidth extension information, configured to provide information (132; 232) for bandwidth expansion based on input audio information; - while the audio encoder is configured to selectively include bandwidth extension information in encoded audio information in a signal-adaptive manner; - wherein the audio encoder comprises a detector (240) configured to determine a spectral tilt of parts of the input audio information and identify parts of the input audio information depending on whether or not a particular spectral tilt is equal to a fixed or variable threshold tilt value; and - while the audio encoder is configured to selectively include bandwidth extension information in encoded audio information for parts of the input audio information identified by the detector. 33. An audio decoder (400; 500) for providing decoded audio information (412; 512) based on encoded audio information (410; 510), the audio decoder comprising: - a low-frequency decoder (420; 520), configured to decode the encoded representation of the low-frequency part to obtain a decoded representation (422; 522) of the low-frequency part; and - bandwidth extension (430; 530), configured to receive a bandwidth extension signal (432; 532) using blind bandwidth expansion for parts of audio content for which bandwidth expansion parameters are not included in the encoded audio information, and receiving a bandwidth extension signal bandwidth using parameter-guided bandwidth expansion for parts of audio content for which bandwidth expansion parameters are included in the encoded audio information Mats - wherein the bandwidth extension is configured to smooth out the energies of the bandwidth extension signal when switching from blindly expanding the bandwidth to the parameter-driven bandwidth extension and / or when switching from the parameterized bandwidth expansion to blindly expanding the bandwidth; - wherein the bandwidth extension is configured to suppress the high-frequency portion of the bandwidth extension signal for the portion of audio content to which the parameter-directed bandwidth extension is applied after the portion of the audio content to which blindband expansion is applied; and - wherein the bandwidth extension is configured to reduce blanking or increase the level for the high-frequency part of the bandwidth extension signal for the part of the audio content to which the bandwidth expansion is applied blindly, after the part of the audio content to which the bandwidth extension directed by the parameters is applied. 34. The method (600) of providing encoded audio information based on the input audio information, the method comprising the steps of: - encode (610) the low-frequency part of the input audio information to obtain an encoded representation of the low-frequency part; and - provide (620) bandwidth extension information based on input audio information; - wherein the bandwidth extension information is selectively included in the encoded audio information in a signal-adaptive manner; - wherein the method comprises the step of identifying parts of the input audio information depending on whether or not the difference between the spectral envelope of the low-frequency part and the spectral envelope of the high-frequency part exceeds a predetermined difference index; and - wherein the method comprises the step of selectively including bandwidth extension information in encoded audio information for the identified parts of the input audio information. 35. The method (600) of providing encoded audio information based on the input audio information, the method comprising the steps of: - encode (610) the low-frequency part of the input audio information to obtain an encoded representation of the low-frequency part; and - provide (620) bandwidth extension information based on input audio information; - wherein the bandwidth extension information is selectively included in the encoded audio information in a signal-adaptive manner; - wherein the method comprises determining the spectral slope of the parts of the input audio information and identifying the parts of the input audio information depending on whether the specific spectral tilt is greater than or equal to a fixed or variable threshold tilt value; and - wherein the method comprises the step of selectively including bandwidth extension information in encoded audio information for the identified parts of the input audio information. 36. The method (700) of providing decoded audio information based on encoded audio information, the method comprising the steps of: - decode (710) the encoded representation of the low-frequency part to obtain a decoded representation of the low-frequency part; and - receive (720) a bandwidth extension signal using blindly bandwidth extension for parts of audio content for which bandwidth extension parameters are not included in the encoded audio information, and - receive (730) a bandwidth extension signal using the bandwidth extension parameters guided for parts of audio content for which the bandwidth expansion parameters are included in the encoded audio information; - wherein the method comprises the step of smoothing the energies of the bandwidth extension signal when switching from blindly expanding the bandwidth to the parameter-driven bandwidth extension and / or when switching from the bandwidth-expanding parameter to blindly bandwidth expansion; - wherein the method comprises the step of quenching the high-frequency part of the bandwidth extension signal for the part of the audio content to which the parameter-directed bandwidth expansion is applied, after the part of the audio content to which the bandwidth expansion is applied blindly; and - wherein the method comprises reducing the blanking or increasing the level for the high-frequency part of the bandwidth extension signal for the part of the audio content to which the bandwidth expansion is applied blindly, after the part of the audio content to which the parameter-directed bandwidth extension is applied. 37. A computer-readable medium that stores a computer program for implementing the method according to claim 34 or 35, when the computer program is executed on a computer. 38. A computer-readable medium that stores a computer program for implementing the method according to claim 36, when the computer program is executed on a computer.

Images