RU2799033C1 - System and method for generating a series of high-frequency subband signals - Google Patents

Abstract

FIELD: computer technologies.

SUBSTANCE: invention relates to computer technology for high-frequency reconstruction/regeneration of audio signals. The technical result is to improve the quality of the high-frequency signal. The technical result is achieved by receiving a number of low-frequency subband signals; receiving a set of target energies, where each target energy covers a different target interval within the high frequency interval and is indicative of the required energy of one or more high frequency subband signals lying within the target interval; generating a plurality of high frequency subband signals from a plurality of low frequency subband signals and from a plurality of spectral gains, respectively, associated with the plurality of low frequency subband signals; and adjusting the energy of the plurality of high frequency subband signals using the set of target energies, wherein the energy adjustment of the plurality of high frequency subband signals comprises limiting the energy adjustment of the high frequency subband signals lying within the limit interval.

EFFECT: improving the quality of the high-frequency signal.

3 cl, 14 dwg

Description

TECHNICAL APPLICATION

This application relates to HFR (High Frequency Reconstruction/Regeneration) of audio signals. In particular, the application relates to a method and system for performing HFR audio signals containing large changes in energy levels within the low frequency range, which is used to reconstruct the high frequencies of the audio signal.

BACKGROUND OF THE INVENTION

HFR technologies such as Spectral Band Replication (SBR) can significantly improve the coding efficiency of conventional perceptual audio codecs. HFR combined with MPEG-4 Advanced Audio Coding (AAC) forms an extremely efficient audio codec that is already in use in XM Satellite Radio and Digital Radio Mondiale systems, and is also standardized in 3GPP, DVD Forum, and others. Combination of AAC and SBR is called aacPlus. It is part of the MPEG-4 standard, where it is referred to as the High Efficiency AAC Profile (HE-AAC). In general, HFR technology can be combined with any perceptual audio codec in a top-down and bottom-up compatibility order, making it possible to upgrade existing broadcast systems such as the MPEG Layer-2 system used in the Eureka DAB system. HFR techniques can also be combined with speech codecs to enable wideband speech at ultra-low bit rates.

The basic idea behind HFR is the observation that for the same signal, there is usually a strong correlation between the characteristics of the high frequency range of the signal and the characteristics of the low frequency range of the signal. Therefore, a good approximation to represent the original input high frequency range of the signal can be achieved by converting the signal from the low frequency range to the high frequency range.

This transformation concept was established in WO 98/57436, which is incorporated herein by reference, as a method of recreating a high frequency band from a low frequency band of an audio signal. By using this concept, significant bit rate savings can be achieved when encoding audio and/or speech. In the following, reference will be made to audio coding, however, it should be noted that the described methods and systems are equally applicable to speech coding and unified speech and audio coding (USAC).

отображается в синусоиду с частотой

, где

— фиксированный сдвиг частоты. Иными словами, высокочастотный сигнал может генерироваться из низкочастотного сигнала при помощи операции «копирования вверх» низкочастотных поддиапазонов до высокочастотных поддиапазонов. Дальнейший подход к созданию сигнала возбуждения высоких частот может включать гармоническое преобразование низкочастотных поддиапазонов. Гармоническое преобразование порядка Т, как правило, предназначается для отображения синусоиды с частотой

низкочастотного сигнала в синусоиду высокочастотного сигнала с частотой

, где Т>1.The high frequency reconstruction may be performed in the time domain or in the frequency domain using a selected filter bank or transform. This process usually involves several steps, where the two main operations are to first create a high frequency excitation signal, and then to shape the high frequency excitation signal into a shape that approximates the spectrum envelope of the original high frequency spectrum. The step of creating a high frequency excitation signal may, for example, be based on single sideband (SSB) signal modulation, where a sinusoid with frequency

mapped to a sinusoid with a frequency

, Where

is a fixed frequency shift. In other words, a high frequency signal can be generated from a low frequency signal by copying up the low frequency subbands to the high frequency subbands. A further approach to creating a high frequency excitation signal may involve harmonic transposition of the low frequency subbands. The harmonic transformation of the order T is usually intended to display a sinusoid with a frequency

low-frequency signal into a sinusoid of a high-frequency signal with a frequency

, where T >1.

HFR technology can be used as part of source encoding systems where classified control information for controlling the HFR process is passed from an encoder to a decoder along with a narrowband/low frequency signal representation. For systems where an additional control signal cannot be transmitted, the process can be applied at the decoder side with the appropriate control data estimated at the decoder side based on the available information.

The aforementioned high frequency drive envelope adjustment aims to improve the shape of the spectrum that resembles the original high frequency band. To implement this adjustment, the shape of the spectrum of the high-frequency signal must be modified. In other words, the adjustment that is intended to be applied to the high frequency band is a function of the existing spectrum envelope and the desired target spectrum envelope.

For systems operating in the frequency domain, such as HFR systems implemented in a pseudo-QMF filter bank, the methods of the current state of the art are suboptimal in this regard, since creating a high band signal by combining multiple contributions from the original frequency band introduces a high frequency band into the high band. the band that is subjected to envelope adjustment, the unnatural envelope of the spectrum. In other words, the high frequency band or high frequency signal generated from the low frequency signal during the HFR process typically exhibits an unnatural spectrum envelope (typically including spectrum discontinuities). This presents difficulties for the spectrum envelope controller, since the controller must not only be able to apply the desired spectrum envelope with the proper time and frequency resolution, but must also be able to cancel the spectral characteristics artificially introduced by the HFR signal generator. This introduces complex design constraints for the envelope control. As a result, these difficulties tend to result in a perceptible loss of high frequency energy and audible discontinuities in the spectral shape of the high band signal, particularly for speech type signals. In other words, conventional HFR signal generators tend to introduce discontinuities and level changes in the high band signal for signals that have significant level changes within the low frequency range, such as hissing signals. When such a high band signal is then accessed by an envelope controller, it cannot consistently and reasonably separate the newly introduced discontinuity from any natural spectral response of the low band signal.

The present document describes a solution to the aforementioned problem, which results in improved perceived audio quality. In particular, the present document describes a solution to the problem of generating a high band signal from a low band signal, where the spectral envelope of the high band signal is effectively adjusted to resemble the original spectral envelope in the high band without introducing unwanted artifacts.

BRIEF DESCRIPTION OF THE INVENTION

The present document proposes an additional correction step as part of high frequency reconstruction signal generation. As a result of the additional correction step, the quality of the audio signal of the high frequency component, or high frequency signal, is improved. The additional correction step can be applied to all source encoding systems that use high frequency reconstruction techniques, as well as to any single complete method or post-processing system that aims to recreate the high frequencies of an audio signal.

In one aspect, a system is described that is configured to generate a number of high frequency subband signals covering a high frequency interval. The system may be configured to generate a plurality of high frequency subband signals from a plurality of low frequency subband signals. The set of low-frequency subband signals may be low-band audio or narrow-band audio subband signals that can be determined using an analysis filter bank or transform. In particular, a number of low frequency subband signals can be determined from the low frequency band signal in the time domain using a QMF (Quadrature Mirror Filter) or FFT (Fast Fourier Transform) analysis filter bank. The set of generated high frequency subband signals may correspond to an approximation of the high frequency subband signals of the original audio signal from which the set of low frequency subband signals was derived. In particular, a number of low frequency subband signals and a number of (re-)generated high frequency subbands may correspond to subbands of the QMF filter bank and/or FFT transform.

The system may include means for receiving a number of low frequency subband signals. As such, the system may be placed downstream of an analysis filter bank or transform that generates a number of low-frequency subband signals from a low-band signal. The low band signal may be an audio signal that has been decoded from the received bitstream at the core decoder. The bitstream may be stored in memory on a storage medium such as a CD or DVD, or the bitstream may be received by a decoder via a transmission medium such as an optical or radio transmission medium.

The system may include means for receiving a set of target energies, which may also be referred to as scale factor energies. Each target energy may cover a different target interval, which may also be referred to as a scalefactor band, within the high frequency interval. As a rule, a set of target intervals, which corresponds to a set of target energies, completely covers the high frequency interval. The target energy from a set of target energies is typically indicative of the required energy for one or more high frequency subband signals lying within the respective target interval. In particular, the target energy may correspond to the average required energy for one or more high frequency subband signals that lie within the respective target interval. The target energy of the target interval is generally obtained from the signal energy of the high band of the original audio signal within the target interval. In other words, the set of target energies, as a rule, describes the envelope of the spectrum of the high-frequency part of the original audio signal.

The system may include means for generating high frequency subband signals from a plurality of low frequency subband signals. To this end, the means for generating the plurality of high frequency subband signals may be configured to perform copy up transformation on the plurality of low frequency subband signals and/or to perform harmonic transformation on the plurality of low frequency subband signals.

In addition, the means for generating the plurality of high frequency subband signals may take into account a plurality of spectrum gain factors during the process of generating the plurality of high frequency subband signals. A number of spectrum gains may be respectively associated with a number of low frequency subband signals. In other words, each low frequency subband signal from the set of low frequency subband signals may comprise a corresponding spectrum gain from the set of spectrum gains. A spectrum gain from a set of spectrum gains may be applied to the corresponding low band signal.

The set of spectrum gains may be related to the energy of the corresponding set of low frequency subband signals. In particular, each spectrum gain may be associated with the energy of its corresponding low-frequency subband signal. In one embodiment of the invention, the spectrum gain is determined based on the energy of the corresponding low frequency subband signal. To this end, it is possible, on the basis of a number of energy values for a number of low-frequency subband signals, to determine a frequency-dependent curve. In this case, the method for determining the set of gains may be based on a frequency dependent curve that is determined from the (eg, logarithmic) representation of the energies of the set of low frequency subband signals.

In other words, a number of spectrum gains can be derived from a frequency dependent curve approximating the energy of a number of low frequency subband signals. In particular, the frequency dependent curve may be a polynomial of a predetermined order/degree. Alternatively, or in addition, the frequency dependent curve may include different curve segments, where the different curve segments are mapped to the energy of a number of low frequency subband signals in different frequency intervals. The different segments of the curve may represent different polynomials of a predetermined order. In one embodiment of the invention, the various curve segments are zero order polynomials, and thus the curve segments represent energy averages for the energy of a number of low frequency subband signals within a respective frequency interval. In a further embodiment of the invention, the frequency dependent curve is approximated to the energy of a number of low frequency subband signals by performing a moving average filtering operation over the various frequency intervals.

In one embodiment of the invention, the gain from the set of gains is derived from the difference between the average energy of the set of low frequency subband signals and the corresponding value of the frequency dependent curve. The corresponding value of the frequency-dependent curve may be the value of the curve at a frequency lying within the frequency range of the low-frequency subband signal to which the gain corresponds.

As a rule, the energy of a number of low frequency subband signals is determined on a certain time grid, for example on a frame-by-frame basis, i.e. the energy of the low-frequency sub-band signal within a certain period of time defined by the time grid corresponds to the average energy of the discrete values of the low-frequency sub-band signal within this period of time, for example, within a frame. Therefore, in the selected time grid, a different set of spectrum gain factors may be determined, for example, a different set of spectrum gain factors may be determined for each frame of the audio signal. In one embodiment of the invention, a set of spectrum gain factors may be determined based on successive samples, for example, by determining the energy of a number of low frequency subbands using a floating window over the samples of each low frequency subband signal. It should be noted that the system may include means for determining a set of spectrum gains from a set of low frequency subband signals. These means may be configured to perform the above methods in order to determine a number of spectrum gain factors.

The means for generating the plurality of high frequency subband signals may be configured to amplify the plurality of low frequency subband signals using an appropriate plurality of spectrum gain factors. Although reference is made to "gain" in the following description, the "gain" operation may be replaced by other operations such as a "multiply" operation, a "zoom" operation, or an "adjust" operation. The amplification can be carried out by multiplying the discrete value of the low-frequency subband signal by the spectrum gain factor corresponding to it. In particular, means for generating a plurality of high frequency subband signals may be configured to determine a sample value of the high frequency subband signal at a given time from the samples of the low frequency subband signal at a given time and at least one prior time. In addition, the samples of the low-frequency subband signal can be amplified by the corresponding spectrum gain factor from the plurality of spectrum gain factors. In one embodiment, the means for generating a plurality of high frequency subband signals are configured to generate a plurality of high frequency subband signals from a plurality of low frequency subband signals in accordance with the "copy up" algorithm defined in the MPEG-4 SBR. The set of low frequency subband signals used in said "copy up" algorithm may be amplified using a number of spectrum gain factors, where the "gain" operation may be performed as described above.

The system may include means for adjusting the energy of a number of high frequency subband signals using a set of target energies. This operation is generally referred to as spectrum envelope adjustment. Spectrum envelope adjustment can be performed by adjusting the energy of a number of high frequency subband signals such that the average energy of the high frequency subband signals lying within the target interval matches the corresponding target energy. This can be done by determining the envelope adjustment value from the energy values of a number of high frequency subband signals lying within the target interval and from the corresponding target energy. In particular, the envelope adjustment value may be determined based on the ratio of the target energy and the energy values of a number of high frequency subband signals lying within the corresponding target interval. The specified envelope adjustment value can be used to adjust the energy of a number of high frequency subband signals.

In one embodiment of the invention, the means for adjusting the energy include means for limiting the adjustment of the energy of the high frequency subband signals lying within the limit interval. As a rule, the limit interval covers more than one target interval. Limiting means are typically applied to avoid unwanted noise amplification within certain high frequency subband signals. For example, the clipping means may be configured to determine an average envelope adjustment value from the envelope adjustment values corresponding to target intervals covered by or within the clipping interval. In addition, the clipping means may be configured to limit the energy adjustment of the high frequency subband signals lying within the clipping interval to a value proportional to the average value of the envelope trim.

Alternatively or in addition, the means for adjusting the energy of a number of high frequency subband signals may include means for ensuring that the adjusted high frequency subband signals lying within a certain target interval have the same energy. The latter means are often referred to as "interpolation" means. In other words, the "interpolation" means ensures that the energy of each of the high frequency subband signals lying within a certain target interval corresponds to the target energy. The "interpolation" means may be implemented by adjusting each high frequency subband signal within a certain target interval individually so that the energy of the adjusted high subband signal matches the target energy associated with the determined target interval. This can be done by determining a different envelope adjustment value for each high frequency subband signal within a certain target interval. The different envelope adjustment value may be determined based on the energy of the determined high frequency subband signal and the target energy corresponding to the determined target interval. In one embodiment of the invention, the envelope adjustment value for a particular high frequency subband signal is determined based on the ratio of the target energy and the energy of the determined high frequency subband signal.

The system may also include means for receiving control data. The control data can be an indication of whether a set of spectrum gains should be applied to generate a set of high frequency subband signals. In other words, the control data can serve as an indication of whether additional gain adjustment of the low frequency subband signals should be performed or not. Alternatively, or in addition, the control data may be indicative of the method to be applied to determine the set of spectrum amplification signals. For example, the control data may be indicative of a predetermined polynomial order to be applied to determine a frequency dependent curve approximating the energies of a number of low frequency subband signals. The control data is typically received from an appropriate encoder, which parses the original audio signal and informs the appropriate decoder, or HFR system, on how to decode the bitstream.

According to another aspect, an audio signal decoder configured to decode a bit stream including a low frequency audio signal and including a set of target energies describing a spectrum envelope of a high frequency audio signal is described. In other words, an audio signal decoder configured to decode a bit stream indicative of a low frequency audio signal and indicative of a set of target energies describing a spectrum envelope of a high frequency audio signal is described. The audio signal decoder may include a base decoder and/or a transform unit configured to determine from the bitstream a number of low frequency subband signals associated with the low frequency audio signal. Alternatively or in addition, the audio decoder may include a high frequency generator in accordance with the system described herein, where the system can be configured to determine a number of high frequency subband signals from a number of low frequency subband signals and a set of target energies. Alternatively or in addition, the decoder may include a merger and/or demapper configured to generate an audio signal from a plurality of low frequency subband signals and a plurality of high frequency subband signals. The merge and inverse transform block may include a synthesis filter block or a transformation, such as an inverse QMF filter block or an inverse FFT.

According to the following aspect, an encoder configured to generate control data from an audio signal is described. The audio encoder may include means for analyzing the spectrum shape of the audio signal and for determining the degree of discontinuity in the spectrum envelope introduced by regenerating the high frequency component of the audio signal from the low frequency component of the audio signal. As such, the encoder may include certain elements of the corresponding decoder. In particular, the encoder may include the HFR system described herein. This may allow the encoder to determine the degree of discontinuity in the spectrum envelope that might be introduced into the high frequency component of the audio signal at the decoder side. Alternatively, or in addition, the encoder may include means for generating control data for controlling regeneration of the high frequency component based on the degree of discontinuity. In particular, the control data may correspond to the control data received by the corresponding decoder of the HFR system. The control data can be indicative of whether to use a set of spectrum gain factors during the HFR process and/or which predetermined polynomial order to use to determine the set of spectrum gain factors. In order to determine this information, it is possible to determine the ratio of the selected parts of the low frequency range, i.e. a range of frequencies covered by a number of low frequency subband signals. Information about this relationship can be determined by examining the lowest frequencies in the low band and the highest frequencies in the low band to estimate the change in the spectrum of the low band signal, which will then be used in the decoder for high frequency reconstruction. A high ratio may indicate an increased degree of discontinuity. Control data can also be determined using signal type detectors. For example, the detection of speech signals may indicate an increased degree of discontinuity. On the other hand, the detection of pronounced sinusoids in the original audio signal may lead to the fact that during the HFR process, a number of spectrum gain factors should not be applied.

In another aspect, a method for generating a plurality of high frequency subband signals covering a high frequency interval from a plurality of low frequency subband signals is described. The method may include the steps of receiving a number of low frequency subband signals and/or receiving a set of target energies. Each target energy may cover a different target interval within the high frequency interval. In addition, each target energy may be indicative of the desired energy of one or more high frequency subband signals lying within the target interval. The method may include the step of generating a plurality of high frequency subband signals from a plurality of low frequency subband signals and from a plurality of spectrum gains, respectively, associated with the plurality of low frequency subband signals. Alternatively, or in addition, the method may include the step of adjusting the energy of a number of high frequency subband signals using a set of target energies. The step of adjusting the energy may include the step of restricting the adjustment of the energy of the signals of the high frequency subbands lying within the limit interval. As a rule, the limit interval covers more than one target interval.

According to the following feature, a method for decoding a bit stream indicative of or including a low frequency audio signal and a set of target energies describing the spectral envelope of a corresponding high frequency audio signal is described. As a rule, low-frequency and high-frequency audio signals correspond to the low-frequency and high-frequency components of the same original audio signal. The method may include the step of determining a number of low frequency subband signals associated with a low frequency audio signal from the bitstream. Alternatively or in addition, the method may include the step of determining a set of high frequency subband signals from a set of low frequency subband signals and a set of target energies. This step is typically performed in accordance with the HFR methods described herein. Subsequently, the method may include the step of generating an audio signal from a plurality of low frequency subband signals and from a plurality of high frequency subband signals.

According to another aspect, a method for generating control data from an audio signal is described. The method may include the step of analyzing the spectrum shape of the audio signal to determine the degree of discontinuity introduced by regenerating the high frequency component of the audio signal from the low frequency component of the audio signal. In addition, the method may include the step of generating control data for controlling the regeneration of the high frequency component based on the degree of discontinuities.

According to the following feature, a program implemented in software is described. A program implemented in software may be adapted to execute on a processor and to perform the steps of the methods described herein when executed on a computing device.

According to another aspect, a storage medium is described. The storage medium may include a program implemented in software adapted to execute on a processor and to perform the steps of the methods described herein when executed on a computing device.

According to the following feature, a computer program product is described. The computer program may include executable instructions for performing the steps of the methods described herein when implemented on a computer.

It should be noted that the methods and systems, including their preferred embodiments, as described in this patent application, can be used alone or in combination with other methods and systems disclosed in this document. In addition, all features of the methods and systems described in this patent application can be arbitrarily combined. In particular, some characteristic features of the claims can be combined in an arbitrary manner with other characteristic features.

BRIEF DESCRIPTION OF GRAPHICS

The invention is explained below by means of illustrative examples with reference to the accompanying drawings, where

Fig. 1a illustrates the absolute spectrum of one example of a high band signal before spectrum envelope adjustment;

Fig. 1b illustrates an example of the relationship between audio data time frames and envelope time boundaries for spectrum envelopes;

Fig. 1c illustrates the absolute spectrum of one example of a high band signal before spectrum envelope adjustment and the corresponding scale factor bands, cutoff bands, and HF (high frequency) glues;

Fig. 2 illustrates an embodiment of an HFR system where an additional gain control step is added to the copy-up process;

Fig. 3 illustrates an approximation of the coarse spectral envelope for an example lowband signal;

Fig. 4 illustrates an embodiment of an additional gain controller operating on optional control data, QMF subband samples, and outputting a gain curve;

Fig. 5 illustrates a more detailed embodiment of the additional gain control of FIG. 4;

Fig. 6 illustrates an embodiment of an HFR system with a narrowband signal as an input signal and a wideband signal as an output signal;

Fig. 7 illustrates an embodiment of an HFR system included in an audio decoder module SBR;

Fig. 8 illustrates an embodiment of a high frequency reconstruction module using an audio signal decoder as an example;

Fig. 9 illustrates an embodiment of an example encoder;

Fig. 10a illustrates a spectrogram of an example of a vocal passage that has been decoded using a conventional decoder;

Fig. 10b illustrates a spectrogram of the vocal passage of FIG. 10a that has been decoded using a decoder applying additional gain control processing; And

Fig. 10c illustrates a spectrogram of the vocal passage of FIG. 10a for the original uncoded signal.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following embodiments of the invention are merely illustrative of the principles of the present invention "PROCESSING AUDIO SIGNALS DURING HIGH FREQUENCY RECONSTRUCTION". It should be understood that modifications and changes to the circuits and details described herein will be apparent to those skilled in the art. Therefore, the intention is to limit only the scope of the following claims and not the specific details provided herein for the purpose of describing and explaining embodiments of the invention.

As described above, audio decoders using HFR techniques typically include an HFR block for generating a high frequency audio signal, followed by a spectrum envelope adjuster for adjusting the spectrum envelope of the high frequency audio signal. Adjustment of the spectrum envelope of the audio signal is typically accomplished through some implementation of a filter bank or through filtering in the time domain. The adjustment may either seek to correct the absolute envelope of the spectrum, or it may be performed by filtering, which also corrects the phase characteristics. For any path, adjustment is typically a combination of two steps: removing the current spectrum envelope and overlaying the target spectrum envelope.

It is important to note that the methods and systems described herein are not only aimed at eliminating the envelope spectrum of an audio signal. The methods and systems seek to perform appropriate spectral envelope correction of the low band signal spectrum as part of the high frequency regeneration step so as not to introduce discontinuities in the high band spectrum envelope created by combining different low band fragments, i.e. low-frequency signal, shifted or converted to other frequency ranges of the high-frequency band, i.e. high frequency signal.

On FIG. 1a shows a stylistically depicted spectrum 100, 110 of the HFR block's output signal before passing into the envelope controller. On the top panel, to generate the high band signal 105 from the low band signal 101, a copy-up (two-splice) method is used, such as the copy-up method used in MPEG-4 SBR (spectral band replication), which is described in the document "ISO/IEC 14496-3 Information Technology - Coding of audio-visual objects - Part 3: Audio" and which is incorporated by reference into this document. The copy-up method translates parts of the lower frequencies 101 into higher frequencies 105. In the lower panel, a harmonic transformation method (with two splices) is applied to generate the high band signal 115 from the low band signal 111, for example, the harmonic transformation method from MPEG-D USAC, which is described in "MPEG-D USAC: ISO/IEC 23003-3 - Unified Speech and Audio Coding" and which is incorporated herein by reference.

In the subsequent envelope adjustment step, the frequency components 105, 115 are overlaid with the target spectrum envelope. As can be seen from the spectrum 105, 115 passing into the envelope controller, in the form of a spectrum of the high frequency drive signal 105, 115, i.e. of the high-frequency band signal included in the envelope control, discontinuities are observed (especially at the boundaries of the glues). These discontinuities result from multiple low frequency contributions 101, 111 being used to generate the high band 105, 115. As can be seen, the spectral shape of the high band signal 105, 115 is related to the spectral shape of the low band signal 101, 111. Accordingly, certain spectrum shapes of the low band signal 101, 111, such as the gradient shape shown in FIG. 1a can lead to discontinuities in the overall spectrum 100, 110.

In addition to the spectrum 100, 110 of FIG. 1a illustrates an example of frequency bands 130 of spectrum envelope data representing the target spectrum envelope. These frequency bands 130 are referred to as scalefactor bands or target intervals. As a rule, the target value of energy, i.e. the scale factor energy, determined for each target interval, i.e. for the scale factor band. In other words, the scale factor bands determine the effective frequency resolution of the target spectral envelope, since they typically represent a single target energy per target interval. Using the scalefactors, or target energies, specified for the scalefactor bands, the subsequent envelope adjuster seeks to adjust the highband signal so that the energy of the highband signal within the scalefactor bands is equal to the energy of the received spectrum envelope data, i.e. target energy for the respective scale factor bands.

On FIG. 1c, using an example of an audio signal, a more detailed description is provided. The graph shows the spectrum of the actual audio signal 121 entering the envelope control, as well as the corresponding original signal 120. In this particular example, the SBR range, i.e. the high-frequency signal range starts at 6.4 kHz and continues into three different replications of the low-frequency range. The frequency ranges of the various replications are labeled "glue 1", "glue 2" and "glue 3". It is clearly seen from the spectrogram that gluing introduces discontinuities in the spectrum envelope at about 6.4 kHz, 7.4 kHz, and 10.8 kHz. In this example, these frequencies correspond to the gluing boundaries.

Fig. 1c also illustrates scalefactor bands 130 as well as limit bands 135, the function of which will be described in more detail below. In the illustrated embodiment, an MPEG-4 SBR envelope control is used. This envelope control operates using the QMF filter block. The main features of the operation of such an envelope control are:

• calculating the average energy within the scale factor band 130 of the input signal to the envelope controller, ie. a signal output from the HFR block; in other words, within each scalefactor band 130/each target interval 130, the average energy of the regenerated highband signal is calculated;

• determining a gain amount, also referred to as an envelope adjustment value, for each scalefactor band 130, where the envelope adjustment value is the square root of the energy ratio between the target energy (ie, the target energy obtained from the encoder) and the average energy of the regenerated signal 121 a high frequency band within the corresponding scale factor band 130;

• applying the appropriate envelope adjustment value to the frequency band of the regenerated high band signal 121, where the band corresponds to the corresponding scale factor band 130 .

In addition, the envelope control can include additional steps and changes, such as:

• a limiting function that limits the maximum allowable envelope adjustment value applicable in a particular frequency band, ie. on the bounding bar 135. The maximum allowable envelope adjustment value is a function of the envelope adjustment values determined for the various scale factor bands 130 that fall within the bounding band 135. In particular, the maximum allowable envelope adjustment value is a function of the average of the envelope adjustment values determined for the various scalefactor bands 130 that fall within the limit band 135. For example, the maximum allowable envelope adjustment value may be the average of the respective envelope adjustment values multiplied by a limit factor (such as, for example, 1.5). The clipping function is generally used to limit the introduction of noise into the regenerated high band signal 121 . This is especially significant for audio signals that include pronounced sinusoids, i.e. audio signals that have a spectrum with distinct peaks at certain frequencies. In the absence of a limiting function, significant values could be determined for scalefactor bands 130 for which the original audio signal includes distinct peaks. As a result, the scale factor band 130 could be fully adjusted (not just its distinct peak), thus introducing noise;
• An interpolation function that allows calculation of envelope adjustment values for each individual QMF subband within a scalefactor band instead of calculating a single envelope adjustment value for the entire scalefactor band. Since scalefactor bands typically include more than one QMF subband, the envelope adjustment value can be calculated as the ratio of the energy of a specific QMF subband within the scalefactor band and the target energy received from the encoder, instead of calculating the ratio of the average energy for all QMF subbands within the scalefactor band and the target energy received from the encoder. Thus, for each QMF subband within the scalefactor band, a different envelope adjustment value can be computed. It should be noted that the received value of the target energy for the scalefactor band, as a rule, corresponds to the average energy of this frequency band in the original signal. Exactly how to apply the received average target energy to the corresponding frequency band of the regenerated highband signal is up to the operation of the decoder. This can be done by applying a common envelope adjustment value to the QMF subbands within the scale factor band of the regenerated highband signal, or by applying an individual envelope adjustment value to each QMF subband. The latter approach can be thought of as if the received envelope information (i.e., one target energy per scalefactor band) were "interpolated" over the QMF subbands within the scalefactor band to provide greater frequency resolution. Therefore, this approach is referred to in MPEG-4 SBR as "interpolation".

Returning to Fig. 1c, it can be seen that the envelope controller would have to apply high envelope adjustment values in order to bring the spectrum 121 of the signal entering the envelope controller into line with the spectrum 120 of the original signal. It can also be seen that large changes in envelope adjustment values occur due to discontinuities within the limit bands 135. As a result of these large changes, envelope adjustment values that correspond to the local minima of the regenerated spectrum 121 will be limited by the limiting function of the envelope regulator. As a result, discontinuities in the regenerated spectrum 121 will persist even after the envelope adjustment operation has been performed. On the other hand, if the limiting function is not used, unwanted noise may be introduced, as described above.

Thus, a problem in regenerating the high band signal occurs for any signal that contains large changes in level for the low band range. This problem occurs due to the discontinuities introduced during high frequency regeneration of the high frequency band. When the envelope controller is subsequently exposed to this regenerated signal, it cannot consistently and reasonably separate the newly introduced discontinuity from any "real" spectral response of the low band signal. The results of this problem are twofold. First, spectral shapes are introduced into the high-band signal that the envelope control cannot compensate for. Accordingly, the output signal has an irregular spectrum shape. Secondly, because this effect enters and exits as a function of the spectral characteristics of the low frequency band, the effect of instability is perceived.

The present document aims to solve the above problem by describing a method and system that provides an envelope controller with an HFR high band signal that does not exhibit spectral discontinuities. To this end, it is proposed to eliminate, or lower, the spectrum envelope of the low-frequency band signal when performing high-frequency regeneration. By doing so, you will be able to avoid introducing any discontinuities in the high-band signal before making the envelope adjustment. As a result, the envelope controller does not have to manipulate the specified discontinuities in the spectrum. In particular, a conventional envelope control can be used where a clipping function is applied to avoid introducing noise into the regenerated high band signal. In other words, the described method and system can be applied to regenerate an HFR high band signal containing little or no discontinuity and having a low noise level.

It should be noted that the time resolution of the envelope controller may differ from the time resolution of the proposed spectral envelope processing during highband signal generation. As noted above, the spectral envelope processing during highband signal regeneration is intended to modify the spectrum envelope of the lowband signal to facilitate processing in a subsequent envelope controller. This processing, i.e. modification of the spectrum envelope of the low band signal may be performed, for example, once per frame of the audio signal, where the envelope control may adjust the spectrum envelope over several time intervals, i.e. using several accepted spectrum envelopes. This is described in FIG. 1b, where the top panel shows a time grid 150 of the spectrum envelope data and the bottom panel shows a time grid 155 for processing the spectrum envelope of the low band signal during regeneration of the high band signal. As seen in the example of Fig. 1b, the time boundaries of the spectrum envelope data vary with time, while the spectrum envelope processing of the low band signal operates on a fixed time grid. It can also be seen that multiple envelope adjustment cycles (represented by time boundaries 150) can be performed during a single spectrum envelope processing cycle of the low band signal. In the illustrated example, the spectral envelope processing of the lowband signal operates on a frame-by-frame basis, which means that a different set of spectrum gain factors is determined for each frame. It should be noted that the low band signal processing can operate on any time grid and that the time grid of said processing need not be the same as that of the spectral envelope data.

On FIG. 2 shows a 200 HFR system based on a filter bank. The HFR system 200 operates using a pseudo-QMF filter bank, and the system 200 can be used to obtain the high-band and low-band signal 100 illustrated in the top panel of FIG. 1a. However, an additional gain adjustment step is added as part of the high frequency generation process, which in the illustrated example is a copy-up process. The low frequency input signal is analyzed by the 32-band QMF 201 to generate a series of low frequency subband signals. Some, or all, of the low frequency subband signals are glued together at higher frequency positions in accordance with a high frequency (HF) generation algorithm. In addition, a number of low-frequency subbands are directly included in the block 202 synthesis filters. The above synthesis filter bank 202 is a 64-band inverse QMF 202. For the specific application illustrated in FIG. 2, the use of a 32-band QMF analysis filter bank 201 and the use of a 64-band QMF synthesis filter bank 202 will result in an output sample rate of the output signal that is twice the input sample rate of the input signal. However, it should be noted that the systems described herein are not limited to systems with different input and output sample rates. Those skilled in the art can imagine many different ratios of sampling rates.

As described in FIG. 2, lower frequency subbands are mapped to higher frequency subbands. The gain adjustment step 204 is introduced as part of this copy-up process. The generated high frequency signal, i.e. the generated set of high frequency subband signals is input to the envelope controller 203 (which optionally includes a limiter and/or interpolation function) before being combined with the set of low frequency subband signals in the synthesis filter bank 202 . By employing such an HFR system 200, and in particular by employing the gain control step 204, the introduction of spectral envelope discontinuities shown in FIG. 1. To this end, the envelope adjustment step 204 modifies the spectrum envelope of a number of low band signals so that the modified low band signal can be used to generate the high band signal, i. a number of high-frequency subband signals that do not show discontinuities, in particular discontinuities at the gluing boundaries. With reference to Fig. 1c, the additional gain control step 204 causes the spectrum envelope 101, 111 of the low band signal to be modified such that there are no or limited discontinuities in the generated high band signal 105, 115.

Modification of the spectrum envelope of the low band signal can be performed by applying a gain curve to the spectrum envelope of the low band signal. Said gain curve may be determined by the gain curve determiner 400 illustrated in FIG. 4. The module 400 receives as input the QMF data 402 corresponding to the low band signal used to recreate the high band signal. In other words, a number of low frequency subband signals are input to the gain curve determiner 400 . As already noted, only a subset of the available QMF subbands of the low band signal can be used to generate the high band signal, i. only a subset of the available QMF subbands can be input to gain curve determiner 400 . In addition, module 400 may receive optional control data 404, such as control data sent from the associated encoder. Module 400 outputs a gain curve 403 that is intended to be applied during the high frequency regeneration process. In one embodiment, the gain curve 403 is applied to the QMF subbands of the low band signal that are used to generate the high band signal. Those. the gain curve 403 may be applied in the copy-up process of the HFR process.

The optional control data 404 may include coarse spectral envelope resolution information to be estimated at module 400 and/or information on the suitability of applying the gain control process. Thus, the control data 404 can control the amount of additional processing during the gain control process. The control data 404 may also bypass the additional gain control process if the signals are not well suited for estimating the coarse spectral envelope, for example if the signals include single sine waves.

On FIG. 5 describes a more detailed view of the module 400 of FIG. 4. The QMF data 402 of the lowband signal is input to an envelope estimator 501, which estimates the spectrum envelope, for example, on a logarithmic energy scale. The spectrum envelope then enters a module 502 which estimates the coarse spectrum envelope from the high resolution spectrum envelope (in frequency) received from the envelope estimator 501 . In one embodiment of the invention, this is done by approximating a low order polynomial, i.e. polynomial with an order in the interval, for example, 1, 2, 3, 4, to the spectral envelope data. The coarse envelope of the spectrum can also be determined by performing a moving average operation of the envelope of the high-resolution spectrum on the frequency axis. The determination of the coarse spectrum envelope 301 of the low band signal is shown in FIG. 3. It can be seen that the absolute spectrum 302 of the low-frequency band signal, i.e. energies of the QMF bands 302 are approximated by the coarse envelope 301 of the spectrum, i. e. by means of a frequency-dependent curve approximating the envelope of the spectrum for a number of low-frequency subband signals. In addition, it is shown that only 20 QMF subband signals are used to generate the high band signal, i.e. only a portion of the 32 QMF subbands is used in the HFR process.

The method used to determine the coarse spectrum envelope from the high resolution spectrum envelope, and in particular the order of the polynomial that is approximated to the high resolution spectrum envelope, may be controlled by control data 404. The order of the polynomial may be a function of the size of the frequency band 302 of the lowband signal, for which it is necessary to determine the coarse envelope 301 of the spectrum, and/or a function of other parameters related to the overall coarse shape of the spectrum of the corresponding frequency band 302 of the low band signal. The polynomial fit calculates a polynomial that fits the data in terms of the standard error. The following preferred embodiment of the invention is described by means of Matlab code:

In the code above, the input is the spectrum envelope (LowEnv) of the low band signal, obtained by averaging the QMF subband samples per subband over the time period corresponding to the current time frame of the data being affected by the subsequent envelope control. As noted above, the low band signal gain adjustment processing may be performed on different time grids. In the example above, the estimated absolute envelope of the spectrum is expressed in the logarithmic domain. The data is approximated by a low-order polynomial, in the example above, a polynomial of order 3. For this polynomial, the gain curve (GainVec) is calculated from the difference between the average energy of the low band signal and the curve (lowBandEnvSlope) obtained from the polynomial fitting the data. In the above example, the gain curve determination operation is carried out in the logarithmic region.

The gain curve calculation is performed by the gain curve calculator 503 . As noted above, the gain curve may be determined from the average energy of the low band signal portion used to regenerate the high band signal and from the spectral envelope of the low band signal portion used to regenerate the high band signal. In particular, the gain curve can be determined from the difference between the average energy and the coarse envelope of the spectrum, represented, for example, by a polynomial. Those. the calculated polynomial may be used to determine a gain curve that includes a single gain value, also referred to as a spectrum gain, for each associated QMF subband of the low band signal. This gain curve includes gain values that are then used in the HFR process.

,As an example, the HFR generation process according to MPEG-4 SBR is described below. The generated HF signal can be obtained from the following formula (see MPEG-4 Part 3 (ISO/IEC 14496-3), sub-part 4, section 4.6.18.6.2, which is incorporated herein by reference):

,

,where p is the subband index of the low band signal, i.e. p specifies one of a number of low frequency subband signals. The above formula for generating HF can be replaced by the following formula, which, in combination with generating HF, performs gain control:

,

where the gain curve is called preGain ( p ).

обозначает дискретное значение в момент времени l сигнала низкочастотного поддиапазона, имеющего индекс поддиапазона р. Данное дискретное значение в сочетании с предшествующими дискретными значениями используется для генерирования дискретного значения сигнала высокочастотного поддиапазона

, имеющего индекс поддиапазона k.Further details of the copy-up process, such as those relating to the relationship between p and k , are defined in the aforementioned MPEG-4 Part 3 document. In the above formula,

denotes a discrete value at time l of the low-frequency subband signal having subband index p . This sample value, in combination with previous samples, is used to generate a sample value of the high-frequency subband signal.

, which has subrange index k .

It should be noted that the gain control feature can be used in any high frequency reconstruction system based on a filter bank. This is illustrated in FIG. 6, where the present invention is part of a separate HFR block 601 that operates on a narrowband, or low frequency, signal 602 and outputs a wideband, or high frequency, signal 604. Module 601 may receive additional control data 603 as input, where control data 603 may , among other things, to determine the amount of processing applied to the described gain adjustment, as well as information about the target spectrum envelope of the high band signal. However, these parameters are only examples of optional control data 603. In one embodiment of the invention, the corresponding information can also be obtained from the narrowband signal 602 included in the module 601, or by other means. Those. control data 603 may be determined in module 601 based on information supplied to module 601. analysis/synthesis filter blocks, or transforms, may be placed outside the HFR block 601.

As noted above, it may be useful to signal the activation of gain control processing in the bitstream from the encoder to the decoder. For some signal types, such as a single sine wave, gain control processing may not be appropriate, and therefore it may be useful to allow the encoder/decoder system to turn off additional processing in order not to introduce undesirable properties into signals in such extreme cases. To this end, the encoder may be configured to analyze the audio signals and to generate control data that turns on and off the gain control processing in the decoder.

On FIG. 7, the proposed gain control step is included in the high frequency reconstruction block 703, which is part of the audio codec. One example of such an HFR block 703 is the MPEG-4 Spectral Band Replication tool used as part of the High Efficiency AAC or MPEG-D USAC (Unified Speech and Audio Codec) codec. In this embodiment, the bitstream 704 is received by the audio decoder 700. The bitstream is demultiplexed in the demultiplexer 701. The SBR portion of the bitstream 708 is provided to the SBR module or HFR unit 703, and the core encoder related bitstream 707, such as AAC or USAC core decoder data, is sent to the core encoder module 702. In addition, a low frequency, or narrowband, signal 706 passes from the base decoder 702 to the HFR block 703. The present invention, for example, in accordance with the system described in FIG. 2 is included in the HFR block 703 as part of the SBR process. The HFR block 703, using the processing described herein, outputs a wideband, or high frequency, signal 705.

On FIG. 8 describes one embodiment of the high frequency reconstruction module 703 in more detail. Fig. 8 illustrates that HF (high frequency) signal generation can be obtained from different HF generating modules at different times. The HF generation may be based on either the QMF-based copy-up converter 803, or the HF generation may be based on the FFT-based harmonic converter 804. In both HF signal generation modules, the low band signal is processed 801, 802 as part of the HF generation to determine the gain curve that is applied in the copy up process 803 or harmonic transformation 804. The outputs of the two converters are selectively input to an envelope controller 805. The decision as to which transducer signal to use is controlled by bit stream 704 or 708. It should be noted that, due to the copy-up nature, the shape of the lowband signal spectrum envelope in a QMF-based transducer is maintained more distinctly than with a harmonic transducer. Typically, this results in more pronounced discontinuities in the spectrum envelope of the high-band signal when using copy-up transducers. This is illustrated in the top and bottom panels of FIG. 1a. Accordingly, it may be sufficient to introduce gain control only into the QMF-based copy-up method performed in module 803. However, applying gain control to the harmonic transformation performed in module 804 may also be useful.

On FIG. 9 describes the corresponding encoder module. The encoder 901 may be configured to analyze a particular input signal 903 and to determine the amount of gain control processing suitable for a particular type of input signal 903. In particular, the encoder 901 may determine the amount of discontinuity in the high frequency subband signal that will be caused by the HFR block 703 in the decoder. To this end, encoder 901 may include an HFR block 703, or at least corresponding portions of an HFR block 703. Based on the analysis of the input signal 903, control data 905 for the corresponding decoder can be generated. Information 905 regarding the gain adjustment to be performed in the decoder is combined in the multiplexer 902 with the audio bitstream 906, thus forming a complete bitstream 904, which is transmitted to the corresponding decoder.

On FIG. 10 shows the output spectrum of a real signal. On FIG. 10a shows the output of an MPEG USAC decoder decoding a mono bitstream at a bit rate of 12 kbps. This snippet of the actual signal is the vocal part of the a cappella recording. The abscissa corresponds to the time axis, while the ordinate corresponds to the frequency axis. Comparison of the spectrogram of FIG. 10a from Fig. 10c, which shows the corresponding spectrogram of the original signal, clearly shows the presence of holes (see references 1001, 1002) occurring in the spectrum of the fricative parts of the vocal passage. On FIG. 10b shows the output of an MPEG USAC decoder incorporating the present invention. It can be seen from the spectrogram that the holes in the spectrum have disappeared (see references 1003, 1004 corresponding to references 1001, 1002).

The complexity of the proposed gain control algorithm is calculated as a weighted MOPS where functions such as POW/DIV/TRIG are weighted as 25 operations and all other operations are weighted as one operation. Under these assumptions, the computed complexity is approximately 0.1 WMOPS and negligible RAM/ROM usage. In other words, the proposed gain control processing requires low processing and memory performance.

The present document describes a method and system for generating a high band signal from a low band signal. The method and system are adapted to generate a high frequency band signal containing little or no discontinuity in the spectrum, thereby improving the perceptual performance of high frequency reconstruction methods and systems. The method and system can easily be incorporated into existing audio encoding/decoding systems. In particular, the method and system can be incorporated into the envelope adjustment processing of existing audio encoding/decoding systems without the need to modify them. In particular, this applies to the limiting function and the interpolation function of the envelope adjustment processing, which can perform their intended functions. The described method and system as such can be used to regenerate high frequency band signals with little or no discontinuity and low noise. In addition, the use of control data is described, where the control data can be used to adapt the parameters of the described method and system (and computational complexity) to the type of audio signal.

The methods and systems described herein may be implemented as software, firmware, and/or hardware. Some components may be implemented, for example, as software running on a digital signal processor or microprocessor. Other components may be implemented, for example, as hardware or as ASICs. The signals encountered in the described methods and systems may be stored in memory media such as random access memory or optical storage media. They may be transmitted over networks such as radio networks, satellite networks, wireless networks, or wired networks such as the Internet. Typical devices using the methods and systems described herein are portable electronic devices or other equipment at the user's premises that are used for memory storage and/or processing of audio signals. The methods and systems may also be applied to computer systems, such as Internet web servers, that store and make available for download audio signals, such as music signals.

Claims (11)

1. A system (601, 703) configured to generate a set of high frequency subband audio signals (604) covering a high frequency interval based on a set of low frequency subband signals (602), where the system (601, 703) comprises: – means for receiving a number of signals (602) of low-frequency subbands; – means for receiving a set of target energies, where each target energy covers a different target interval (130) within the high frequency interval and is indicative of the required energy of one or more high frequency subband signals lying within the target interval (130); – means for generating a set of high frequency subband signals (604) from a set of low frequency subband signals (602) and from a set of spectrum gains, respectively, associated with a set of low frequency subband signals (602); And – means for regulating the energy (203) of a number of high frequency subband signals (604) using a set of target energies, the means for regulating including means for limiting the regulation of the energy of the high frequency subband signals (604) lying within the limit interval (135). 2. A method for generating a series of audio signals (604) of high frequency subbands covering a high frequency interval, based on a series of signals (602) of low frequency subbands, where the method includes: – reception of a number of signals (602) of low-frequency subbands; – receiving a set of target energies, where each target energy covers a different target interval (130) within the high frequency interval and is indicative of the required energy of one or more high frequency subband signals (604) lying within the target interval (130); – generating a set of high frequency subband signals (604) from a set of low frequency subband signals (602) and from a set of spectrum gains, respectively, associated with a set of low frequency subband signals (602); And – regulation of the energy of a number of signals (604) of high frequency subbands using a set of target energies, while the regulation of the energy of a number of signals of high frequency subbands includes limiting the regulation of the energy of signals (604) of high frequency subbands lying within the limit interval (135). 3. A storage medium that includes a program implemented in software, adapted for execution on a processor and for performing the steps of the method according to claim 2 when carried out on a computing device.

Images