WO2023117146A1 - Procédé et appareil pour remplissage spectral amélioré dans le temps d'un espace spectral dans codage audio à l'aide d'un filtrage - Google Patents

Procédé et appareil pour remplissage spectral amélioré dans le temps d'un espace spectral dans codage audio à l'aide d'un filtrage Download PDF

Info

Publication number
WO2023117146A1
WO2023117146A1 PCT/EP2022/052151 EP2022052151W WO2023117146A1 WO 2023117146 A1 WO2023117146 A1 WO 2023117146A1 EP 2022052151 W EP2022052151 W EP 2022052151W WO 2023117146 A1 WO2023117146 A1 WO 2023117146A1
Authority
WO
WIPO (PCT)
Prior art keywords
spectral
value
frequency
filling
audio
Prior art date
Application number
PCT/EP2022/052151
Other languages
English (en)
Inventor
Christian Helmrich
Eleni FOTOPOULOU
Richard Füg
Goran MARKOVIC
Markus Multrus
Franz REUTELHUBER
Original Assignee
Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. filed Critical Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
Priority to PCT/EP2022/087818 priority Critical patent/WO2023118605A1/fr
Priority to TW111149823A priority patent/TW202333143A/zh
Publication of WO2023117146A1 publication Critical patent/WO2023117146A1/fr

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/18—Vocoders using multiple modes
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/0212—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using orthogonal transformation
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/038—Speech enhancement, e.g. noise reduction or echo cancellation using band spreading techniques
    • G10L21/0388—Details of processing therefor

Definitions

  • Embodiments according to the invention are related to methods and apparatuses for spectrotemporally improved spectral gap filling in audio coding using a filtering.
  • Embodiments according to the invention are related to methods and apparatuses for spectrotemporally improved spectral gap filling in audio coding using different noise filling methods.
  • Embodiments according to the invention are related to methods and apparatuses for spectrotemporally improved spectral gap filling in audio coding using a tilt.
  • Embodiments according to the invention are related to decoders, encoders and methods using a spectral tilt information for audio coding.
  • FIG. 1 is a diagrammatic representation of an exemplary embodiment of the invention.
  • a Perceptual Noise Substitution (PNS) decoder may insert pseudo-random values into zero-quantized bands, scaled such that the inserted signal energy matches the signaled target energy.
  • PPS Perceptual Noise Substitution
  • many bits may have to be reserved for a signaling of the zero-quantized band energies.
  • only fully zero-quantized spectral bands may be substituted, hence such an approach may lack flexibility.
  • noise filling approaches may allow to replace zero-quantized spectral coefficients with pseudo-random values upon decoding, above a certain "noise fill start frequency", however a large signaling overhead may therefore be required, especially when many bands are zero-quantized.
  • inventive embodiments will be explained in the context of a decoder and other inventive embodiments will be explained in the context of an encoder. It is to be noted that features, functionalities and details that are explained in the context of a decoder may be implemented analogously in or added to or used with a corresponding encoder, individually or taken in combination. Vice versa, features, functionalities and details as disclosed for inventive encoders may be incorporated in corresponding decoders. Accordingly, it is to be noted that decoders and corresponding encoders (or vice versa) may be based on similar and/or equivalent inventive concepts and may hence comprise corresponding advantages.
  • Embodiments according to a first aspect of the invention comprise an audio decoder for providing a decoded audio information on the basis of an encoded audio information, wherein the audio decoder is configured to derive a spectral tilt information, e.g. T’st, from the encoded audio information. Furthermore, the audio decoder is configured to use filling values, e.g. gap fill coefficients; e.g. noise values of a noise filling; e.g. gap filling values of an intelligent gap filling, in order to fill spectral holes of a decoded set of spectral values.
  • filling values e.g. gap fill coefficients
  • noise values of a noise filling e.g. gap filling values of an intelligent gap filling
  • the audio decoder is configured to apply, e.g. in a multiplicative manner, a frequency variable scaling, a spectral tilt of which is determined by the spectral tilt information, to the filling values, e.g. to the noise samples substituted for the zero- quantized samples, wherein, as an example, the spectral tilt of the frequency variable scaling is determined by the spectral tilt information.
  • an audio decoder may be configured to derive a spectral tilt information from an encoded audio information based on which the frequency variable scaling may be determined.
  • one main idea according to embodiments of the first aspect of the invention is a calculation and, for example, low-bit-rate signaling of a difference curve, for example in logarithmic intensity domain, between a frame's (and/or a subframe's), e.g. true, spectral envelope (e.g. its input signal envelope) and the frame's (and/or subframe's) masking envelope, e.g. its noise shaping envelope. Since the masking envelope may be transmitted to the decoder, e.g. additional, transmission of the difference may allow to, in a spectral hole filling procedure, e.g. in a gap or noise filling decoding procedure, reconstruct the, e.g.
  • a spectral hole filling procedure e.g. in a gap or noise filling decoding procedure
  • the difference curve may be characterized by the spectral tilt information.
  • a good accuracy and/or quality of the audio information may be achieved, for example with few side information bits.
  • the spectral tilt information may, for example, be a frame-wise and/or a subframe-wise spectral tilt information.
  • the spectral tilt information may comprise a tilt index, e.g. t s t, based on which, as an example, an information T’ s r may be determined, wherein T’ S f may, for example, be multiplied with a frequency dependent term, e.g. f, for example, in order to apply the frequency variable scaling to the filling values.
  • T’ S f may, for example, be multiplied with a frequency dependent term, e.g. f, for example, in order to apply the frequency variable scaling to the filling values.
  • no explicit transmission of target energies in zero-quantized nonoverlapping frequency ranges may be transmitted, hence signaling effort may be kept at a low level.
  • a spectral envelope of the audio information may be recovered from a masking envelope (e.g. noise shaping envelope, e.g. a masking envelope corresponding to or associated with scaling values or scaling factors of the frames and/or subframes) of the audio information with only few additional signaling bits.
  • the audio decoder is configured to derive a noise level information, e.g. L s t, from the encoded audio information and the audio decoder is configured to use the noise level information, for example in addition to the frequency-variable scaling, in order to obtain the filling values.
  • the noise level information may, for example, be derived or reconstructed from a noise level index, e.g. an N-bit noise level index ( I S f ⁇ 2 N .
  • the noise level information and/or the noise level index may, for example, be transmitted from a corresponding encoder to the decoder.
  • the noise level information and/or the noise level index may, for example, comprise an information about the spectral tilt information (for example, i.e. further information about the difference curve), e.g. an offset, e.g. O s t.
  • the decoder may be configured to derive an information about the spectral tilt information (for example, i.e. further information about the difference curve) from the noise level information and/or from the noise level index.
  • the inventors recognized that using the noise level information for decoding may allow to determine improved filling values, e.g. allowing for a good reconstruction of the encoded audio signal.
  • the audio decoder is configured to apply the frequency variable scaling, such that the frequency variable scaling describes, e.g. within a tolerance of +/-3dB or +/-2dB or +/-1 dB, a linear decrease of intensity, e.g. of the filling values, with increasing frequency on a logarithmic intensity scale.
  • an improved reconstruction of the spectral envelope of the audio information may be achieved.
  • an influence of a pre- emphasis tilt applied during the calculation of a masking envelope of the audio information may be compensated, such that the spectral envelope may be recovered, at least approximately.
  • the spectral tilt information describes a spectral tilt in a logarithmic domain, for example wherein a spectral tilt, e.g. the spectral tilt information, may, for example, be used in a logarithmic domain and/or in a linear domain. It is to be noted that embodiments according to the invention are not limited to spectral tilt information in a logarithmic domain.
  • the spectral tilt information may, for example, be used in a logarithmic domain and/or in a linear domain. Usage in the logarithmic domain may allow a computation of the, e.g. spectrally tilted, filling values with low computational costs.
  • the spectral tilt information describes a line function with a spectral tilt in a logarithmic domain.
  • the inventors recognized that this form of function, with the spectral tilt in the logarithmic domain, allows for an efficient decoding of the audio information with good accuracy.
  • the audio decoder is configured to obtain scaling values for the frequency-variable scaling in a logarithmic domain, and the audio decoder is configured to convert the scaling values for the frequency-variable scaling from the logarithmic domain to a linear domain, e.g. using an exponential function; e.g. using an exponential function for a basis of 10; e.g. using a function of the form 10 x .
  • a calculation domain e.g. a logarithmic domain or a linear domain
  • a linear domain may for example, be changed or adapted for different processing steps.
  • the inventors recognized that such a switching or changing of domains may improve the flexibility of inventive audio coding concepts.
  • computational costs may be reduced by performing different processing steps in respective, suitable domains.
  • the audio decoder is configured to obtain scaling values for the frequency variable scaling in dependence on a product of a tilt value, e.g. T’ S f, which is based on the tilt information, and of a frequency value, e.g. f, e.g. a frequency value describing the frequency, or a frequency value describing a frequency offset relative to a reference value.
  • the tilt value may, for example be scaled by a constant, e.g. an additional constant, in order to maintain, on average, a value range of a noise level information, e.g. L S f.
  • a constant e.g. an additional constant
  • scaling values for the frequency variable scaling may, for example, be obtained with low computational effort using the product of the tilt value and the frequency value.
  • the audio decoder is configured to obtain a plurality of scaling values for the frequency variable scaling associated with different frequency bands, e.g. such that the scaling values are associated with different frequency bands.
  • the inventors recognized that using scaling values associated with different frequency bands, a decoding of the audio information may be improved in, e.g., complexity or flexibility.
  • the audio decoder is configured to obtain scaling values for the frequency variable scaling using start frequencies of respective frequency bands or using center frequencies of respective frequency bands; wherein, for example, a scaling value associated with a first frequency band is obtained using a multiplication of a (e.g. lower) start frequency of the first frequency band and a tilt value, and wherein, for example, a scaling value associated with a second frequency band is obtained using a multiplication of a (e.g.
  • a scaling value associated with a first frequency band is obtained using a multiplication of a center frequency of the first frequency band and a tilt value
  • a scaling value associated with a second frequency band is obtained using a multiplication of a center frequency of the second frequency band and the tilt value
  • embodiments according to the invention are not limited to a specific choice of a frequency representation of a respective frequency band. As explained before, start frequencies and/or center frequencies may be used. However, other, e.g. applications specific advantageous choices of frequency band information may be implemented. Hence, an inventive concept according to embodiments may provide a high flexibility.
  • the audio decoder is configured to obtain scaling values for the frequency variable scaling using start frequency bin indices of respective frequency bands or using center frequency bin indices of respective frequency bands; wherein, for example, a scaling value associated with a first frequency band is obtained using a multiplication of a (e.g. lower) start frequency bin index of the first frequency band and a tilt value, and wherein, for example, a scaling value associated with a second frequency band is obtained using a multiplication of a (e.g.
  • a scaling value associated with a first frequency band is obtained using a multiplication of a center frequency bin index of the first frequency band and a tilt value
  • a scaling value associated with a second frequency band is obtained using a multiplication of a center frequency bin index of the second frequency band and the tilt value
  • frequency bin indices e.g. instead of frequency values, may allow to reduce computational costs.
  • the audio decoder is configured to obtain filling values using a noise intensity information, e.g. L S f; e.g. using a frequency-independent noise scaling value, which may, for example, be derived from the encoded audio information; which may, for example, be derived from k
  • the audio decoder may, for example be configured to determine or obtain filling values using a noise level information and/or a noise intensity information.
  • the decoder may, for example, be configured to derive the noise intensity information.
  • the noise level information may, for example, be equal to the noise intensity information.
  • the audio decoder is configured to obtain a filling value using a multiplication of a noise value, of a frequency-independent noise scaling value, e.g. L S f, and of a frequency-variable noise scaling value, e.g. 10 rsf , f , which is determined considering the spectral tilt; wherein the noise value is a random noise value or a pseudo-random noise value, e.g. having a predetermined amplitude or having an amplitude within a predetermined amplitude range.
  • a frequency-independent noise scaling value e.g. L S f
  • a frequency-variable noise scaling value e.g. 10 rsf , f
  • the frequency-variable noise scaling value may allow to shape, e.g. tilt with respect to frequency, a masking envelope of the audio information in order to better approximate the spectral envelope of the originally encoded audio information.
  • the audio decoder is configured to apply a scaling, which is based on a masking envelope, to, e.g. non-zero, decoded spectral values and to filling values, e.g. such that, in effect, a masking envelope is applied to the full spectrum, optionally including the filling values.
  • an application of the inventive scaling may improve the decoded audio information when not only filling values, but other decoded spectral values are affected by the scaling.
  • the decoded spectrum of the audio information may, for example, be adapted, e.g. tilted in dependence on the frequency.
  • FIG. 1 For embodiments according to the first aspect of the invention, comprise an audio encoder for providing an encoded audio information on the basis of an input audio information, wherein the audio encoder is configured to encode a plurality of quantized spectral values and wherein the audio encoder is configured to determine a spectral tilt information, (e.g. a spectral tilt information describing a line function with a spectral tilt in a logarithmic domain e.g. wherein a spectral tilt, e.g. the spectral tilt information may, for example, be used in a logarithmic domain and/or in a linear domain) on the basis of a spectral energy information, e.g. a spectral envelope, and a masking envelope information, e.g. such that the spectral tilt information describes an average frequency variation of a difference between the spectral energy and the masking envelope.
  • the audio encoder is configured to encode the spectral tilt information.
  • the spectral tilt information may describe a shape difference between the spectral energy of the audio information and the masking envelope for encoding the audio information.
  • This shape difference may, for example, be expressed in the form of a frequency dependent tilt (in the frequency - amplitude plane).
  • the spectral tilt information may be transmitted to a corresponding decoder, and the spectral tilt information may, for example, be used as an correction factor to adapt a transmitted masking envelope in order to better reconstruct the spectral envelope of the audio information.
  • the audio encoder is configured to determine the spectral tilt information, such that the spectral tilt information describes a frequency variation of a difference between the spectral energy information, e.g. a “true spectral envelope” or a smoothened (e.g. in a frequency direction) version of the spectral values, and the masking envelope information, e.g. represented by scale factors or by one or more prediction coefficients, over frequency, e.g. such that the tilt information describes an average of a frequency variation, or, for example, such that the tilt information describes a tilt of a (e.g. linear) regression line of a difference between the spectral energy information and the masking envelope information over frequency.
  • the spectral tilt information describes a frequency variation of a difference between the spectral energy information, e.g. a “true spectral envelope” or a smoothened (e.g. in a frequency direction) version of the spectral values
  • the masking envelope information e.g. represented by scale factors or by one
  • an idea according to embodiments of the invention is a calculation and low-bit-rate signaling of a frequency variation, e.g, of a difference curve, e.g. in logarithmic intensity domain, between a frame's (and/or a subframe's), e.g. true, spectral energy, e.g. spectral envelope, e.g. its input signal envelope and the frame's (and/or the subframe’s) masking envelope.
  • This information may be transmitted using the spectral tilt information. Therefore, as an example, by providing the masking envelope and the spectral tilt information and hence an information about said difference curve a reconstruction of the spectral energy of the audio information may be performed with good accuracy and with low signaling effort.
  • noise filled or spectral gap filled coefficients may, for example, be adapted or corrected using the spectral tilt information, therefore reducing a difference between the “original’’ spectrum and the reconstructed or decoded spectrum of the audio information.
  • the spectral tilt information describes a line function with a spectral tilt in a logarithmic domain.
  • the inventors recognized that this may allow to signal a correction information for the masking envelope to better approximate the original spectrum of the audio information with few signaling bits and good accuracy.
  • the audio encoder is configured to determine the spectral tilt information in a logarithmic domain, e.g. using a logarithmized (e.g. frequency-dependent) representation of a spectral energy information and, for example, using a logarithmized (e.g. frequency-dependent) representation of the masking envelope information.
  • the audio encoder is configured to obtain the spectral tilt information using a linear regression, wherein the spectral tilt information may, for example, be a regression coefficient obtained by the linear regression, e.g. of an evolution of a difference between the (true) spectral envelope and the masking envelope over frequency in a logarithmic intensity domain.
  • the spectral tilt information may, for example, be a regression coefficient obtained by the linear regression, e.g. of an evolution of a difference between the (true) spectral envelope and the masking envelope over frequency in a logarithmic intensity domain.
  • a linear regression may allow to approximate a correction term or difference term or, e.g. monotonic, difference curve between the (e.g. true) spectral envelope and the masking envelope with limited complexity and good approximation results.
  • the spectral tilt information may, for example, be obtained.
  • the correction term or difference term or , e.g. monotonic, difference curve may be the spectral tilt information.
  • the audio encoder is configured to obtain the spectral tilt information on the basis of spectral-band- wise, e.g. summed-up, energy values or spectral band wise root-mean-square values representing an energy of spectral values in a plurality of respective spectral bands and on the basis of spectral band-wise, e.g. summed-up, energy values or spectral-band-wise root-mean-square values representing, e.g. an energy level of the masking threshold in a plurality of respective spectral bands.
  • spectral-band- wise e.g. summed-up, energy values or spectral band wise root-mean-square values representing an energy of spectral values in a plurality of respective spectral bands
  • spectral band-wise representing, e.g. an energy level of the masking threshold in a plurality of respective spectral bands.
  • the audio encoder is configured to determine separate spectral tilt information, e.g. separate spectral tilt values, for different audio frames and/or for different audio subframes.
  • frame-wise or subframe-wise spectral tilt information may allow to determine an effective correction information, e.g. a spectral tilt to be transmitted to a corresponding decoder, in order to improve a fitting of a decoded spectrum of the audio information to the original spectrum of the audio information.
  • an effective correction information e.g. a spectral tilt to be transmitted to a corresponding decoder
  • the audio encoder is configured to determine a difference value (e.g. O S f or T s t, e.g. an offset value Ost; or, for example, a tilt value T S f, e.g. a value which is quantized e.g. to a value t sf ; and/or which may, for example, be transmitted, e.g. to a noise-filling decoder; and/or which may, for example, be used, e.g.
  • a difference value e.g. O S f or T s t, e.g. an offset value Ost
  • a tilt value T S f e.g. a value which is quantized e.g. to a value t sf
  • a noise-filling decoder e.g.
  • a noise filling encoder representing, in the form of a single value, a difference between the spectral energy information and the masking envelope information over a frequency range comprising a plurality of spectral bins, e.g. over a frequency band, or even over a plurality of spectral bands, or even over all of the frequency bands.
  • the audio encoder is configured to obtain a noise level information, which may, for example, describe a noise level over a plurality of spectral bands, or even over all frequency bands, e.g. l S f, in dependence on the difference value.
  • O S f may be an offset, which may not really be needed or which may not need to be encoded (but may optionally be used).
  • T S f may be the value which is quantized (e.g. into t S f) and which may be transmitted, and which may, for example, be used (e.g. in a negated form) in a noise filling encoder (and/or in a noise filling decoder).
  • a tilt information may be determined that may describe a tilt of the masking envelope over frequency with respect to an original spectrum of the audio information.
  • a decoder sided correction of, for example, zero quantized spectral coefficients based on filling values, adapted according to the masking envelope and corrected using the tilt information may allow for an efficient audio information reconstruction.
  • the audio encoder is configured to obtain the difference value, (e.g. Osf or Tsf, e.g. an offset value Osf; or, for example, a tilt value Tsf, e.g. a value which is quantized e.g. to a value tsf; and/or which may, for example, be transmitted, e.g. to a noise-filling decoder; and/or which may, for example, be used, e.g. in a negated form, in a noise filling encoder) using a linear regression, e.g. using the linear regression mentioned above.
  • the difference value e.g. Osf or Tsf, e.g. an offset value Osf
  • a tilt value Tsf e.g. a value which is quantized e.g. to a value tsf
  • a linear regression e.g. using the linear regression mentioned above.
  • a difference between a original, e.g. “true” audio signal spectral envelope and a masking envelope may comprise an approximately linear, e.g. in logarithmic frequency domain, characteristic.
  • an intensity difference between the true spectral envelope and masking envelope may change monotonically with frequency.
  • the monotonic difference curve may resemble a straight line most of the time.
  • the audio encoder is configured to encode the spectral tilt information using three bits.
  • the audio encoder is configured to encode the spectral tilt information such that the encoded spectral tilt information always represents a negative spectral tilt, e.g. a decrease with increasing frequency.
  • a negative spectral tilt may, for example, allow a good adaptation or correction or improvement of the reconstructed audio information.
  • a correction of filling values with a negative spectral tilt information may compensate for an undesirable influence of a pre-emphasis.
  • the audio encoder is configured to perform the following functionality for one or more frames or subframes sf, e.g. audio frames or audio subframes:
  • Further embodiments according to the first aspect of the invention comprise a method for providing a decoded audio information on the basis of an encoded audio information, the method comprising deriving a spectral tilt information, e.g. T’ S f, from the encoded audio information and using filling values (e.g. gap fill coefficients; e.g. noise values of a noise filling; e.g. gap filling values of an intelligent gap filling), in order to fill spectral holes of a decoded set of spectral values.
  • the method further comprises applying, e.g. in a multiplicative manner, a frequency variable scaling, a spectral tilt of which is determined by the spectral tilt information, to the filling values, e.g. to the noise samples substituted for the zero-quantized samples.
  • Embodiments according to a second aspect of the invention comprise an audio decoder for providing a decoded audio information on the basis of an encoded audio information, wherein the audio decoder is configured to fill spectral holes of a decoded set of spectral values, e.g. using a substitution of spectral coefficients quantized to zero on the basis of respective filling values. Furthermore, the audio decoder is configured to obtain a prediction lag information, e.g. a frequency domain long-term-prediction lag value p 8 r; e.g. a prediction lag information indicating a prediction period in a frequency direction, e.g. a spectral (LTP, e.g. Long-Term Prediction) distance value p s r, e.g. from a bitstream or from the encoded audio information.
  • a prediction lag information e.g. a frequency domain long-term-prediction lag value p 8 r
  • a prediction lag information e.g
  • the audio decoder is configured to switch between a first spectral filling method, e.g. a “noise filling" + FD LTP, e.g. if p S f is not zero, in which a frequency filtering or a frequency prediction, e.g. a TNS or a LTP, (e.g. a filtering in which a spectral value associated with a first frequency has an influence on a spectral value associated with a second frequency) is used to obtain filling values which are used to fill spectral holes, and one or more further spectral filling methods (e.g. the second spectral filling method or the third spectral filling method, e.g.
  • a first spectral filling method e.g. a “noise filling" + FD LTP, e.g. if p S f is not zero
  • a frequency filtering or a frequency prediction e.g. a TNS or a LTP, (
  • noise filling without FD-LTP; e.g. “gap filling”, e.g. if p S f is zero), in which no frequency filtering and no frequency prediction are used, e.g. in which neither a frequency filtering nor a frequency prediction are used, to obtain filling values which are used to fill spectral holes, in dependence on the prediction lag information.
  • An idea according to embodiments according to the second aspect of the invention is to adaptively, e.g. based on a (e.g. sub)frame's signal characteristic, switch between a first spectral filling method, e.g. a noise filling solution, and a second (or a plurality of second, e.g. a second and a third) spectral filling method, e.g. a gap filling solution.
  • the first spectral filling method may comprise a frequency filtering or a frequency prediction, e.g. a frequency-domain long-term prediction (FD-LTP)
  • the second spectral filling method may comprise no frequency filtering and no frequency prediction.
  • the decoder may, for example, switch between different methods for generating an “artificial” spectral content for the filling of zero-quantized spectral coefficients.
  • a decoder may be configured to switch or choose, depending on the prediction lag information, e.g. depending on a FD-LTP lag value p S f, between a noise filling with FD-LTP, and a tonality based gap filling without FD-LTP (e.g. similar to IGF in EVS) or a noise filling without FD-LTP (e.g. similar to that in EVS or MPEG-D).
  • the prediction lag information may, for example, comprise, e.g. only, integer values, in order to lower the computational complexity.
  • an inventive coding concept according to the second aspect of the invention may provide a good flexibility, e.g. in the switching or choice of spectral hole filling methods, in order to achieve a better coding efficiency for the audio information.
  • the prediction lag information may comprise an information about a relationship of, e.g. zero quantized, spectral coefficients of different frequency blocks.
  • the prediction lag information may comprise an information about a periodicity or about an abatement of spectral coefficients.
  • the prediction lag information may, for example, be an indicator whether a relationship between, e.g. zero quantized, spectral coefficients is sufficient or well suited, in order to reconstruct or to approximate spectral coefficients in dependency on corresponding related spectral coefficients. In such a case, a good hearing impression can, for example, be achieved although bits may be saved.
  • the inventors discovered experimentally that, for example, applause-like, rain-like, and low frequency, LF, male speech signals can benefit from improved reconstruction of the high frequency, HF, fine temporal signal envelope during decoder-side spectral hole filling, e.g. gap or noise filling.
  • the fine temporal structure of a specific (e.g. sub)frame can be parameterized by a prediction lag information, e.g. a frequency-domain long-term prediction (FD-LTP) information.
  • FD-LTP frequency-domain long-term prediction
  • prediction lag information lag and gain values e.g. FD-LTP lag and gain values
  • prediction lag information lag and gain values can, for example, be obtained, e.g. directly, in the audio codec's transform domain.
  • the choice of spectral hole filling to be applied in a decoder can be made and signaled to the decoder depending on the value of said prediction lag information, e.g. FD-LTP lag p or p S f, transmitted in the audio bitstream.
  • the audio decoder is configured to, e.g. selectively, use the first spectral filling method if the prediction lag information, e.g. a prediction lag value; e.g. a quantized FD LTP, e.g. Long- Term Prediction, lag value; e.g. p S f, is non-zero.
  • the audio decoder is configured to, e.g. selectively use the first spectral filling method if the prediction lag information, e.g. a prediction lag value; e.g.
  • a quantized FD LTP lag value e.g p sf
  • the audio decoder is configured to, e.g. selectively, use one of the one or more further spectral filling methods otherwise, e.g. if the prediction lag information is zero, or if the prediction lag information is smaller than or equal to zero.
  • the prediction lag information may allow to implement a simple distinction of cases.
  • the prediction lag information may be non-zero, or larger than zero.
  • the decoder may, for example, use the second spectral filling, e.g. in case the prediction lag information is zero, which may be associated with a small dependency between spectral coefficients.
  • the audio decoder is configured to use an encoded representation of a prediction lag value, e.g. a quantized and encoded representation, which is included in the encoded audio information, in order to obtain the prediction lag value.
  • a prediction lag value e.g. a quantized and encoded representation
  • the audio decoder is configured to determine a, e.g. final, filling value, e.g. a replacement for c(i); e.g. c(i), using a prediction or filtering, e.g. using a computation rule d*c(i) +G’ s *c(i- P’sf), such that a given filling value, e.g. 6(i), which is associated with a given frequency, e.g. with a given frequency bin, is obtained in dependence on another spectral value, e.g. c(i-P’sf), or c (i-P’sf), which is associated with a different frequency, (e.g.
  • a different frequency bin e.g. with a different frequency bin having a frequency bin index i-P’srt e.g. with a frequency or frequency bin having a spectral distance P’ S f or a spectral distance d s r from the given frequency or from the given frequency bin
  • i-P’srt e.g. with a frequency or frequency bin having a spectral distance P’ S f or a spectral distance d s r from the given frequency or from the given frequency bin
  • the audio decoder is configured to adapt a filtering strength (e.g. a weighting of a spectral value associated with the different frequency, e.g. by selectively setting the filtering strength to G’ S f or 1/2G’ S f) in dependence on an encoded or quantized spectral value (e.g. a spectral value as it is, for example originally, determined by the encoded representation of individual spectral values in the encoded audio information; e.g. by a spectral value before a noise filling is applied; e.g. by a spectral value directly after an arithmetic decoding) associated with the different frequency (e.g. with the different frequency bin; e.g. with the different frequency bin having a frequency bin index i-P’st) when using the first spectral filling method.
  • a filtering strength e.g. a weighting of a spectral value associated with the different frequency, e.g. by selectively setting the filtering strength to G’
  • a filling value, associated with a given frequency may be determined or obtained or calculated based on, or using a spectral value, which is associated with a different frequency, e.g. in case the prediction lag information is non-zero, and hence, as an example, indicating a transientness of a signal.
  • a decoding and/or a reconstruction of the audio information may be improved by adapting a filtering strength in dependence on the encoded or quantized spectral value associated with the different frequency.
  • the first spectral filling method is chosen, e.g. in case a noise filling with FD-LTP is selected (e.g. if the prediction lag information is non-zero, as an example, if the FD-LTP lag is nonzero)
  • application of a long-term predictive filter in a spectral domain (e.g. the MDCT domain) of the audio transform codec may be performed, during the decoder-side noise filling routine, e.g. depending on whether a "current" coded FD coefficient is zero and on whether a corresponding "previous" coded FD coefficient located at a distance from the current coefficient (e.g. specified by the transmitted prediction lag information, e.g. by the transmitted FD-LTP lag) is zero.
  • an infinite impulse response (HR) LTP-like filter is may be used for the filtering.
  • the filtering strength determines an impact of the other spectral value, e.g. of c(i-P’ S f), onto the given filling value.
  • the inventors recognized that adapting the impact of the other spectral value onto the given filling value based on the filtering strength may improve the quality of the decoded audio information.
  • the audio decoder is configured to adapt the filtering strength in dependence on the spectral value associated with the different frequency as it is, e.g. originally, determined by the encoded representation of individual spectral values in the encoded audio information.
  • the inventors recognized that using a value which is represented by the encoded representation for an adaptation of the filtering strength allows to use or exploit an information provided by the encoded representation rather than a filtered version thereof, which may, for example, be alternated. It has been found that using such a criterion is more reliable for the selection of a filter strength than using a criterion that is depending on a value that was already preprocessed on the decoder side.
  • the audio decoder is configured to adapt the filtering strength in dependence on the spectral value associated with the different frequency before a noise filling is applied.
  • the audio decoder is configured to adapt the filtering strength in dependence on whether the spectral value associated with the different frequency (or value) is quantized to zero or not.
  • the audio decoder is configured to adapt the filtering strength in dependence on whether a noise filling is applied to the spectral value associated with the different frequency (or value or not.
  • the filter strength adaptation may be performed based on an information whether a respective spectral value was quantized to zero, e.g. in addition to whether for the respective frequency of the spectral value a noise filling is intended to be performed or was performed. This may comprise usage flags.
  • the inventors recognized that, as an example, zero-quantized spectral values may be approximated or estimated based on or using the filtering or prediction in the frequency direction. Hence, a dependency of spectral values of different spectral values in frequency direction may, for example, be exploited.
  • the audio decoder is configured to apply the prediction or the filtering, in order to determine the given, e.g. final, filling value, e.g. 6(i), on the basis of a random or pseudo-random noise values, e.g. c(i).
  • a random or pseudo random noise value may, for example, be adapted using the prediction or the filtering, in order to calculate a e.g. final filing value that may provide a good approximation for a zero-quantized spectral value of an e.g. original e.g. input spectrum of the audio information.
  • the audio decoder is configured to perform a weighted combination of a noise value associated with the given frequency, and of a noise value associated with the other frequency in order to obtain the given, e.g. final, filling value, e.g. c(i).
  • the audio decoder may be configured to perform a combination d*c(i) + G’s*c(i-P’sf), with weight d for the noise value c(i) associated with the given frequency, and weight G’ S f for the noise value associated with the other frequency, or a combination d*c(i) + 1 /2*G' S *c(i-P’ S f), with weight d for the noise value c(i) associated with the given frequency, and weight %*G’ sf for the noise value associated with the other frequency.
  • the audio decoder is configured to perform a weighted combination of a noise value associated with the given frequency, and of a filling value associated with the other frequency in order to obtain the given, e.g. final, filling value, e.g. c(i).
  • the audio decoder is configured to adjust a weight, e.g. G’ s r or 1 /2*G’ S f, given to the noise value associated with the other frequency or the weight, e.g. G’ S f or %*G’ S f, given to the filling value associated with the other frequency in dependence on whether a noise filling has been applied for a spectral value associated with the other frequency.
  • a weight e.g. G’ s r or 1 /2*G’ S f
  • the inventors recognized that the e.g. final, filling value may, for example, be calculated using different frequency dependent quantity, e.g. a noise value associated with the given frequency, and/or associated with the other frequency, and/or a filling value associated with the other frequency.
  • an inventive concept may allow to determine or to obtain or to calculate the e.g. final filling value with good flexibility, such that, according to a specific situation, a filing value may be obtain that may be well or even best suitable for a reconstruction of the e.g. original audio information spectrum.
  • Choice of the respective quantity to be used for obtaining the e.g. final filling value may, for example, be performed based on the prediction lag information.
  • an adaptation or adjustment of a respective weight of a corresponding noise value or filling value associated with the other frequency may improve the determination of the e.g. final filling value and hence the reconstruction of the audio information.
  • the audio decoder is configured to perform a weighted combination of a noise value associated with the given frequency, and of a noise value associated with the other frequency in order to obtain the given, e.g. final, filling value, e.g. c(i).
  • the audio decoder may be configured to perform a combination d*c(i) + G’ S f*c(i-P’sf), with weight d for the noise value c(i) associated with the given frequency, and weight G’ S f for the noise value associated with the other frequency, or a combination d*c(i) + 1 Z>*G’ S f*c(i-P’ S f), with weight d for the noise value c(i) associated with the given frequency, and weight %*G’ S f for the spectral (or noise) value associated with the other frequency.
  • the audio decoder is configured to perform a weighted combination of a noise value associated with the given frequency, and of a filling value associated with the other frequency in order to obtain the given, e.g. final, filling value, e.g. c(i).
  • the audio decoder is configured to adjust a weight, e.g. G’ S f or 1 /4*G’ S f, given to the noise value associated with the other frequency or to a spectral value associated with the other frequency or to the weight, e.g. G’ S f or V ⁇ G’st, given to the filling value associated with the other frequency or to a spectral value associated with the other frequency in dependence on whether a noise filling has been applied for a spectral value associated with the other frequency.
  • a weight e.g. G’ S f or 1 /4*G’ S f
  • the audio decoder is configured to determine a spectral distance, e.g. P’ S f (e.g. a spectral distance d S f based on P’ S f), between the filling value associated with the given frequency and the other spectral value associated with the different frequency on the basis of an encoded information, e.g. an encoded value, describing the spectral distance, which is included in the encoded representation of the audio information.
  • an encoded information e.g. an encoded value, describing the spectral distance, which is included in the encoded representation of the audio information.
  • filling value e.g. a noise sample, e.g.
  • c(i) substituted for a zero-quantized sample may be filtered such that the filtering strength depends on a quantized value c(i - d S f) located at spectral distance d S f from i.
  • d S f may be equal to P’ S f-
  • the spectral distance may, for example, be used in order to improve the determination of the spectral filling values.
  • the audio decoder is configured to determine a weight, e.g. d, which is applied to the noise value associated with the given frequency, on the basis of a gain information, e.g. a gain value, e.g. g S f, which is included in the encoded representation of the audio information, wherein the weight, which is applied to the noise value associated with the given frequency, is a positive value, e.g. in a range between 0.5 and 1.
  • the inventors recognized that such a determination and application of a weight may, for example, allow to adjust the noise value associated with the given frequency, in order to better approximate an original spectral envelope of the audio information.
  • a respective noise value, or a respective filling value associated with the other frequency may, for example, be adapted with the weight that is determined in dependence on the gain information. This may allow to shape said noise or filling value to improve its matching with a corresponding spectral value of the originally encoded audio information.
  • a sign information e.g. a sign value, e.g. S S f, which is included in the encoded representation of the audio information.
  • the weight determination may, for example, be improved.
  • the sign information may allow an adaptation of a phase relation of the e.g. final filling value with respect to the noise value and/or the filling value associated with the other frequency, it may be based on.
  • c(i) designates a spectral coefficient which is obtained using a noise filling and having a spectral index i
  • d designates an attenuation coefficient
  • G’ s t designates a weight which is based on a gain value that is included in the encoded audio representation
  • c(i-P’ S f) designates a spectral coefficient (which may, for example, be obtained using a noise, or which may, for example, be obtained without using a noise filling, and which may, for example, be obtained using a prediction or a filtering) having a spectral index i-P’ S f, wherein P’ s r is a prediction parameter or a filtering parameter which is based on a prediction parameter information that is included in the encoded audio representation.
  • constant B may be chosen according to whether a given frame has more than one subframe.
  • the inventors recognized that using the above equations a good trade-off between signaling effort, complexity and effectivity of the decoding may be achieved.
  • the audio decoder is configured to mark noise-filled zero-quantized spectral coefficients, and the audio decoder is configured to selectively use a reduced filtering strength, e.g. %*G' S f, which is applied to spectral coefficients which are not marked, as noise-filled zero- quantized spectral coefficients.
  • a reduced filtering strength e.g. %*G' S f
  • the audio decoder is configured to switch between a second spectral filling method, e.g. a “noise filling”, in which random or pseudo-random filling values are used to fill spectral holes, e.g. without using a frequency filtering and without using a frequency prediction in order to obtain the filling values, and a third spectral filling method, e.g. “gap filling”, in which filling values which are obtained using a copying of non-zero spectral coefficients are used to fill spectral holes, in dependence on a prediction lag information and/or in dependence on a tonality of the audio information.
  • a second spectral filling method e.g. a “noise filling”
  • a third spectral filling method e.g. “gap filling”
  • the tonality may, for example, be judged in dependence on a presence of a tonality information and/or in dependence on a tonality information, and/or in dependence on a HPF data.
  • the second spectral filling method and the third spectral filling method are, for example, “one or more further spectral filling methods”.
  • the inventors recognized that a decoding of the audio information may, for example, be improved by switching between a usage of random or pseudo-random filling values and a copying of non-zero spectral, e.g. within a frequency distance, e.g. withing a frequency distance that is determined by the prediction lag information. Furthermore, the inventors recognized that such a switching may be performed based on the prediction lag information and/or in dependence on a tonality of the audio information.
  • the classification of the audio information or e.g. of a subframe sf as "tonal” may be performed based upon the prior-art audio tonality data, e.g., by classifying sf as "tonal” if the audio tonality data is present (e.g. the TD-LTP / HPF data is nonzero).
  • sf may only be classified "tonal” if the TD-LTP / HPF gain value is transmitted and maximum.
  • the audio decoder is configured to obtain a tonality information, e.g. a tonality value quantitatively describing a tonality of an audio content of the encoded audio information, or, for example, a tonality flag indicating whether an audio content of the encoded audio information is tonal or not, on the basis of the encoded audio information, e.g., to obtain a frame-wise or subframe-wise (e.g. audio frame wise or audio subframe wise) temporal (audio tonality) pitch information j S f from a bitstream.
  • the audio decoder is configured to switch between a second spectral filling method, e.g.
  • noise filling e.g. a noise filling which is based on random or pseudo-random noise values
  • a third spectral filling method e.g. a “gap filling”, in dependence in or in dependence on the tonality information.
  • the audio decoder is configured to obtain a prediction lag information, e.g. a frequency domain long-term-prediction lag value p S f, e.g. a spectral (LTP, e.g. Long-Term Prediction) distance value p S f, e.g. from a bitstream or from the encoded audio information.
  • a prediction lag information e.g. a frequency domain long-term-prediction lag value p S f, e.g. a spectral (LTP, e.g. Long-Term Prediction) distance value p S f, e.g. from a bitstream or from the encoded audio information.
  • the audio decoder is configured to judge, e.g. to determine or to decide, whether the audio information is tonal in dependence on a tonality information which is included in the encoded audio representation, and which may be extracted from the encoded audio information by the audio decoder, and/or in dependence on an information, e.g. a flag, indicating whether a tonality information is included in the encoded audio information, and/or in dependence on a filtering gain value and/or in dependence on a prediction gain value, e.g. a TD-LTP gain value, and/or in dependence on a time-domain post-filter gain value, e.g. a HPF gain value, e.g. a harmonic post-filter gain value.
  • an information e.g. a flag, indicating whether a tonality information is included in the encoded audio information
  • a filtering gain value and/or in dependence on a prediction gain value e.g. a TD-LTP gain value
  • an inventive decoder may comprise a good flexibility for inspecting the tonality information.
  • the audio decoder is configured to apply a high frequency noise gain adjustment for a filling of spectral holes in an upper frequency region below an, e.g. upper, noise filling end frequency.
  • a spectral energy of filling values for filling spectral holes may be adjusted to allow for a better reconstruction of the e.g. original audio input spectrum.
  • the audio decoder is configured to obtain a high frequency energy information, e.g. a high frequency energy delta value, on the basis of the encoded audio information, e.g. using a decoding of an encoded high frequency energy information value included in the encoded audio information.
  • a high frequency energy information e.g. a high frequency energy delta value
  • the high frequency energy information may represents an original energy, e.g. the original RMS energy, of the spectrotemporally normalized spectral coefficients of the audio information, e.g. slightly below the noise filling end frequency (e.g., in the 8 -10 kHz frequency range) which were quantized to zero.
  • the high frequency energy information may, for example, be quantized like the scale factors in AAC, e.g., logarithmically in steps of 1.51 dB.
  • the audio information may be decoded and/or reconstructed efficiently.
  • the gain adjustment may be performed based on the high frequency energy information.
  • the audio decoder is configured to obtain a high frequency energy delta value, e.g. nrFac S f, in dependence on a high frequency energy value, e.g. EHF S f (which may, for example, be included, in an encoded form, in the encoded audio representation), in dependence on a global gain value, e.g. GG S f (which may, for example, be included, in an encoded form, in the encoded audio representation), and in dependence on a, e.g. broadband, noise level information, which may, for example, be associated with a frequency region which is wider than the frequency region to which the high frequency energy value is associated, e.g. L s t; which may, for example, be included, in an encoded form, in the encoded audio representation.
  • the audio decoder is configured to apply the high frequency energy delta value to obtain one or more noise filling values.
  • the information about the energy value may be transmitted as a delta value relative to the global gain value, e.g. a core coder's global gain, and the noise level information, e.g. a noise level product, e.g., as a "noise gain normalized" value, e.g. GG S f*L S f.
  • This may, for example, be realized by transmitting a rounded scaled result of a logarithm of the ratio between the high frequency energy value and the product of the global gain value and noise level information.
  • a HF energy value may be obtained that allows to obtain noise filling values for filling spectral holes that may provide a good reconstruction of an e.g. original input audio spectrum.
  • the audio decoder is configured to selectively multiply one or more intermediate noise filling values which are associated with frequencies in an upper frequency region below an, e.g. upper, noise filling end frequency, with the high frequency energy delta value, e.g. while leaving noise values in a lower frequency region, below the upper frequency region, unaffected by the high frequency energy delta value.
  • a noise intensity in the upper frequency region can be adjusted, e.g. on the basis of the high frequency energy delta value which may be encoded in the bitstream. This way a hearing impression may, for example, be improved.
  • the audio decoder is configured to selectively apply the high frequency noise gain adjustment to spectral values for which a noise filling is performed, e.g. while leaving spectral values for which no noise filling is performed unaffected.
  • a good comprise between a computational effort and an optimization effort may be achieved by, for example only gain adjusting spectral values for which a noise filling is performed.
  • the audio decoder is configured to, e.g. selectively, apply the high frequency noise gain adjustment in a frequency range between 8kHz and 10kHz, e.g. on the basis of a single common high frequency energy value or on the basis of a single common high frequency energy delta value.
  • the high frequency energy value or the high frequency energy delta value represents an, e.g. original, e.g. RMS, energy of a plurality of, e.g. spectro-temporally normalized, spectral coefficients at a frequency below, and, for example, adjacent to, a noise filling end frequency or in a frequency region below, and, for example, adjacent to, the noise filling end frequency which were quantized to zero.
  • a noise in the upper frequency region can be adjusted to be close to an e.g. original, e.g. real intensity.
  • an audio encoder for providing an encoded audio information on the basis of an input audio information, wherein the audio encoder is configured to encode a plurality of quantized spectral values and wherein the audio encoder is configured to obtain, e.g. to determine, a lag value, e.g. a FD-LTP lag; e.g. a lag value P S f, which defines a characteristic of a filtering operation, e.g. in a frequency direction, or of a prediction operation, e.g. in a frequency direction, to be performed by an audio decoder for deriving one or more filling values for filling spectral holes.
  • a lag value e.g. a FD-LTP lag
  • P S f e.g. a lag value P S f
  • the audio encoder is configured to obtain, e.g. to determine, a gain value, e.g. G S f, which defines a characteristic of the filtering operation or of the prediction operation to be performed by an audio decoder for deriving one or more filling values for filling spectral holes.
  • the audio encoder is configured to, e.g. selectively, set, or, for example, change, the lag value to zero if the gain value, e.g. G sf , is smaller than a threshold value, e.g.
  • 3,or if an absolute value of the gain value is smaller than a threshold value, to thereby obtain a modified lag value, e.g. p S f O; and the audio encoder is configured to encode the determined lag value or the modified lag value, wherein, for example, the modified lag value is encoded if the gain value is modified, e.g. using 3 or 4 bits.
  • a decoder-sided filling of spectral holes of a decoded set of spectral values may be performed based on a prediction lag information.
  • the prediction lag information may correspond, e.g. may be or may comprise or may be determined using the lag value or the modified lag value.
  • a decoding and/or reconstruction of the audio information may, for example, be performed efficiently.
  • the lag value may be determined according to the gain value, which is associated with the filtering operation or of the prediction operation to be performed by an audio decoder for deriving one or more filling values for filling spectral holes.
  • a the lag value may be set to zero if the gain value is insignificant. This adaptation may yield a modified lag value.
  • the audio encoder is configured to determine the lag value and the gain value using an autocorrelation information which is applied to a set of spectral values, e.g. to a spectrotemporally normalized spectrum, e.g.
  • the lag value e.g. P S f
  • P S f the lag value
  • the autocorrelation information may be a normalized autocorrelation information.
  • the lag value (or modified lag value) the gain value and/or a sign index for the filtering and/or prediction of spectral coefficients, or corresponding indices may, for example, be calculated in a spectrotemporally normalized domain utilized e.g. before the transform coefficient quantization.
  • the audio encoder is configured to selectively encode the gain value if the encoded lag value, e.g. the lag value or the modified lag value, is non-zero.
  • a prediction or filtering of spectral coefficients may be performed if the gain value is significant, and the lag value is non-zero, hence, as an example, only in such cases signaling bits for gain value and lag value may be provided.
  • the audio encoder is configured to selectively encode a high-frequency energy value, which describes an energy in an upper portion of a spectrum, e.g. of the input audio information or of a pre-processed version thereof, if the encoded lag value is zero.
  • a high-frequency energy value may be provided, e.g. to perform a noise filling or a gap filling with a corresponding spectral energy.
  • the audio encoder is configured to selectively either encode the gain value or a high- frequency energy value, which describes an energy in an upper portion of a spectrum, e.g. of the input audio information or of a pre-processed version thereof, in dependence on the encoded lag value.
  • a signaling effort may be reduced.
  • the audio encoder is configured to encode the gain value and the high-frequency energy value using a same number of bits, wherein, for example, the gain value is encoded using one bit for the sign and one bit for the magnitude and wherein, for example, the high frequency energy value is encoded using 2 bits.
  • the inventors recognized that using a same number of bits for an encoding of the gain value and of the high-frequency energy value an interchangeable encoding may be provided, such that a decision what to encode can be taken with respect to the lag value, without having to adapt a number of bits to be encoded.
  • the audio encoder is configured to determine separate lag values and/or separate gain values for different audio frames and/or for different audio subframes.
  • the inventors recognized that frame-wise and/or subframe-wise lag values and/or gain values may improve the coding of the audio information.
  • the audio encoder is configured to obtain the lag value and/or the gain value in a transform domain, e.g. using a set of spectral values; e.g. using an analysis of a periodicity within the set of spectral values in a frequency direction.
  • the inventors recognized that a determination or obtaining of said information may be performed in a computationally efficient manner.
  • the audio encoder is configured to perform a long term transientness detection and to selectively set the lag value to zero if an audio frame or audio subframe, e.g. designated by sf, is found to be not long-term transient. Since the lag value may, for example, be an indicator for a transientness of the frame or subframe, the value may be set to zero in case no transientness is detected. Hence, the encoder may further suspend a filtering or prediction of zero quantized spectral values in the decoder based on the transientness detection, in case no transientness of the frame or subframe is detected.
  • FIG. 1 For embodiments according to the invention, comprise an audio encoder for providing an encoded audio information on the basis of an input audio information, wherein the audio encoder is configured to encode a plurality of quantized spectral values and wherein the audio encoder is configured to encode a high frequency energy value or a high frequency energy delta value.
  • the high frequency energy value or the high frequency energy delta value represents an, e.g. original, e.g. RMS, energy of a plurality of, e.g. spectro-temporally normalized, spectral coefficients at a frequency below, and, for example, adjacent to, a noise filling end frequency or in a frequency region below, and, for example, adjacent to, the noise filling end frequency which were quantized to zero.
  • the high frequency energy value (or the delta e.g. in case of differential entropy coding) may represent an original energy, e.g. the original RMS energy of the spectro-temporally normalized spectral coefficients slightly below the noise filling end frequency (e.g., in the 8 -10 kHz frequency range) which were quantized to zero.
  • an original energy e.g. the original RMS energy of the spectro-temporally normalized spectral coefficients slightly below the noise filling end frequency (e.g., in the 8 -10 kHz frequency range) which were quantized to zero.
  • the energy value may be transmitted as a delta e.g. relative to the global gain and noise level product e.g., as a "noise gain normalized" value. This may, for example, be realized by transmitting a rounded scaled result of a logarithm of the ratio between the high frequency energy value and the product of global gain and noise level.
  • the zero quantized spectral coefficients may be reconstructed, e.g. using a gap filling, such that the energy of said zero-quantized coefficients (e.g. of the original audio signal) is at least approximated.
  • the audio encoder is configured to logarithmically quantize the high frequency energy value or the high frequency energy delta value.
  • the inventors recognized that a logarithmic quantization may be performed in a computationally efficient manner.
  • the audio encoder is configured to encode a high frequency energy delta value, which describes the energy of a plurality of, e.g. spectro-temporally normalized, spectral coefficients at a frequency below, and, for example, adjacent to, a noise filling end frequency or in a frequency region below, and, for example, adjacent to, the noise filling end frequency which were quantized to zero, relative to a product of a global gain, which is encoded by the audio encoder, and of a noise level, which is encoded by the audio encoder.
  • a high frequency energy delta value which describes the energy of a plurality of, e.g. spectro-temporally normalized, spectral coefficients at a frequency below, and, for example, adjacent to, a noise filling end frequency or in a frequency region below, and, for example, adjacent to, the noise filling end frequency which were quantized to zero, relative to a product of a global gain, which is encoded by the audio encoder, and of a noise level
  • the audio encoder is configured to obtain a rounded scaled result of a logarithm of a ratio between the high frequency energy value and a product of a global gain and of a noise value, in order to encode the high frequency energy value, e.g. in the form of a high frequency energy delta value.
  • the inventors realized that the rounded scaled result may be obtained in a computationally efficient manner.
  • FIG. 1 For embodiments according to the invention, comprise a method for providing a decoded audio information on the basis of an encoded audio information, the method comprising filling spectral holes of a decoded set of spectral values, e.g. using a substitution of spectral coefficients quantized to zero on the basis of respective filling values and obtaining a prediction lag information, e.g. a frequency domain long-term- prediction lag value p S f, e.g. a prediction lag information indicating a prediction period in a frequency direction, e.g. a spectral (LTP, e.g. Long-Term Prediction) distance value psf, e.g. from a bitstream or from the encoded audio information.
  • a prediction lag information e.g. a frequency domain long-term- prediction lag value p S f
  • a prediction lag information indicating a prediction period in a frequency direction
  • LTP e.g. Long-Term Prediction
  • the method comprises a switching between a first spectral filling method, e.g. a “noise filling" + FD LTP, in which a frequency filtering or a frequency prediction, e.g. a TNS or a LTP, e.g. a filtering in which a spectral value associated with a first frequency has an influence on a spectral value associated with a second frequency, is used to obtain filling values which are used to fill spectral holes, and one or more further spectral filling methods, e.g. the second spectral filling method or the third spectral filling method, e.g. “gap filling”, in which no frequency filtering and no frequency prediction are used to obtain filling values which are used to fill spectral holes, in dependence on the prediction lag information.
  • a first spectral filling method e.g. a “noise filling” + FD LTP
  • a frequency filtering or a frequency prediction e.g. a
  • the method comprises encoding the determined lag value or the modified lag value, wherein, for example, the modified lag value is encoded if the gain value is modified, e.g. using 3 or 4 bits.
  • the audio decoder is configured to perform a weighted combination of a noise value associated with the given frequency, and of a spectral value, e.g. a noise value or a filling value, or a processed or unprocessed encoded value, associated with the other frequency or a weighted combination of a filling value associated with the given frequency, and of a spectral value, e.g.
  • the audio decoder is configured to adjust a weight given to the spectral value associated with the other frequency in dependence on whether a noise filling has been applied for the spectral value associated with the other frequency.
  • the audio encoder is configured to encode the gain value and to selectively encode a lag value, which defines a characteristic of the filtering operation or of the prediction operation to be performed by an audio decoder for deriving one or more filling values for filling spectral holes, if a quantized gain value or an encoded gain value is non-zero.
  • the audio encoder is configured to selectively encode a lag value, which defines a characteristic of the filtering operation or of the prediction operation to be performed by an audio decoder for deriving one or more filling values for filling spectral holes, if the gain value is larger than or equal to a threshold value.
  • FIG. 1 For purposes of this specification, FIG.
  • the audio decoder is configured to fill spectral holes of a decoded set of spectral values, e.g. using a substitution of spectral coefficients quantized to zero on the basis of respective filling values, using respective filling values.
  • the audio decoder is configured to determine a, e.g. final, filling value, e.g. a replacement for c(i); e.g. c(i), using a prediction or filtering, e.g.
  • a given filling value e.g. c(i) +G’ s *c(i- P’sf
  • a given filling value e.g. c(i) +G’ s *c(i- P’sf
  • a given filling value e.g. c(i) +G’ s *c(i- P’sf
  • a given filling value e.g. c(i) +G’ s *c(i- P’sf
  • c (i-P’sf) which is associated with a different frequency, e.g. with a different frequency bin, e.g. with a different frequency bin having a frequency bin index i-P’ S f
  • a frequency or frequency bin having a spectral distance P’ S f or a spectral distance d S f from the given frequency or from the given frequency bin.
  • the audio decoder is configured to adapt a filtering strength, (e.g. a weighting of a spectral value associated with the different frequency, e.g. by selectively setting the filtering strength to G’ sf or 1 /2G’ S f) in dependence on an encoded or quantized spectral value (e.g. a spectral value as it is (originally) determined by the encoded representation of individual spectral values in the encoded audio information; e.g. by a spectral value before a noise filling is applied; e.g. by a spectral value directly after an arithmetic decoding) associated with the different frequency, e.g. with the different frequency bin; e.g. with the different frequency bin having a frequency bin index i-P’ 8 f.
  • a filtering strength e.g. a weighting of a spectral value associated with the different frequency, e.g. by selectively setting the filtering strength to G’ sf or 1 /2
  • filling values may be determined or calculated or obtained using a prediction or filtering based on other spectral values, which are associated with a different frequency.
  • spectral coefficients e.g. of spectral values of different frequencies, e.g. of different frequency bands.
  • a coding effort may, for example, be reduced by taking advantage of such a correlation and/or a hearing impression may be improved.
  • filling values may be determined with a reduced amount of bits needed to be transmitted, while still providing a good representation of an originally encoded audio signal.
  • a decoding of the encoded audio representation may, for example, be improved by adapting the filtering strength in dependence on an encoded or quantized spectral value associated with the different frequency.
  • a filling value, associated with a given frequency may be determined or obtained or calculated based on, or using a spectral value, which is associated with a different frequency, e.g. in case a prediction lag information is non-zero, and hence, as an example, indicating a transientness of a signal.
  • a first spectral filling method e.g. in case a noise filling with FD-LTP is selected (e.g. if the prediction lag information is non-zero, as an example, if the FD-LTP lag is nonzero)
  • application of a long-term predictive filter in a spectral domain (e.g. the MDCT domain) of the audio transform codec may be performed, during the decoder-side noise filling routine, e.g. depending on whether a "current" coded FD coefficient is zero and on whether a corresponding "previous" coded FD coefficient located at a distance from the current coefficient (e.g. specified by the transmitted prediction lag information, e.g. by the transmitted FD-LTP lag) is zero.
  • an infinite impulse response (HR) LTP-like filter is may be used for the filtering.
  • a filtering strength may be reduced if the spectral value associated with the different frequency is comparatively large, e.g. non-zero. Accordingly, an impact of a large spectral value associated with the different frequency can be reduced, by selectively adapting the filtering strength. Accordingly, it can be avoided that a filling value or a noise value takes an excessively large value.
  • the filtering strength determines an impact of the other spectral value, e.g. of c(i-P’ S f) , onto the given filling value.
  • the filtering strength may represent a weighting factor of the other spectral value.
  • the inventors recognized that the adaptiveness of such an impact or as an example weighting, of the other spectral value may improve the decoded audio information.
  • the audio decoder is configured to adapt the filtering strength in dependence on the spectral value associated with the different frequency as it is, e.g. originally, determined by the encoded representation of individual spectral values in the encoded audio information.
  • the inventors recognized that using a value which is represented by the encoded representation for an adaptation of the filtering strength allows to use or exploit an information provided by the encoded representation rather than a filtered version thereof, which may, for example, be alternated. It has been found that using such a criterion is more reliable for the selection of a filter strength than using a criterion that is depending on a value that was already preprocessed on the decoder side.
  • the audio decoder is configured to adapt the filtering strength in dependence on the spectral value associated with the different frequency before a noise filling is applied.
  • the audio decoder is configured to adapt the filtering strength in dependence on whether the spectral value associated with the different frequency (or value) is quantized to zero or not.
  • the inventors recognized that, for example, a different filtering strength may be applied to spectral values quantized to zero than to spectral values not quantized to zero. This may improve the accuracy of a reconstructed spectrum.
  • the audio decoder is configured to adapt the filtering strength in dependence on whether a noise filling is applied to the spectral value associated with the different frequency (or value) or not.
  • the filter strength adaptation may be performed based on an information whether a respective spectral value was quantized to zero, e.g. in addition to whether for the respective frequency of the spectral value a noise filling is intended to be performed or was performed. This may comprise usage flags.
  • the audio decoder is configured to apply the prediction or the filtering, in order to determine the given, e.g. final, filling value, e.g. c(i), on the basis of a random or pseudo-random noise values, e.g. c(i).
  • a random or pseudo random noise value may, for example, be adapted using the prediction or the filtering, in order to calculate a e.g. final filing value that may provide a good approximation for a zero- quantized spectral value of an e.g. original e.g. input spectrum of the audio information.
  • the audio decoder is configured to perform a weighted combination of a noise value associated with the given frequency, and of a noise value associated with the other frequency (e.g. a combination d*c(i) + G’ S f*c(i-P’ S f) , with weight d for the noise value c(i) associated with the given frequency, and weight G’ S f for the noise value associated with the other frequency, or a combination d*c(i) + 14*G’ S f*c(i-P’ S f), with weight d for the noise value c(i) associated with the given frequency, and weight %*G’ s r for the noise value associated with the other frequency), or a weighted combination of a noise value associated with the given frequency, and of a filling value associated with the other frequency, in order to obtain the given, e.g.
  • the audio decoder is configured to adjust a weight, e.g. G’ s r or >z*G’ S f, given to the noise value associated with the other frequency or the weight, e.g. G’ S f or %*G’ S f, given to the filling value associated with the other frequency in dependence on whether a noise filling has been applied for a spectral value associated with the other frequency.
  • a weight e.g. G’ s r or >z*G’ S f
  • the inventors recognized that the e.g. final, filling value may, for example, be calculated using different frequency dependent quantities, e.g. a noise value associated with the given frequency, or associated with the other frequency, and/or a filling value associated with the other frequency.
  • an inventive concept may allow to determine or to obtain or to calculate the e.g. final filling value with good flexibility, such that, according to a specific situation, a filing value may be obtain that may be well or even best suitable for a reconstruction of the e.g. original audio information spectrum.
  • Choice of the respective quantity to be used for obtaining the e.g. final filling value may, for example, be performed based on the prediction lag information.
  • an adaptation or adjustment of a respective weight of a corresponding noise value or filling value associated with the other frequency may improve the determination of the e.g. final filling value and hence the reconstruction of the audio information.
  • the audio decoder is configured to determine a spectral distance, e.g. P’ S f, between the filling value associated with the given frequency and the other spectral value associated with the different frequency on the basis of an encoded information, e.g. an encoded value, describing the spectral distance, which is included in the encoded representation of the audio information.
  • a spectral distance e.g. P’ S f
  • the decoder may, for example, decide whether to use the prediction or filtering for the determination of the filing value.
  • the distance may be associated with the before explained prediction lag information and/or prediction lag value.
  • a parameter, e.g. a filter order, of a corresponding prediction or filtering may be determined or set or obtained based on the distance.
  • the inventors recognized that the spectral distance may be used in order to improve the determination of the spectral filling values.
  • the audio decoder is configured to determine a weight, e.g. d, which is applied to the noise value associated with the given frequency, on the basis of a gain information, e.g. a gain value, e.g. g S f, which is included in the encoded representation of the audio information, wherein the weight, which is applied to the noise value associated with the given frequency, is, as an example, a positive value, e.g. in a range between 0.5 and 1.
  • a respective noise value associated with the given frequency may be adapted with the weight that is determined in dependence on the gain information. This may allow to shape said noise value to improve its matching with a corresponding spectral value of the originally encoded audio information.
  • a respective noise value, or a respective filling value associated with the other frequency may be adapted with the weight that is determined in dependence on the gain information. This may allow to shape said noise or filling value to improve its matching with a corresponding spectral value of the originally encoded audio information.
  • the audio decoder is configured to determine the weight, g ( g which is applied to the noise value, e.g. c(i-P’st), associated with the other frequency, or to the filling value associated with the other frequency, in dependence on a sign information, e.g. a sign value, e.g. S S f, which is included in the encoded representation of the audio information.
  • a sign information e.g. a sign value, e.g. S S f, which is included in the encoded representation of the audio information.
  • the weight determination may, for example, be improved.
  • the sign information may allow an adaptation of a phase relation of the e.g. final filling value with respect to the noise value and/or the filling value associated with the other frequency, it may be based on.
  • the audio decoder is configured to determine the given filling value d(i) according to ( ) ( ) G’sf*c(i-P’ S f), if the coefficient c(i-P’ S f) was obtained using a noise filling, e.g. if the coefficient c(i-P’ S f) at a spectral location i-P’ S f was marked as a noise filled zero-quantized spectral coefficient in a previous processing and according to c , if the coefficient c(i-P' S f) was not obtained using a noise filling, e.g.
  • c(i) designates a spectral coefficient which is obtained using a noise filling and having a spectral index i
  • d designates an attenuation coefficient
  • G’ S f designates a weight which is based on a gain value that is included in the encoded audio representation
  • c(i-P’ S f) designates a spectral coefficient, which may, for example, be obtained using a noise, or which may, for example, be obtained without using a noise filling, and which may, for example, be obtained using a prediction or a filtering, having a spectral index i-P’ S f
  • P’ S f is a prediction parameter or a filtering parameter which is based on a prediction parameter information that is included in the encoded audio representation.
  • constant B may be chosen according to whether a given frame has more than one subframe.
  • the inventors recognized that using the above equations a good trade-off between signaling effort, complexity and effectivity of the decoding may be achieved.
  • the audio decoder is configured to mark noise-filled zero-quantized spectral coefficients, and the audio decoder is configured to selectively use a reduced filtering strength, e.g. 1 Z>*G’ S f, which is applied to spectral coefficients which are not marked, as noise-filled zero- quantized spectral coefficients.
  • a reduced filtering strength e.g. 1 Z>*G’ S f
  • the audio decoder is configured to perform the following processing for a plurality of subframes (sf):
  • noise filling e.g. using l S fj e.g. using random or pseudo- random noise values which are used to substitute spectral coefficients which are zero, wherein a noise intensity may, for example, be determined by a noise intensity value l S f], and mark, e.g. all, or a plurality of, noise-filled zero-quantized spectral coefficients
  • the inventors recognized that using the above steps a good trade-off between signaling effort, complexity and effectivity of the decoding may be achieved.
  • FIG. 1 A block diagram illustrating an audio decoder.
  • FIG. 1 A block diagram illustrating an audio decoder.
  • FIG. 1 A block diagram illustrating an audio decoder.
  • FIG. 1 A block diagram spectral coefficient
  • FIG. 1 For purposes of the audio decoder is configured to determine a processed spectral value, e.g. a spectral coefficient, using a prediction or filtering, e.g. using a temporal noise shaping (TNS) and/or using a frequency-domain noise shaping (FD-LTP), such that a given processed spectral value, which is associated with a given frequency, e.g. with a given frequency bin, is obtained in dependence on another spectral value, e.g. c(i-P
  • the audio decoder is configured to adapt a filtering strength, e.g. a weighting of a spectral value associated with the different frequency, e.g. by selectively setting the filtering strength to G’st or 1 Z2G’ S f, in dependence on an encoded or quantized or signaled spectral value, e.g.
  • a processed spectral value may, for example, be determined or calculated or obtained using a prediction or filtering based on other spectral values, which are associated with a different frequency.
  • a correlation or a dependency of spectral coefficients e.g. of spectral values of different frequencies, e.g. of different frequency bands, may be exploited, for example not only for filling values but for processed spectral values.
  • a coding effort may, for example, be reduced by taking advantage of such a correlation.
  • spectral values may be determined with a reduced amount of bits needed to be transmitted, while still providing a good representation of an originally encoded audio signal.
  • a decoding of the encoded audio representation may be improved by adapting the filtering strength in dependence on an encoded or quantized spectral value associated with the different frequency.
  • the audio decoder is configured to adapt the filtering strength to, e.g. selectively, reduce a contribution, e.g. a weighting in the prediction or filtering, of a nonzero-quantized, and possibly previously processed, e.g. previously TNS synthesis filtered, e.g. lower- frequency, spectral coefficients included in the prediction or filtering, e.g. when compared to a contribution (e.g. a weighting in the prediction or filtering) of zero-quantized (and possibly previously processed, e.g. previously TNS synthesis filtered)(e.g. lower- frequency) spectral coefficients included in the prediction or filtering.
  • a contribution e.g. a weighting in the prediction or filtering
  • TNS synthesis filtered e.g. lower- frequency
  • the audio decoder is configured to selectively adapt, e.g. reduce the filtering strength (e.g. of a temporal noise shaping filter e.g. of a frequency-domain long-term-prediction; e.g. of a filter which provides a filtered current spectral coefficient on the basis of a weighted combination, e.g. d*c(i)+att*G’ S f*c(i-P’ S f), of an unfiltered current spectral coefficient (e.g. c(i)) and of a filtered or unfiltered previous spectral coefficient (e.g.
  • the filtering strength e.g. of a temporal noise shaping filter e.g. of a frequency-domain long-term-prediction; e.g. of a filter which provides a filtered current spectral coefficient on the basis of a weighted combination, e.g. d*c(i)+att*G’ S f*c(i-P’ S f
  • d is a weight of the unfiltered current spectral coefficient
  • att is an attenuation factor that describes the adaptation of the filtering strength
  • G’ S f is a normal weight of the filtered or unfiltered previous spectral coefficient
  • P’ S f describes a spectral distance between the current spectral coefficient and the previous spectral coefficient) if a current spectral coefficient (e.g. a spectral coefficient c(i) at a current spectral position designated by a spectral index I, e.g. a spectral coefficient c(i) at a current spectral position before application of the filtering; e.g.
  • a transmitted current spectral coefficient or an encoded current spectral coefficient or a quantized current spectral coefficient is zero, e.g. has been quantized to zero, and a previous spectral coefficient, e.g. a spectral coefficient c(i- P’sf); e.g. represented by the "another spectral value”, has not been encoded as zero or has not been quantized to zero, e.g. at the side of an audio encoder.
  • the audio decoder is configured to selectively reduce the filtering strength to a value between 0.25 and 0.75, or, preferably, to a value between 0.4 and 0.6, or, preferably, to a value of 0.5, in order to adapt the filtering strength.
  • the audio decoder is configured to selectively, reduce the filtering strength, e.g. by downscaling filtering coefficients or prediction coefficients; e.g. by downscaling the filtering coefficients or filtering coefficients using a common downscaling factor, of a filtering, which considers a plurality, e.g. d s t, of previous spectral coefficients, e.g. c(i-1 ) to c(i-d S f), in dependence on values, e.g. encoded values or quantized values or signaled values, of a plurality of, e.g. d S f previous, e.g. encoded or quantized or signaled, spectral coefficients, e.g. a plurality of the previous spectral coefficients, if the current spectral coefficient, e.g. c(i), is encoded or quantized as zero.
  • the audio decoder is configured to selectively reduce the filtering strength (wherein the filtering strength may, for example, be defined by a plurality of filter weights, wherein the filter weights may, for example, be selectively down-scaled, e.g. using a common down-scaling factor that may, for example, be equal to 14, in case of a reduction of the filtering strength) if the current spectral coefficient, e.g. c(i), is encoded or quantized or signaled as zero and if all previous spectral coefficients considered in the filtering, e.g.
  • the audio decoder is configured to obtain a filtered current spectral coefficient, e.g. c(i), having spectral index i in dependence on a plurality of, e.g. encoded or quantized or signaled or filtered or predicted, previous spectral coefficients, e.g. c(i-1) to c(i-d S f), having spectral indices i-d s r to i-1 using the filtering or prediction.
  • a filtered current spectral coefficient e.g. c(i)
  • previous spectral coefficients e.g. c(i-1) to c(i-d S f
  • the audio decoder is configured to selectively reduce the filtering strength if, e.g. if and only if, one or more, e.g. signaled, spectral coefficients, or all spectral coefficients, having spectral indices i-d 8 f+1 to i have been quantized or encoded or signaled as zero, and if a spectral coefficient having spectral index i-d S f has not been quantized or encoded or signaled as zero, wherein, for example, d S f is equal to a filter order or prediction order.
  • filter coefficients which are associated with spectral coefficients having spectral indices between i-d S f+ 1 and i-1 are equal to zero.
  • the audio decoder is configured to use encoded or quantized or signaled spectral coefficients, e.g. before a noise filling, for deciding about the filtering strength. Moreover, the audio decoder is configured to use preprocessed spectral coefficients, e.g. after an application of a noise filling and/or after an application of a frequency-domain long-term prediction, as an input for the filtering or prediction.
  • Further embodiments according to the third aspect of the invention comprise a method for providing a decoded audio representation on the basis of an encoded audio representation, the method comprising filling spectral holes of a decoded set of spectral values, e.g. using a substitution of spectral coefficients quantized to zero on the basis of respective filling values, using respective filling values.
  • the method further comprises determining a, e.g. final, filling value, e.g. a replacement for c(i); e.g. c(i), using a prediction or filtering, e.g. using a computation rule d*c(i) +G’ S f*c(i-P’ S f), such that a given filling value, e.g.
  • c(i) which is associated with a given frequency, e.g. with a given frequency bin, is obtained in dependence on another spectral value, e.g. c(i-P’ S f), or c (i- P’sf), which is associated with a different frequency, e.g. with a different frequency bin, e.g. with a different frequency bin having a frequency bin index i-P’sri e.g. with a frequency or frequency bin having a spectral distance P’ S f or a spectral distance d S f from the given frequency or from the given frequency bin.
  • the method comprises adapting a filtering strength, e.g.
  • a weighting of a spectral value associated with the different frequency e.g. by selectively setting the filtering strength to G’ S f or 1/2G’ S f, in dependence on an encoded or quantized spectral value (e.g. a spectral value as it is (originally) determined by the encoded representation of individual spectral values in the encoded audio information; e.g. by a spectral value before a noise filling is applied; e.g. by a spectral value directly after an arithmetic decoding) associated with the different frequency, e.g. with the different frequency bin; e.g. with the different frequency bin having a frequency bin index i-P’ S f.
  • an encoded or quantized spectral value e.g. a spectral value as it is (originally) determined by the encoded representation of individual spectral values in the encoded audio information; e.g. by a spectral value before a noise filling is applied; e.g.
  • FIG. 1 A block diagram illustrating an encoded audio representation.
  • FIG. 1 A block diagram illustrating an encoded audio representation.
  • FIG. 1 A block diagram illustrating an encoded audio representation.
  • the method comprises adapting a filtering strength, e.g. a weighting of a spectral value associated with the different frequency, e.g. by selectively seting the filtering strength to G’ S f or 1 /2G’ S f, in dependence on an encoded or quantized spectral value, e.g. a spectral value as it is (originally) determined by the encoded representation of individual spectral values in the encoded audio information; e.g. by a spectral value before a noise filling is applied; e.g.
  • spectral value directly after an arithmetic decoding associated with the different frequency, e.g. with the different frequency bin; e.g. with the different frequency bin having a frequency bin index i-P’
  • S f- Further embodiments according to the third aspect of the invention comprise a computer program for performing any of the above methods, when the computer program runs on a computer.
  • Fig. 1 shows a schematic view of an audio decoder for providing a decoded audio information on the basis of an encoded audio information according to embodiments according to the first aspect of the invention
  • Fig. 2 shows a schematic example of spectral envelopes according to conventional concepts
  • Fig. 3 shows a schematic example of spectral envelopes (intensity over frequency) according to the first aspect of the invention
  • Fig. 4 shows a schematic view of an audio decoder with additional optional features, according to embodiments according to the first aspect of the invention
  • Fig. 5 shows a schematic view of an audio encoder for providing an encoded audio information on the basis of an input audio information according to embodiments according to the first aspect of the invention
  • Fig. 6 shows a schematic view of an audio encoder with additional optional features, according to embodiments according to the first aspect of the invention
  • Fig. 7 shows an example for a functionality of an encoder, according to embodiments according to the first aspect of the invention
  • Fig. 8 shows a schematic view of an audio decoder for providing a decoded audio information on the basis of an encoded audio information according to embodiments according to the second aspect of the invention
  • Fig. 9 shows a schematic view of a first spectral filling method unit according to embodiments according to the second aspect of the invention.
  • Fig. 10 shows a schematic view of an audio decoder with additional optional features according to embodiments according to the second aspect of the invention.
  • Fig. 11 shows a schematic view of an audio encoder for providing an encoded audio information on the basis of an input audio information according to embodiments according to the second aspect of the invention
  • Fig. 12 shows a schematic view of another audio encoder for providing an encoded audio information on the basis of an input audio information according to embodiments according to the second aspect of the invention.
  • Fig. 13 shows a schematic view of an audio decoder for providing a decoded audio information on the basis of an encoded audio information according to embodiments according to the third aspect of the invention
  • Fig. 14 shows a schematic view of an audio decoder with additional optional features, according to embodiments according to the third aspect of the invention.
  • Fig. 15 shows an example for a functionality of a decoder, according to embodiments according to the third aspect of the invention.
  • Fig. 16 shows a schematic view of an audio decoder with additional optional features, according to embodiments according to the third aspect of the invention
  • Fig. 17 shows a block diagram of a method for providing a decoded audio information on the basis of an encoded audio information according to an embodiment according to the first aspect of the invention
  • Fig. 18 shows a block diagram of a method for providing an encoded audio information on the basis of an input audio information according to an embodiment according to the first aspect of the invention
  • Fig. 19 shows a block diagram of a method for providing a decoded audio information on the basis of an encoded audio information according to an embodiment according to the second aspect of the invention
  • Fig. 20 shows a block diagram of a method for providing an encoded audio information on the basis of an input audio information according to an embodiment according to the second aspect of the invention
  • Fig. 21 shows a block diagram of a first method for providing a decoded audio information on the basis of an encoded audio information according to an embodiment according to the third aspect of the invention
  • Fig. 22 shows a block diagram of a second method for providing a decoded audio information on the basis of an encoded audio information according to an embodiment according to the third aspect of the invention
  • Fig. 23 shows an example plot of the time-domain effect of FD-LTP filtering of a pseudo-random noise spectrum subjected to an inverse transform according to embodiments of the invention
  • Fig. 24 shows a schematic example for a filtering strength reduction according to embodiments of the invention.
  • Fig. 25 shows a schematic example for a an adaptive filtering according to embodiments of the invention.
  • Fig. 1 shows a schematic view of an audio decoder for providing a decoded audio information on the basis of an encoded audio information according to embodiments according to the first aspect of the invention.
  • Fig. 1 shows audio decoder 100 with a spectral tilt information derivation unit 110, a frequency variable scaling unit 120 and a spectral holes filling unit 130.
  • the decoder 100 may comprise a decoding unit 140.
  • the decoder 100 may be provided with an encoded audio information 102. From, or using the encoded audio information, the spectral tilt information derivation unit 110 may be configured to derive or determine or calculate a spectral tilt information 112.
  • the decoder 100 may be configured to decode the encoded audio information 102 or a portion of the encoded audio information using the optional decoding unit 140, in order to obtain a decoded set of spectral values 142.
  • the decoded set of spectral values 142 may as well be provided from an external device.
  • the decoder 100 may be configured to apply a frequency variable scaling, a spectral tilt of which is determined by the spectral tilt information 120, to filling values 122.
  • the filling values 122 may, for example, be gap fill coefficients or noise values of a noise filing or gap filling values of an intelligent gap filling (e.g. spectral values from a different frequency or frequency band).
  • the frequency variable scaling unit may provide scaled filling values 124 to the spectral holes filling unit 130.
  • the decoder 100 may be configured to use modified filling values 122, 124 in order to fill spectral holes of the decoded set of spectral values 142.
  • a decoded audio information 104 may be provided.
  • the frequency variable scaling may, for example, be performed after filling spectral holes of the set decoded set of spectral values 142, wherein the holes may be filled with the unmodified filling values 122.
  • the scaling may then, optionally, be applied to the already modified (e.g. set decoded set of spectral values 142 filled with filling values 122) set of spectral values.
  • the decoded audio information may hence be provided based on the frequency variable scaling unit 120, wherein the frequency variable scaling unit may receive its input from the spectral holes filling unit 130.
  • an adaptation of a spectral envelope of decoded spectral values may allow to beter reconstruct, or approximate an original spectral envelope of the audio information.
  • Fig. 2 shows a schematic example of spectral envelopes (intensity over frequency) according to conventional concepts.
  • An example of an original spectral envelope e.g. representing the original spectral values; e.g. representing the original spectral coefficients of an audio information or of a frame or of a subframe of the audio information is shown with line 210.
  • the dashed line 220 is offset downwards for better visibility of all curves and represents an example for a masking envelope, e.g. a masking threshold, e.g. a noise shaping envelope, that may be associated with scaling factors.
  • Line 230 may show an example of a reconstructed noise envelope according to conventional concepts.
  • noise filling may be performed for signal portions of frequencies between a noise filling start frequency 240 and a noise filling end frequency 250.
  • the reconstructed envelope 230 exceeds the original spectral envelope 210 at high frequencies, thus potentially causing audible noise after decoding, while it remains significantly below the original spectral envelope 120 at lower frequencies, thus likely causing insufficient gap-fill energy and audible spectral holes.
  • a distance between the masking envelope 210 and the effective reconstructed noise envelope 230 may be constant (thin double arrow) and, therefore, not follow the original spectral envelope 210 accurately. This may, for example, be caused, e.g. inter alia, by a pre-emphasis tilt during the calculation of the masking envelope.
  • a decoder as shown in Fig. 1 a reconstructed noise envelope 310, as shown in Fig. 3 may be achieved.
  • Fig. 3 shows a schematic example of spectral envelopes (intensity over frequency) according to the first aspect of the invention.
  • the reconstructed noise envelope 310 may be corrected to reduce a difference between the original signal envelope 210 and the reconstructed envelope 310.
  • the masking envelope 220 may, for example, be an approximation or an interpolation based on a plurality of scaling factors of different frequency bands.
  • Fig. 4 shows a schematic view of an audio decoder with additional optional features, according to embodiments according to the first aspect of the invention.
  • Fig. 4 shows audio decoder 400 comprising a spectral tilt information derivation unit 410, a frequency variable scaling unit 420, a spectral holes filling unit 430 and an optional decoding unit 440, with functionalities, as an example, according to audio decoder 100 from Fig. 1.
  • decoder 400 comprises a noise information derivation unit 450.
  • the noise information derivation unit 450 may be configured to derive a noise information 450 from the encoded audio information 402.
  • the noise information may be or may comprise, for example, a noise level information, e.g. L S f and/or a noise intensity information.
  • the decoder 400 may optionally comprise filling value obtaining unit 460, which may be configured to obtain or to determine or to calculate the filling values 422 using the noise information 450, e.g. the noise level information and/or the noise intensity information.
  • the filling values 422 may be noise filling values, wherein an energy of a respective noise falling value may be set according to the noise level information.
  • the frequency variable scaling unit 420 may be configured to apply the frequency variable scaling, such that the frequency variable scaling describes a linear decrease of intensity with increasing frequency on a logarithmic intensity scale.
  • the spectral tilt information 412 may describe a spectral tilt in a logarithmic domain.
  • the decoder 400 may comprise a scaling value obtaining unit 470.
  • the scaling value obtaining unit may, for example, be configured to obtain scaling values 472 for the frequency-variable scaling.
  • the decoder 400 e.g. the scaling value obtaining unit 470 of the decoder 400 may determine or obtain or derive the scaling values 472 in a logarithmic domain.
  • a conversion from logarithmic domain to linear domain may be performed for any value, e.g. spectral value, or mathematic operation.
  • the scaling values 472 for the frequency- variable scaling from the logarithmic domain to a linear domain may be performed for any value, e.g. spectral value, or mathematic operation.
  • the scaling values 472 may, for example, be derived or obtained or calculated based on or using or in dependence on a product a tilt value 474 which is based on the tilt information 412, and of a frequency information 476, e.g. a frequency value.
  • the tilt value 474 may, for example, be provided by the spectral tilt information derivation unit, e.g. based on the spectral tilt information 412, or for example, directly from the encoded audio information 402.
  • the spectral tilt information 412 may, for example, be the tilt value 474.
  • the frequency information may, for example, be a frequency value or a frequency index, describing or providing an information about the frequency of a spectral value or coefficient that is to be scaled.
  • the frequency variable scaling unit 420 may be provided with the information of the spectral tilt information 412 via the scaling value 472 which is based on the tilt value 474.
  • the scaling value obtaining unit 470 may be configured to obtain a plurality of scaling values for the frequency variable scaling associated with different frequency bands.
  • the frequency information 476 may, for example, comprise start frequencies center frequencies of respective frequency bands of which spectral values, e.g. spectral coefficients, e.g. noise values or gap filling values, e.g. filling values are to be scaled.
  • the scaling value obtaining unit may be configured to using start frequencies of respective frequency bands or using center frequencies of respective frequency band to obtain the scaling values 472.
  • the frequency information 476 may comprise start frequency bin indices or center frequency bin indices of respective frequency bands for obtaining the scaling values 472.
  • the scaling values 472 may, for example, comprise frequency-independent noise scaling values and/or frequency-variable noise scaling values, wherein the frequency-variable noise scaling values may be determined based on the tilt value 474, e.g. spectral tilt.
  • the decoder 400 may be configured, e.g. the frequency variable scaling unit 420 of decoder 200, to obtain a filling value, e.g. a scaled filling value 424 using a multiplication of a noise value (the noise information may, for example, comprise a noise value), of a frequency-independent noise scaling value and of a frequency-variable noise scaling value.
  • the noise value may, for example, be a random noise value or a pseudo-random noise value and may be determined by the noise information derivation unit 450.
  • the audio decoder 400 may be configured to apply a scaling, which is based on a masking envelope, to the decoded spectral values 442 and to the filling values 222.
  • the audio decoder may, for example, be configured to obtain a masking envelope from the encoded audio information.
  • the masking envelop may, for example, be associated with scaling factors, the masking envelope may, for example, be an interpolation of scaling factors.
  • a scaling e.g. based on the masking envelope, may, for example, be applied to the full spectrum e.g. decoded spectral values that were not quantized to zero, and spectral values that were quantized to zero and filled with filling values (or, for example, first scaling the filling values and then filling them in the spectral “holes”).
  • the spectral tilt information derivation unit 410 may, for example, be configured to obtain the spectral envelope from the encoded audio information 402 and may be configured to provide an information about the spectral envelope using the spectral tilt information which may be used to adapt the full spectrum.
  • Fig. 5 shows a schematic view of an audio encoder for providing an encoded audio information on the basis of an input audio information according to embodiments according to the first aspect of the invention.
  • Fig. 5 shows encoder 500 comprising an optional encoding unit 510.
  • the encoding unit may, for example, be configured to encode a plurality of quantized spectral values 512.
  • Encoder 500 further comprises an optional spectral tilt information determination unit 520.
  • the spectral tilt information determination unit 520 may be configured to determine a spectral tilt information 522 on the basis of a spectral energy information 524 and a masking envelope information 526.
  • the masking envelope information 526 may be provided by the processing unit 530, e.g. based on the input audio information.
  • a masking envelope may be calculated in dependency on the input audio information.
  • a fixed masking envelop may be used.
  • the spectral tilt information may, for example, describe an average variation of a difference between a spectral energy of an input audio signal and a masking envelope.
  • encoder 500 may comprise a processing unit 530 which may be configured to provide the spectral energy information 524, e.g. a spectral energy, and the quantized spectral values 512 to the spectral tilt information determination unit 520 and respectively the encoding unit 510, based on the input audio information 502, e.g. an input audio data.
  • a processing unit 530 may be configured to provide the spectral energy information 524, e.g. a spectral energy, and the quantized spectral values 512 to the spectral tilt information determination unit 520 and respectively the encoding unit 510, based on the input audio information 502, e.g. an input audio data.
  • the encoding unit 510 may receive the spectral tilt information 522 and may be configured to encode the spectral tilt information.
  • the encoder e.g. the encoding unit 510 of encoder 500 may be configured to provide an encoded audio information 504, for example comprising an encoded representation of the quantized spectral values 512 and an encoded representation of the spectral tilt information 522.
  • Fig. 6 shows a schematic view of an audio encoder with additional optional features, according to embodiments according to the first aspect of the invention.
  • Fig. 6 shows encoder 600 comprising a spectral tilt information determination unit 620, an optional processing unit 630 and an encoding unit 610 (and corresponding input/output signals), as explained in the context of Fig. 5.
  • the spectral tilt information determination unit 620 may optionally be configured to determine the spectral tilt information 622, such that the spectral tilt information describes a frequency variation of a difference between the spectral energy information 624 and the masking envelope information 626 over frequency.
  • the spectral tilt information 622 may, for example, describe a line function with a spectral tilt in a logarithmic domain.
  • the line function may allow to adjust a tilt of a reconstructed spectrum to better approximate an original spectrum of the input audio information 602.
  • the spectral tilt information determination unit may, for example, be configured to determine the spectral tilt information in a logarithmic domain.
  • the spectral tilt information determination unit 620 may be configured to determine the spectral tilt information 622 on the basis of a difference between a logarithmized representation of a spectral envelope and a logarithmized representation of a masking envelope.
  • the spectral energy information 624 may comprise an information about the spectral envelope of the input audio information 602, as an example, in a logarithmized form and the masking envelope information 626 may, for example, comprise a masking envelope, e.g. comprising scaling factors, for example in a logarithmized form.
  • the encoder may perform any calculations in a logarithmic and/or in a linear domain. Hence, values and or calculations may be transformed in one or the other domain.
  • the spectral tilt information determination unit 620 may, for example, be configured to obtain the spectral tilt information 622 using a linear regression.
  • the inventors recognized that using a linear regression a computational inexpensive calculation with good accuracy for the tilt information may be performed.
  • the spectral tilt information may, for example, be configured to obtain the spectral tilt information on the basis of spectral-band-wise energy values or spectral band wise root-mean-square values representing an energy of spectral values in a plurality of respective spectral bands and on the basis of spectral band-wise energy values or spectral-band-wise root-mean-square values representing the masking threshold in a plurality of respective spectral bands.
  • the spectral energy information 624 may hence comprise spectral band wise root-mean-square values representing an energy of spectral values in a plurality of respective spectral bands and the masking envelope information 626 may, for example comprise spectral band-wise energy values or spectral-band-wise root-mean-square values representing the masking threshold in a plurality of respective spectral bands.
  • the processing unit 630 may be configured to provide said information.
  • the spectral tilt information determination unit 620 may be configured to determine separate spectral tilt information 622 for different audio frames and/or for different audio subframes.
  • encoder 600 may, for example, comprise a difference value determinator 640.
  • the difference value determinator 640 may, for example, be configured to determine a difference value 642 representing, in the form of a single value, a difference between the spectral energy information 624 and the masking envelope information 626 over a frequency range comprising a plurality of spectral bins.
  • the encoder 600 may optionally comprise a noise level information obtaining unit 650, which may be configured to obtain or determine or to calculate a noise level information 652 in dependence or based on the difference value 642.
  • a noise level information obtaining unit 650 may be configured to obtain or determine or to calculate a noise level information 652 in dependence or based on the difference value 642.
  • the encoding unit 610 may, for example, receive the noise level information 652 and may be configured to encode the noise level information in the encoded audio information.
  • the difference value determinator 640 may, for example, be configured to obtain the difference value 642 using a linear regression.
  • the encoding unit 610 may, for example, be configured to encode the spectral tilt information 622 using three bits.
  • the encoding unit 610 may, for example, be configured to encode the spectral tilt information 622 such that the encoded spectral tilt information always represents a negative spectral tilt.
  • Fig. 7 shows an example for a functionality of an encoder, e.g. encoder 500 shown in Fig. 5 or encoder 600 shown in Fig. 6, according to embodiments according to the first aspect of the invention.
  • an inventive encoder may be configured to perform the following steps:
  • Fig. 8 shows a schematic view of an audio decoder for providing a decoded audio information on the basis of an encoded audio information according to embodiments according to the second aspect of the invention.
  • Fig 8 shows decoder 800 comprising a spectral holes filling unit 810, which is configured to fill spectral holes of a decoded set 812 of spectral values.
  • the result of the hole filling may, for example, be a decoded audio information 802.
  • decoder 600 may comprise a prediction lag information obtaining unit 820.
  • the prediction lag information obtaining unit 820 may be configured to obtain or determine or calculate a prediction lag information 822.
  • the prediction lag information obtaining unit 820 may receive an encoded audio information 804 that may be used to determine the prediction lag information 822.
  • decoder 800 comprises a decoding unit 830.
  • Decoding unit 830 may be configured to provide the decoded set 812 of spectral values based on the encoded audio information 804.
  • the decoder 800 may comprise a first spectral filling method unit 840 and a second spectral filling method unit 850 (Optionally, decoder 800 may comprise a plurality of second, e.g. of further spectral filing method units or the second spectral filling method unit may be configured to provide the functionality of a plurality of further spectral filling methods).
  • the respective spectral filling method unit may, for example, be configured to provide filling values 814 to the spectral holes filling unit 810 in order to fill the spectral holes.
  • a switching (using switch 860) may be performed between the first spectral filling method unit 840 and a second spectral filling method unit 850 (or for example a plurality of other spectral filling method units) for the provision of filling values 814 to the spectral holes filling unit 810.
  • a frequency filtering or a frequency prediction may be used to obtain filling values which are used to fill spectral holes
  • no frequency filtering and no frequency prediction may be used to obtain filling values which are used to fill the spectral holes.
  • decoder 800 may, for example, be configured to use the first spectral filling method if the prediction lag information 822 is non-zero, or to use the first spectral filling method if the prediction lag information 822 is larger than zero and to use the second (e.g. one of the one or more further) spectral filling methods otherwise.
  • the prediction lag information obtaining unit may, for example, be configured to use an encoded representation of a prediction lag value which is included in the encoded audio information 804, in order to obtain the prediction lag information 822, e.g. an prediction lag value.
  • Fig. 9 shows a schematic view of a first spectral filling method unit according to embodiments according to the second aspect of the invention.
  • Fig. 9 may show a schematic view of details of the first spectral filling method unit 840 of Fig. 8.
  • Fig. 9 shows a prediction or filtering unit 910 and a filtering strength adaptation unit 920.
  • An inventive audio decoder may, for example, be configured to determine, e.g. using prediction or filtering unit 910 to obtain a filling value 912, which is associated with a given frequency in dependence on another spectral value 914, which is associated with a different frequency.
  • the prediction or filtering unit 910 may therefore be configured to use or to apply a prediction or a filtering.
  • an inventive audio decoder may, for example, be configured to adapt, e.g. using filtering strength adaptation unit 920, a filtering strength information 922, e.g. a filtering strength, in dependence on an encoded or quantized spectral value 924 associated with the different frequency.
  • spectral value 914 may e.g. alternatively be used to adapt the filtering strength.
  • the filtering strength information 922 may, for example, comprise a filtering strength, wherein the filtering strength determines an impact of the other spectral value 914 onto the filling value 912.
  • an inventive decoder e.g. using filtering strength adaptation unit 920, may be configured to adapt the filtering strength information, e.g. the filtering strength, in dependence on the spectral value 924 associated with the different frequency as it is determined by the encoded representation of individual spectral values in the encoded audio information.
  • adaptation of the filtering information 922 e.g. of the filtering strength in dependence on the spectral value 924 associated with the different frequency may, for example, be performed before a noise filling is applied.
  • the filtering strength adaptation unit may be configured to adapt the filtering information 922, e.g. the filtering strength in dependence on whether the spectral value 924 or 914 associated with the different frequency (or value) is or was quantized to zero or not.
  • the filtering strength may be adjusted in dependence on a masking envelope, e.g. scaling factors.
  • the inventors recognized that zero quantized spectral values may be filtered differently in order to improve the decoding of the audio information.
  • an inventive decoder may be configured to adapt, e.g. using filtering strength adaptation unit 920, the filtering strength in dependence on whether a noise filling is or was applied to the spectral value 924 or 914 associated with the different frequency (or value) or not.
  • an inventive decoder may be configured to, e.g. using prediction or filtering unit 910, to selectively apply a filtering in a frequency direction or a prediction in a frequency direction for spectral values 924 or 914 for which a noise filling is applied.
  • an inventive decoder may be configured, e.g. using prediction or filtering unit 910, to apply the prediction or the filtering, in order to determine the given filling value 912 on the basis of random or pseudo-random noise values.
  • An optional noise value information 916 for example comprising the random or pseudo-random noise values, may therefore be provided to the prediction or filtering unit 910.
  • the optional noise value information 916 may comprise random and/or pseudo-random noise values. Such values may, for example, be provided by a noise generator (not shown).
  • the decoder may optionally comprise a noise generator.
  • the noise value information may comprise a noise generator signal, e.g. the random and/or pseudo-random noise values.
  • the spectral value associated with the different frequency 924 or 914 may, for example be a filling value associated with the other frequency.
  • the spectral value associated with the different frequency 924 or 914 may, for example, be noise value associated with the other frequency.
  • the inventors recognized that the filling value 912 may, for example, be determined using a weighted combination of a noise value associated with the given frequency, and of a noise value associated with the other frequency or a weighted combination of a noise value associated with the given frequency, and of a filling value associated with the other frequency.
  • the prediction and/or filtering unit 910 may be configured to perform one or both weighted combinations in order to obtain the given filling value 912.
  • an inventive decoder may comprise a weight adjustment unit 930.
  • the weight adjustment unit may, for example, be configured to adjust a weight given to the noise value associated with the other frequency or the weight given to the filling value associated with the other frequency in dependence on whether a noise filling has been applied for a spectral value 924 or 914 associated with the other frequency.
  • an inventive audio decoder may comprise a spectral distance determination unit 940, which may be configured to determine a spectral distance between the filling value 912 associated with the given frequency and the other spectral value 924 or 914 associated with the different frequency on the basis of an encoded information describing the spectral distance, which is included in an encoded representation 804 of the audio information.
  • a spectral distance determination unit 940 may be configured to determine a spectral distance between the filling value 912 associated with the given frequency and the other spectral value 924 or 914 associated with the different frequency on the basis of an encoded information describing the spectral distance, which is included in an encoded representation 804 of the audio information.
  • the weight adjustment unit 930 may receive the encoded representation 804 of the audio information.
  • the weight adjustment unit 930 may be configured to determine a weight information 932, e.g. weight, which is applied to the noise value associated with the given frequency, on the basis of a gain information which is included in the encoded representation 804 of the audio information.
  • the weight adjustment unit 930 may be configured to determine a weight information 932, e.g. weight, which is applied to the noise value associated with the other frequency, or to the filling value associated with the other frequency, in dependence on a gain information, which is included in the encoded representation 804 of the audio information.
  • a weight information 932 e.g. weight, which is applied to the noise value associated with the other frequency, or to the filling value associated with the other frequency, in dependence on a gain information, which is included in the encoded representation 804 of the audio information.
  • the weight adjustment unit 930 may be configured to determine a weight information 932, e.g. weight, which is applied to the noise value associated with the other frequency, or to the filling value associated with the other frequency, in dependence on a sign information, which is included in the encoded representation of the audio information.
  • a weight information 932 e.g. weight, which is applied to the noise value associated with the other frequency, or to the filling value associated with the other frequency, in dependence on a sign information, which is included in the encoded representation of the audio information.
  • weight information 932) designates a weight which is based on a gain value that is included in the encoded audio representation 804; and wherein c(i-P' s r) designates a spectral coefficient having a spectral index i-P’ S f, wherein P’ sf is a prediction parameter or a filtering parameter which is based on a prediction parameter information that is included in the encoded audio representation.
  • the filtering strength adaptation unit may, for example, be configured to selectively use a reduced filtering strength which is applied to spectral coefficients which are not marked as noise-filled zero-quantized spectral coefficients.
  • the decoder may comprise a marking unit for marking noise-filled zero-quantized spectral coefficients (not shown).
  • Fig. 10 shows a schematic view of an audio decoder with additional optional features according to embodiments according to the second aspect of the invention.
  • Fig. 10 shows decoder 1000 comprising a spectral holes filling unit 1010, a prediction lag information obtaining unit 1020, a decoding unit 1030, a first spectral filling method unit 1040, a second spectral filling method unit 1050 and a switch 1060.
  • the functionality of these elements may, for example, be similar or analogous to the respective elements of Fig. 8 and respectively 9.
  • decoder 1000 comprises a third spectral filling method unit 1070.
  • Decoder 1000 may, for example, be configured to switch between a second spectral filling method (e.g. using spectral filling method unit 1050) in which random or pseudo-random filling values are used to fill spectral holes (e.g. providing respective filling values 1014 to the spectral holes filling unit 1010) and a third spectral filling method 1070, in which filling values 1014, which are obtained using a copying of non-zero spectral coefficients, are used to fill spectral holes, in dependence on a prediction lag information and/or in dependence on a tonality information 1082, e.g. a tonality, of the audio information.
  • a second spectral filling method e.g. using spectral filling method unit 1050
  • random or pseudo-random filling values are used to fill spectral holes
  • a third spectral filling method 1070 in which filling values 1014
  • decoder 1000 may comprise a tonality information obtaining unit 1080, which may be configured to obtain the tonality information 1082 on the basis of the encoded audio information 1004.
  • tonality information obtaining unit 1080 may, for example, be configured to judge whether the audio information is tonal in dependence on a tonality information which is included in the encoded audio representation 1004 and/or in dependence on an information indicating whether a tonality information is included in the encoded audio information, and/or in dependence on a filtering gain value and/or in dependence on a prediction gain value and/or in dependence on a time-domain post-filter gain value.
  • the tonality information obtaining unit 1080 may therefore be configured to determine or extract the respective information for the judgement form the encoded audio information 1004.
  • tonality information obtaining unit 1080 may receive the encoded audio information and/or, for example, at least one of an information indicating whether a tonality information is included in the encoded audio information, a filtering gain value, a prediction gain value and/or a time-domain post-filter gain value.
  • the spectral holes filling 1010 unit may, for example, be configured to apply a high frequency noise gain adjustment for a filling of spectral holes in an upper frequency region below an noise filling end frequency. Therefore, the spectral holes filling unit 1010 may be provided with an high frequency (HF) energy information 1032.
  • HF high frequency
  • the decoding unit 1030 may be configured to obtain the high frequency energy information 1032 on the basis of the encoded audio information 1004.
  • decoder 1000 e.g. decoding unit 1030
  • decoder 1000 may, for example, be configured to obtain a high frequency energy delta value in dependence on a high frequency energy value, in dependence on a global gain value, and in dependence on a noise level information.
  • the HF energy information 1032 may comprise the high frequency energy delta value.
  • the high frequency energy value, the global gain value and/or the noise level information may, for example, be included in an encoded form in the encoded audio information 1004.
  • the audio decoder may be configured to apply the high frequency energy delta value to obtain one or more noise filling values.
  • the filling value 1014 may be a noise filling value
  • the spectral holes filling unit 1010 may be configured to apply the high frequency energy delta value provided by the decoding unit to adapt the filling values 1014 to “fill” the noise filling values in the decoded set of spectral values.
  • the audio decoder 1000 e.g. the spectral holes filling unit, may be configured to selectively multiply one or more intermediate noise filling values (e.g. filling values 1014) which are associated with frequencies in an upper frequency region below an noise filling end frequency, with the high frequency energy delta value
  • the audio decoder 1000 e.g. the spectral holes filling unit 1010, may be configured to selectively apply the high frequency noise gain adjustment to spectral values for which a noise filling is performed.
  • the high frequency noise gain adjustment may be applied in a frequency range between 8 kHz and 10 kHz.
  • the high frequency energy value or the high frequency energy delta value may represent an energy of a plurality of spectral coefficients at a frequency below a noise filling end frequency or in a frequency region below the noise filling end frequency which were quantized to zero.
  • Fig. 11 shows a schematic view of an audio encoder for providing an encoded audio information on the basis of an input audio information according to embodiments according to the second aspect of the invention.
  • Fig. 11 shows encoder 1100 comprising an optional encoding unit 1110.
  • the encoding unit 1110 may be configured to encode a plurality of quantized spectral values 1112.
  • encoder 1100 comprises a lag value obtaining unit 1120, which may be configured to obtain a lag value 1122, which defines a characteristic of a filtering operation or of a prediction operation to be performed by an audio decoder for deriving one or more filling values for filling spectral holes.
  • encoder 1100 comprises a gain value obtaining unit 1130, which may be configured to obtain a gain value 1132 which defines a characteristic of the filtering operation or of the prediction operation to be performed by an audio decoder for deriving one or more filling values for filling spectral holes.
  • the encoder 1100 may comprise in addition a lag value modification unit 1140, which may be configured to set the lag value 1122 to zero if the gain value 1132 is smaller than a threshold value or if an absolute value of the gain value is smaller than a threshold value, to thereby obtain a modified lag value 1142.
  • a lag value modification unit 1140 which may be configured to set the lag value 1122 to zero if the gain value 1132 is smaller than a threshold value or if an absolute value of the gain value is smaller than a threshold value, to thereby obtain a modified lag value 1142.
  • the encoding unit 1110 may, be configured to encode the determined lag value 1122 or the modified lag value 1142.
  • the quantized spectral values 1112 and the (modified) lag value 1122/1142 may be encoded using the encoding unit 1110 in an encoded audio information 1102.
  • encoder xx may comprise an optional processing unit 1150 for providing the quantized spectral values 1112 to the encoding unit 1110 based on an input audio information 1104.
  • the lag value 1122 and the gain value 1132 may be determined or calculated using or based on an autocorrelation information which is applied to a set of spectral values 1152, which may, for example, be associated with the spectral values 1112.
  • the gain value 1132 may, for example, be determined in dependence on a peak of an autocorrelation function which is obtained on the basis of the set of spectral values.
  • the processing unit 1150 may be configured to provide the set of spectral values 1152 to the lag value obtaining unit 1120 and to the gain value obtaining unit 1130.
  • the spectral values 1152 may, for example, be quantized and may, for example be equal to the quantized spectral values 1112.
  • the encoding unit 1110 may be configured to encode the gain value 1132 if the encoded lag value is non-zero.
  • the lag value 1122/1142 may comprise an information about a dependency or a correlation between spectral coefficients, e.g. spectral values, for example over different frequency bands. In case such a correlation exists, the lag value 1122 may be non-zero and hence a dependency may be characterized by the gain value 1132.
  • the lag value 1122 may describe a distance in the frequency domain of a spectral value with a given frequency to another spectral value with a different frequency.
  • the gain value may, for example, describe or quantize the correlation in between the spectral values.
  • one spectral value may be determined by the other and the gain and lag information.
  • a transmission of the second spectral value may not be necessary with known lag and gain information.
  • the processing unit 1150 may, for example, be configured to determine or calculate a high-frequency (HF) energy value 1154.
  • the HF energy value 1154 may comprise an information for a adjusting a HF gap filling range.
  • the encoding unit 1110 may be configured to selectively encode the high- frequency energy value 1154, which may describe an energy in an upper portion of a spectrum, e.g. of the quantized spectral values if the encoded lag value is zero.
  • a lag value 1122/1142 of zero may indicate that no correlation between spectral coefficients may be exploited for an encoding of the spectral values.
  • the HF energy value may be encoded to perform a gap filling, e.g. such that a spectral energy in the gap filling range is adapted in the decoder according to the HF energy value 1154.
  • the encoding unit 1110 may be configured to selectively either encode the gain value 1132 or the high-frequency energy value 1154 in dependence on the encoded lag value.
  • the encoding unit 1110 may optionally be configured to encode the gain value 1132 and the high-frequency energy value 1154 using a same number of bits.
  • an encoding scheme e.g. a number of bits reserved for a specific information in an encoded bitstream, may be kept constant in either case, or in other words irrespective of whether the gain value or the high-frequency energy value are encoded.
  • encoder 1100 may, optionally, be configured to determine separate lag values 1122/1142 and/or separate gain values 1132 for different audio frames and/or for different audio subframes.
  • the lag value 1122/1142 and/or the gain value 1132 may be determined or calculated in a transform domain.
  • the lag value obtaining unit 1120 may be configured to perform a long term transientness detection and to selectively set the lag value 1122 to zero if an audio frame or audio subframe is found to be not long-term transient.
  • Fig. 12 shows a schematic view of another audio encoder for providing an encoded audio information on the basis of an input audio information according to embodiments according to the second aspect of the invention.
  • Fig. 12 shows encoder 1200 comprising an optional processing unit 1210 and an optional encoding unit 1220.
  • Encoding unit 1220 may be configured to encode a plurality of quantized spectral values 1222. Furthermore, the encoding unit 1220 may be configured to encode a high frequency energy value or a high frequency energy delta value 1224. Hence, encoder 1200 may provide an encoded audio information 1202 comprising an encoded representation of quantized spectral values and/or HF energy (delta) values.
  • the processing unit 1210 may be configured to provide said quantizes spectral values 1222 based on the input audio information 1204. Moreover, the processing unit 1210 may be configured to provide said HF energy (delta) values 1224 using the input audio information 1204.
  • the high frequency energy value or the high frequency energy delta value may represent an energy of a plurality of spectral coefficients at a frequency below a noise filling end frequency or in a frequency region below the noise filling end frequency which were quantized to zero.
  • the HF energy value (or delta e.g. in case of differential entropy coding) 1224 may represent the original RMS energy of the spectro-temporally normalized spectral coefficients slightly below the noise filling end frequency (e.g., in the 8 -10 kHz frequency range) which were quantized to zero.
  • the processing unit 1210 may further be configured to logarithmically quantize the high frequency energy value or the high frequency energy delta value, as an example hence providing quantized representations of the high frequency energy value or the high frequency energy delta value to the encoding unit 1220.
  • the processing unit 1210 may, for example, be configured to provide a global gain 1212, e.g. GG S f, and/or a noise information 1214, e.g. a noise level, e.g. L S f, to the encoding unit 1220 (that may be determined based on the input audio information 1204).
  • the encoding unit 1220 may optionally be configured to encode the high frequency energy delta value, which may optionally describe the energy of a plurality of spectral coefficients at a frequency below a noise filling end frequency or in a frequency region below the noise filling end frequency which were quantized to zero, relative to a product of the global gain 1212 and of the noise level 1214.
  • the encoding unit 1220 may, for example, be configured to obtain a rounded scaled result of a logarithm of a ratio between the high frequency energy value and a product of the global gain 1212 and of the noise information 1214, e.g. in the form of a noise value , in order to encode the high frequency energy value.
  • processing unit 1210 may be configured to determine the quantized high frequency energy delta value according to
  • Ehf S f 1+round(A*log 2 (EHFsf/(GGsf*L sf )), wherein EHF is a high frequency energy value, wherein GG s t is a global gain 1212, wherein L S f is a noise level 1214, and wherein A is a constant.
  • Fig. 13 shows a schematic view of an audio decoder for providing a decoded audio information on the basis of an encoded audio information according to embodiments according to the third aspect of the invention.
  • Fig. 13 shows audio decoder 1300 comprising an optional spectral holes filling unit 1310 which may be configured to fill spectral holes of a decoded set 1312 of spectral values using respective filling values 1314.
  • the decoder 1300 may further comprise an optional prediction or filtering unit 1320, which may be configured to determine a respective filling value 1314, using a prediction or filtering, such that a given filling value 1314, which is associated with a given frequency, is obtained in dependence on another spectral value 1322, which is associated with a different frequency.
  • an optional prediction or filtering unit 1320 which may be configured to determine a respective filling value 1314, using a prediction or filtering, such that a given filling value 1314, which is associated with a given frequency, is obtained in dependence on another spectral value 1322, which is associated with a different frequency.
  • the decoder 1300 may optionally comprise a filtering strength adaptation unit 1330.
  • the filtering strength adaptation unit 1330 may provide a filtering strength information 1332, e.g. an information about a filtering strength, to the prediction or filtering unit 1320.
  • the filtering strength adaptation unit 1330 may be configured to adapt the filtering strength in dependence on an encoded or quantized spectral value 1334, optionally, e.g. as an alternative, with the spectral value 1322 provided to the prediction or filtering unit 1320, associated with the different frequency.
  • the decoder 1330 may comprise a decoding unit 1340, which may be configured to provide the decoded set 1312 of spectral values to the spectral holes filling unit 1310 using or based on the encoded audio information 1302.
  • the decoding unit 1300 may provide the spectral value 1322 associated with the different frequency, e.g. determined from the encoded audio information 1302, to the prediction or filtering unit 1320 and/or to the filtering strength adaptation unit 1330.
  • the filtering strength determines an impact of the other spectral value 1322 onto the given filling value 1314.
  • the filtering strength adaptation unit 1330 may, for example, be configured to adapt the filtering strength in dependence on the spectral value 1334 associated with the different frequency as it is determined by the encoded representation of individual spectral values in the encoded audio information 1302.
  • the filtering strength may be adapted in dependence on the spectral value 1334 associated with the different frequency before a noise filling is applied and/or in dependence on whether the spectral value 1334 associated with the different frequency (or value) is quantized to zero or not and/or in dependence on whether a noise filling is applied to the spectral value 1334 associated with the different frequency (or value) or not.
  • the prediction or filtering unit 1320 may be configured to selectively apply a filtering in a frequency direction or a prediction in a frequency direction for spectral values for which a noise filling is applied.
  • Fig. 14 shows a schematic view of an audio decoder with additional optional features, according to embodiments according to the third aspect of the invention.
  • Fig. 14 shows decoder 1400, comprising an optional decoding unit 1410, a spectral holes filling unit 1420, a prediction or filtering unit 1430 and a filtering strength adaptation unit 1440. These elements may comprise the same or similar or analogous functionalities and corresponding input and/or output signals as explained in the context of Fig. 13.
  • the audio decoder 1400 e.g. the prediction or filtering unit 1440 of audio decoder 1400, may be configured to apply the prediction or the filtering, in order to determine the given filling value 1432 on the basis of a random or pseudo-random noise values.
  • a noise value information 1436 for example comprising the random or pseudo-random noise values may be provided to the prediction or filtering unit 1430.
  • the noise value information 1436 may, for example, comprise a noise value associated with the given frequency and/or a noise value associated with the other frequency.
  • the spectral value 1454 or 1434 associated with the different frequency may, for example, be a filling value associated with the other frequency.
  • the decoder 1400 may comprise means to provide the noise value information 1436 and or the filling value associated with the different frequency (e.g. therefore in this case optionally not being provided by the decoding unit 1410.
  • the decoding unit 1410 may, for example, provide an information that spectral value was quantized to zero, an said spectral value may be replaced on the basis of the noise value information or on the filling value with the different frequency, e.g. as explained below).
  • the audio decoder 1400 e.g. the prediction or filtering unit 1430 thereof, may be configured to perform a weighted combination of a noise value associated with the given frequency, and of a noise value associated with the other frequency or a weighted combination of a noise value associated with the given frequency, and of a filling value associated with the other frequency, in order to obtain the given filling value 1432.
  • decoder 1400 may comprise a weight adjustment unit 1450, which may be configured to adjust a weight given to the noise value associated with the other frequency or the weight given to the filling value associated with the other frequency in dependence on whether a noise filling has been applied for a spectral value 1454 or 1434 associated with the other frequency. Therefore, a weight information 1452, e.g. comprising an information about a respective weight, may be provided form the weight adjustment unit 1450 to the prediction or filtering unit 1430.
  • a weight information 1452 e.g. comprising an information about a respective weight
  • decoder 1400 may comprise a spectral distance determination unit 1460 which may be configured to determine a spectral distance between the filling value 1430 associated with the given frequency and the other spectral value 1454 or 1434 associated with the different frequency on the basis of the encoded information describing the spectral distance, which is included in the encoded representation 1402 of the audio information.
  • a spectral distance determination unit 1460 which may be configured to determine a spectral distance between the filling value 1430 associated with the given frequency and the other spectral value 1454 or 1434 associated with the different frequency on the basis of the encoded information describing the spectral distance, which is included in the encoded representation 1402 of the audio information.
  • the audio decoder 1400 e.g. the weight adjustment unit 1450, may be configured to determine a weight (wherein the weight information 1452 may comprise the weight), which is applied to the noise value (wherein the noise value information 1436 may comprise the noise value) associated with the given frequency, on the basis of a gain information which is included in the encoded representation 1402 of the audio information.
  • the audio decoder 1400 e.g. the weight adjustment unit 1450, may be configured to determine a weight (wherein the weight information 1452 may comprise the weight), which is applied to the noise value (wherein the noise value information 1436 may comprise the noise value) associated with the other frequency, or to the filling value associated with the other frequency, in dependence on a gain information which is included in the encoded representation of the audio information
  • the encoded representation 1402 of the audio information may comprise an encoded representation of the gain information, that may, for example, be decoded by the decoding unit 1410 and provided to the weight adjustment unit 1450.
  • an adjustment information 1454 optionally comprising the gain information may be provided to the weight adjustment unit 1450.
  • the audio decoder e.g. the weight adjustment unit 1450
  • the audio decoder may be configured to determine the weight, which is applied to the noise value associated with the other frequency, or to the filling value associated with the other frequency, in dependence on a sign information which is included in the encoded representation 1402 of the audio information.
  • the encoded representation 1402 of the audio information may comprise an encoded representation of the sign information, that may, for example, be decoded by the decoding unit 1410 and provided, e.g. in the adjustment information 1454, to the weight adjustment unit 1450.
  • the audio decoder e.g. weight adjustment unit 1450
  • the decoding unit 1410 may for example, provide a parameter information 1412, e.g. comprising the prediction parameter or filtering parameter P’ S f, and/or the constant B and/or the attenuation coefficient d to the prediction or filtering unit 1430.
  • the decoding unit 1410 may optionally only provide the decoded set of spectral values 1422 to the spectral holes filling unit 1420, and the decoder 1400 may comprise one or more dedicated obtaining and/or calculation and/or determination units for providing the respective information, e.g. based on the encoded audio information 1402.
  • the audio decoder 1400 may be configured to mark noise- filled zero-quantized spectral coefficients, and to selectively use a reduced filtering strength which is applied to spectral coefficients which are not marked.
  • Fig. 15 shows an example for a functionality of a decoder, e.g. decoder 1400 shown in Fig. 14 or decoder 1300 shown in Fig. 13, according to embodiments according to the third aspect of the invention.
  • an inventive decoder may be configured to perform the following steps:
  • Fig. 16 shows a schematic view of an audio decoder with additional optional features, according to embodiments according to the third aspect of the invention.
  • Fig. 16 shows decoder 1600 comprising an optional prediction or filtering unit 1610, which may be configured to determine a processed spectral value 1612 using a prediction or filtering, such that a given processed spectral value 1612, which is associated with a given frequency, is obtained in dependence on another spectral value 1614 which is associated with a different frequency.
  • the decoder 1600 may comprise a decoding unit 1620, which may be configured to provide the spectral value 1614 associated with the different frequency to the prediction or filtering unit 1610 based on an encoded audio representation 1602.
  • the decoder 1600 may comprise a filtering strength adaptation unit 1630, that may be configured to adapt a filtering strength in dependence on an encoded or quantized spectral value 1634, e.g. optionally alternatively 1614 associated with the different frequency. Therefore, the filtering strength adaptation unit may provide a filtering strength information 1632 to the prediction or filtering unit 1610. Optionally spectral value 1634 may be provided by the decoding unit based on the encoded audio representation.
  • the filtering strength adaptation unit 1630 may be configured to adapt the filtering strength to reduce a contribution of a nonzero-quantized spectral coefficients included in the prediction or filtering.
  • Fig. 17 shows a block diagram of a method for providing a decoded audio information on the basis of an encoded audio information according to an embodiment according to the first aspect of the invention.
  • Method 1700 comprises deriving 1710 a spectral tilt information from the encoded audio information, using 1720 filling values, in order to fill spectral holes of a decoded set of spectral values and applying 1730 a frequency variable scaling, a spectral tilt of which is determined by the spectral tilt information, to the filling values.
  • Fig.18 shows a block diagram of a method for providing an encoded audio information on the basis of an input audio information according to an embodiment according to the first aspect of the invention.
  • Method 1800 comprises encoding 1810 a plurality of quantized spectral values, determining 1820 a spectral tilt information on the basis of a spectral energy information and a masking envelope information and encoding 1830 the spectral tilt information
  • Fig.19 shows a block diagram of a method for providing a decoded audio information on the basis of an encoded audio information according to an embodiment according to the second aspect of the invention.
  • Method 1900 comprises filling 1910 spectral holes of a decoded set of spectral values, obtaining 1920 a prediction lag information and switching 1930 between a first spectral filling method, in which a frequency filtering or a frequency prediction is used to obtain filling values which are used to fill spectral holes, and one or more further spectral filling methods, in which no frequency filtering and no frequency prediction are used to obtain filling values which are used to fill spectral holes, in dependence on the prediction lag information.
  • Fig. 20 shows a block diagram of a method for providing an encoded audio information on the basis of an input audio information according to an embodiment according to the second aspect of the invention.
  • Method 2000 comprises encoding 2001 a plurality of quantized spectral values, obtaining 2002 a lag value, which defines a characteristic of a filtering operation or of a prediction operation to be performed by an audio decoder for deriving one or more filling values for filling spectral holes, obtaining 2003 a gain value, which defines a characteristic of the filtering operation or of the prediction operation to be performed by an audio decoder for deriving one or more filling values for filling spectral holes, setting 2004 the lag value to zero if the gain value is smaller than a threshold value or if an absolute value of the gain value is smaller than a threshold value, to thereby obtain a modified lag value and encoding 2005 the determined lag value or the modified lag value.
  • Method 2100 comprises filling 2101 spectral holes of a decoded set of spectral values using respective filling values, determining 2102 a filling value using a prediction or filtering, such that a given filling value, which is associated with a given frequency, is obtained in dependence on another spectral value , which is associated with a different frequency and adapting 2103 a filtering strength in dependence on an encoded or quantized spectral value associated with the different frequency.
  • Fig. 22 shows a block diagram of a second method for providing a decoded audio information on the basis of an encoded audio information according to an embodiment according to the third aspect of the invention.
  • Method 2200 comprises determining 2201 a processed spectral value using a prediction or filtering, such that a given processed spectral value, which is associated with a given frequency, is obtained in dependence on another spectral value, which is associated with a different frequency and adapting 2202 a filtering strength in dependence on an encoded or quantized spectral value associated with the different frequency.
  • aspect 1 as explained before may correspond to the following aspect 1.
  • Aspect 2 as explained before, may, for example, correspond to the following aspects 2, 3 and 4.
  • Aspect 3 as explained before, may, for example, correspond to the following aspects 2, 3 and 4.
  • any features, functionalities and details according to any embodiment may be incorporated or used with any other embodiment ,e.g. irrespective of a categorization to different aspects.
  • Such a categorization may for example only be used to provide an example for a clustering of embodiments to facilitate a person skilled in the art to develop a better understanding of the invention.
  • any embodiments as defined by the claims can be supplemented by any of the details (features and functionalities) described in the above mentioned chapters and by any of the details (features and functionalities) described in the context of any other embodiment as disclosed herein.
  • the embodiments described in the above mentioned chapters as well as any of the before discussed embodiments can be used individually, and can also be supplemented by any of the features in another chapter or from any other embodiment, or by any feature included in the claims.
  • features and functionalities disclosed herein relating to a method can also be used in an apparatus (configured to perform such functionality).
  • any features and functionalities disclosed herein with respect to an apparatus can also be used in a corresponding method.
  • the methods disclosed herein can be supplemented by any of the features and functionalities described with respect to the apparatuses.
  • any of the features and functionalities described herein can be implemented in hardware or in software, or using a combination of hardware and software, as will be described in the section “implementation alternatives”.
  • aspects have been described or will be described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.
  • Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be executed by such an apparatus.
  • An inventive encoded audio signal can be stored on a digital storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.
  • a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.
  • embodiments of the invention can be implemented in hardware or in software.
  • the implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.
  • Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
  • embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer.
  • the program code may for example be stored on a machine readable carrier.
  • inventions comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
  • an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
  • a further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.
  • the data carrier, the digital storage medium or the recorded medium are typically tangible and/or non- transitionary.
  • a further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein.
  • the data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.
  • a further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
  • a further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
  • a further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver.
  • the receiver may, for example, be a computer, a mobile device, a memory device or the like.
  • the apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.
  • a programmable logic device for example a field programmable gate array
  • a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein.
  • the methods are preferably performed by any hardware apparatus.
  • the apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.
  • the apparatus described herein, or any components of the apparatus described herein, may be implemented at least partially in hardware and/or in software.
  • the methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.
  • the present invention relates to, for example perceptually, improved ways of calculating spectral envelopes, e.g. as applied in modern audio transform codecs, and/or to improved ways of reconstructing the spectral and/or temporal fine-structure of spectral regions quantized to zero in an encoder.
  • the invention relates to spectral envelopes representing time and/or frequency variant masking thresholds, for example, used during spectral quantization in conventional audio codecs, whereby, as an example, each spectrum may, for example, be divided by the associated masking threshold e.g. prior to quantization and, for example multiplied by it after quantization, yielding, as an example, spectral shaping of the quantization distortion, e.g.
  • spectral envelopes may, for example traditionally, involve the application of some spectral tilt, also, as an example or for example often referred to as "pre-emphasis", to the envelope data prior to quantization, e.g. in order to ensure, for example during the coding bit allocation, a higher coding SNR at low than at high frequencies and, as an example thereby, higher audio quality.
  • some spectral tilt also, as an example or for example often referred to as "pre-emphasis”
  • pre-emphasis to the envelope data prior to quantization, e.g. in order to ensure, for example during the coding bit allocation, a higher coding SNR at low than at high frequencies and, as an example thereby, higher audio quality.
  • EVS Enhanced Voice Services
  • the invention relates to spectral substitution, or "filling", of spectral gaps (zero-quantized frequency coefficients after encoding) caused by coarse quantization at relatively low target bit-rates.
  • low-frequency (LF) spectral content is, for example generally, coded sufficiently accurately e.g. by the abovementioned approach, for example, since the LF SNR is, e.g. due to the application of the spectral tilt onto the actual spectral envelope during the calculation of the masking envelope, for example relatively, high.
  • HF high-frequency
  • the PNS method may signal to the decoder the target energy of a spectral band which has, for example, been quantized to zero in the encoder, and the PNS decoder may insert pseudo-random values into the zero-quantized band, e.g. scaled such that the inserted signal energy matches the signaled target energy.
  • this scheme can, for example, preserve the spectral energy (and, thereby as an example, original spectral envelope) for example quite accurately, e.g. at low bit-rates, it may tend to require many bits, e.g. for signaling of the zero-quantized band energies, which may be counterproductive.
  • the noise filling approaches employed in MPEG-D Extended HE-AAC and 3GPP EVS may improve upon the PNS design, e.g. by allowing to replace zero-quantized spectral coefficients, for example, with pseudo-random values e.g. upon decoding, for example above a certain "noise fill start frequency", e.g. even when a certain spectral band was not entirely quantized to zero in the encoder.
  • the MPEG-D codec may, however, still signal band-wise target energy data for all fully zero-quantized bands, thus as an example increasing the signaling overhead e.g. especially when many bands are zero-quantized.
  • the noise filling method in 3GPP EVS may avoid the transmission of such band-wise energies and, instead, may make use of e.g. only a transmitted spectrally global noise level / and/or a predefined spectral tilt t.
  • the spectral envelope reconstructed in zero-quantized spectral regions may, for example, not directly be given by the original signal's spectral envelope (i.e. , the signal envelope; solid thick black curve) but, for example, by a scaled version of the masking threshold (i.e., the inverse of the normalization envelope used prior to quantization, e.g. used for the spectral shaping of the quantization distortion, for example, as described in the introduction; dashed thick black curve).
  • the masking threshold i.e., the inverse of the normalization envelope used prior to quantization, e.g. used for the spectral shaping of the quantization distortion, for example, as described in the introduction; dashed thick black curve.
  • the decoding result in zero-quantized spectral regions may be the product of the inserted pseudo-random values, the transmitted spectral global noise level 0 ⁇ L ⁇ 1 , and the transmitted masking envelope - no representation of the true spectral envelope may be conveyed from the encoder to the decoder.
  • Figure 2 compares, or may for example show a comparison of, the spectral envelope, e.g. targeted by the EVS gap filling (or, for example, noise filling) algorithm, e.g. in the absence of any spectral tilt compensation.
  • the dashed masking threshold curve has been offset downwards for better visibility of all curves. It can be seen that, e.g.
  • the distance between the masking envelope and the effective reconstructed noise envelope may be constant (thin double arrow) and, for example therefore, may not or does not follow the original spectral envelope accurately.
  • the reconstructed noise envelope may exceed the original spectral envelope at high frequencies, e.g. thus potentially causing audible noise after decoding, while it may remain significantly below the original spectral envelope, e.g. at lower frequencies, for example thus likely causing insufficient gap-fill energy and/or audible spectral holes.
  • a spectral tilt e.g. applied during the calculation of the masking envelope i.e. , the noise shaping envelope, e.g. as explained
  • the noise shaping envelope e.g. as explained
  • Figure 3 illustrates, or may for example show an illustration of an example of, the, for example desired, spectral gap filling behavior.
  • the distance between the noise shaping envelope and the effective reconstructed envelope in the zero-quantized spectral regions (gray curve) is not constant but tilted downwards towards higher frequencies (length of thin double arrow decreases with frequency).
  • This tilt which may, for example, be applied multiplicatively to / in (e.g. to and/or in) a frequency dependent fashion, may intend to compensate for the pre-emphasis tilt applied e.g. during the calculation of the masking envelope, for example in order to recover - or at least approximate - the true spectral envelope of the input signal, e.g. during the gap filling.
  • this tilt t may be a predefined constant, but it may, for example, be observed that, e.g. due to the quantization of the masking envelope (for example by way of low-rate vector quantization in EVS and derived codecs) and/or some input signal dependency, the optimal value of t per frame or transform may, for example, vary quite a bit.
  • A, for example, signal adaptive, frame-wise or transform-wise signaled t would or may, for example, therefore, be, for example, more desirable than e.g. a constant value for t.
  • Signal adaptive choice for example on a per-frame and/or per-subframe basis, e.g. between different methods, e.g. for generating the "artificial" spectral content, for example, used during gap filling, with the choice being signaled, as an example, to an audio transform decoder, e.g. by means of a frequency-domain long-term prediction (FD-LTP) lag parameter.
  • FD-LTP frequency-domain long-term prediction
  • the general idea according to embodiments is to, for example depending on the FD-LTP lag value, choose between a) noise filling with FD-LTP, b) tonality based gap filling, for example, without FD-LTP, e.g. similar to IGF in EVS (a prior-art method), and c) e.g. conventional noise filling, for example, without FD-LTP, e.g. similar to that in EVS or MPEG-D.
  • noise filling with FD-LTP is selected (wherein, as an example, the FD-LTP lag is nonzero)
  • application of a long-term predictive filter e.g. in a spectral domain (e.g. the MDCT domain) for example of the audio transform codec, e.g. during the decoder-side noise filling routine, for example depending on whether a "current" coded FD coefficient is zero and for example on whether a corresponding "previous" coded FD coefficient located at a distance from the current coefficient (specified by the transmitted FD-LTP lag) is zero
  • This aspect or examples of this aspect according to embodiments is described in Sec. 4.3.
  • gap filling without FD-LTP is selected (wherein, for example, the FD-LTP lag is zero)
  • application of a signal adaptive, e.g. copy-up and/or tonality based spectral gap filling procedure for example, similar to the Intelligent Gap Filling (IGF) method used in, e.g., 3GPP EVS and MPEG-H Audio may, for example, be performed.
  • Copy-up may indicate a reconstruction of a zero-quantized FD coefficient from a lower-frequency nonzero-quantized FD coefficient and tonality based may, for example, mean that the copy-up process may be guided by transmitted (sub)frame-wise (e.g.
  • audio tonality data e.g. known from conventional solutions (here, a time-domain LTP or a HPF lag).
  • This e.g. final aspect or examples of this aspect according to embodiments is described in Sec. 4.4.
  • embodiments according to the invention may comprise means in order to fulfill the following prerequisites.
  • embodiments according to the invention may comprise the following features:
  • Prerequisite b transmission of noise shaping envelope (as an example i.e., masking envelope) which is being derived as an example in the encoder, for example, by obtaining spectral band-wise energy or RMS values of the input spectrum.
  • noise shaping envelope as an example i.e., masking envelope
  • Prerequisite c transmission of, e.g. some kind of, frame and/or subframe audio tonality information to a decoder, e.g., by way of a time-domain long-term prediction (TD-LTP) and/or a harmonic post-filtering (e.g. HPF) lag and gain. If such information is present, the (sub)frame can, for example, be considered tonal.
  • TD-LTP time-domain long-term prediction
  • HPF harmonic post-filtering
  • frame and subframe may be used interchangeably.
  • Embodiment E.g. according to Aspect 1 (As an Example, Adaptive Tilt Correction)
  • the general idea according to embodiments behind the transmission of the tilt correction value may, for example, be the calculation and/or low-bit-rate signaling of a difference curve, e.g. in logarithmic intensity domain, for example, between a subframe's e.g. true spectral envelope (as an example, i.e., its input signal envelope, solid black curve in Figs. 2 and 3) and the subframe's masking envelope (as an example, i.e., the noise shaping envelope, dashed black curve in Figs. 2 and 3).
  • the masking envelope may be transmitted to the decoder (e.g. according to, prerequisite b)
  • additional transmission of the difference may, for example allow to, e.g. in the gap and/or noise filling decoding procedure, reconstruct the e.g. true spectral envelope, for example from the masking envelope and/or the tilt related difference curve, for example with better accuracy than in the conventional solutions and/or with fewer side information bits.
  • Figure 3 indicates that the intensity difference between the e.g. true spectral envelope and masking envelope may change monotonically, for example, with frequency.
  • a logarithmic intensity domain e.g., base-10 logarithm
  • the gap and/or noise filling spectral region for example between the two thin vertical arrows
  • the monotonic difference curve was found to resemble a straight line, e.g. most of the time. Therefore, it is proposed, e.g. according to embodiments, to calculate and to parameterize the difference curve, for example, by means of e.g.
  • both envelopes are in said logarithmic domain.
  • f the desired frequency (or, for example, equivalently, the offset of the transform coefficient)
  • T the tilt - or slope - value
  • O an intensity offset
  • both envelopes may, for example, preferably be, represented by spectral band-wise energy (e.g.
  • T and O can, for example, then, be conducted as follows:
  • the value of O S f can, for example, be accounted for in the calculation of 1st (as an example, i.e., can be compensated for by l S f itself). Hence, it may, for example, not be necessary to quantize and to signal O S f to the decoder, e.g. rendering the method very low-rate (only the e.g. 3- bit t S f may, or for example must, be transmitted).
  • T' S f may, for example, still be in the logarithmic domain, as an example i.e., T' S f f may be an additive product, e.g. in the logarithmic domain. Therefore, negating this logarithmic-domain product in the derivation of l S f may imply a division by a linear- domain e.g. equivalent of the product (e.g., 10 rsf ' *) for example in case of calculations performed in a linear domain.
  • the encoder-side step 7 may, for example, be inversely applied, e.g. as follows:
  • Reconstruct tilt value from t S f (e.g. providing) decoded tilt T'sf, use T' s r f for example during multiplication of e.g. final noise level L S f.
  • linear-domain equivalents of the tilt correction product T' s r f may, for example, be multiplied as well, e.g. in a frequency offset (f) dependent fashion.
  • the value range of L S f, T'sf may, for example, be scaled by some constant.
  • the frequency offset f of, for example, each spectral band can, for example, represent e.g. either a) the start frequency of that band (or, for example, equivalently, the offset of the first transform coefficient associated with that band) or b) the band's center frequency (or for example, equivalently, the offset of the first transform coefficient in the band plus half the width, .e.g in number of transform coefficients, of the band). Both options were found to result in almost identical accuracy of the approach.
  • Embodiment E.g. according to Aspect 2 (As an Example, Adaptive Gap-Fill Choice)
  • the state of the art provides for example at least two different approaches to reconstruct zero-quantized spectral regions in audio transform coding: simple noise filling (or PNS), e.g. using pseudo-randomly generated transform coefficient values, and for example more intelligent gap filling (or for example spectral band replication, SBR), e.g. applying copy-up or copy-over from nonzero-quantized spectral coefficients.
  • simple noise filling or PNS
  • PNS pseudo-randomly generated transform coefficient values
  • SBR spectral band replication
  • the general idea according to embodiments and for example behind this aspect or examples of this aspect according to embodiments of the invention is to provide means to adaptively, e.g. based on the (sub)frame's signal characteristic, switch between a noise filling and gap filling solution, the former with optional improved fine temporal shaping, for example as follows.
  • applause-like, rain-like, and/or LF male speech signals can, for example, benefit from improved reconstruction, e.g. of the HF fine temporal signal envelope, e.g. during decoder-side gap and/or noise filling.
  • the for example fine temporal structure of a specific (sub)frame sf can, for example, be parameterized by frequency-domain long-term prediction (FD-LTP) information.
  • FD-LTP frequency-domain long-term prediction
  • FD-LTP lag and/or gain values can, for example, be obtained directly in the audio codec's transform domain; a detailed description follows in Sec. 4.3.
  • the choice of noise and/or gap filling to be applied in a decoder can, for example, be made and/or signaled to the decoder for example depending on the value of said FD-LTP lag p e.g. transmitted in the audio bitstream, for example as follows:
  • the "long-term transientness" detection can, for example, be performed conventionally as in state-of-the- art audio encoders, e.g., by comparing, for example for each subframe, calculated instantaneous (and, for example, possibly temporally smoothed) spectral and/or temporal flatness measurement values to predefined thresholds and for example classifying sf as "long-term transient” e.g. if the temporal flatness is below and the spectral flatness is above the thresholds.
  • the decoder can, for example, select which of the types of spectral filling to apply - gap filling or noise filling with or without FD-LTP filtering - for example, as follows:
  • sf as "tonal” in step 2 can, for example, be based upon the prior-art audio tonality data, e.g., by classifying sf as "tonal” if the audio tonality data is present (as an example, i.e., the TD- LTP / HPF data is nonzero).
  • sf may, for example only, be classified "tonal” if the TD-LTP / HPF gain value is transmitted and maximum.
  • Embodiment E.g. according to Aspect 3 (As an Example, Noise Filling with FD-LTP)
  • the temporal fine structure of coded audio signals can, for example, be reconstructed e.g. more accurately by means of FD-LTP filtering for example during the decoder-side noise filling process.
  • an infinite impulse response (HR) LTP-like filter may, as an example, according to this aspect, be applied to the pseudo-random noise coefficients, e.g, generated during the decoder-side noise filling, resulting, as an example, in a fine temporally shaped noise filling signal.
  • the decision whether to apply noise filling with FD-LTP filtering may for example, be based on FD predictor parameters that may, for example, be determined in the encoder. These predictor parameters - lag index p S f, gain index g sf , and/or sign index s s r - may , for example, preferably, be calculated in the spectrotemporally normalized domain, e.g. utilized before the transform coefficient quantization, as an example i.e., on the (if applicable) TNS analysis filtered transform vector which has, for example, been perceptually normalized (as an example i.e., divided) e.g. by the noise shaping envelope.
  • the TNS analysis filtering may, for example, effectively remove the subframe's coarse temporal envelope while the perceptual normalization may, for example, remove the coarse spectral envelope, e.g. leaving for example only the fine temporal envelope (e.g. to be parameterized by the FD-LTP) and/or the fine spectral envelope (which, for said "long-term transient" signals, may be expected to be negligible).
  • the FD-LTP parameter calculation can, for example, be applied e.g. analogous to conventional TD- LTP and/or HPF calculations:
  • Constant B is, as an example, described in Sec. 4.5.
  • Figure 23 illustrates (or may show an illustration of an example of) the time-domain effect of FD-LTP filtering of a pseudo-random noise spectrum subjected to an inverse transform (as an example i.e., frequency-to-time transformation e.g. using an inverse MDCT). It shows that, e.g. depending on the choices of p S f and/or s S f, the number and location of the shaped peaks can, for example, be varied.
  • decoding steps 4 and 5 may, for example, effectively limit the contribution of lower-frequency non-zero quantized spectral coefficients, e.g. on a given substituted zero quantized spectral coefficient, for example during the FD-LTP filtering.
  • the e.g. same approach may be applied during FD-LPC filtering, e.g., Temporal Noise Shaping (TNS) synthesis filtering, for example to reduce the likelihood of audible clicks e.g. in low-bit-rate audio coding.
  • TMS Temporal Noise Shaping
  • the contribution of nonzero quantized (and for example possibly previously TNS synthesis filtered) lower-frequency spectral coefficients included in the filtering operation can, for example, be limited by attenuating their (e.g. filter output) values, e.g. by X as in step 5, for example when using those values during the filtering operation.
  • Aspect 3 or embodiments according to aspect 3 addressed the need for more accurate fine temporal noise shaping.
  • the desire for more accurate fine spectral noise shaping e.g. especially on highly tonal and/or harmonic audio signals (for example such as speech or isolated musical instruments like acoustic or electric guitars, harpsichords, trumpets)
  • tonality based spectral gap filling method according to further embodiments of the invention, which may, for example, be similar to, e.g., the IGF scheme in 3GPP EVS.
  • the main three differences between the following proposal and the IGF technique may be a) the dependency on an audio tonality parameter - e.g.
  • a TD-LTP or HPF parameter - and/or b) the application of said tonality based gap filling at lower frequencies, as an example i.e., in a HF spectral region usually targeted by noise filling, and/or c) the use of only one HF energy value (or for example delta) - LF spectral shaping may, for example, be realized via l S f and/or the tilt line.
  • step 4 when, in the (sub)frame-wise procedure in Sec. 4.2.1 , step 4 is executed (as an example i.e., tonality based gap filling is selected and the FD-LTP is disabled), harmonically continuous gap substitution may, for example, be applied e.g. according to the "zero filling" approach e.g. described in European patent EP21185666 (Integral Bandwise Parametric Coder, 2021) by Markovic et al. for example with the notable exception that this method is utilized exclusively on the spectrotemporally normalized spectrum in the HF gap filling region in question.
  • This region may, for example, be the spectral range between the typical noise filling start frequency (e.g., 2 kHz) and noise filling end frequency (e.g., 10 kHz), where the latter, in case of superwideband and/or fullband coding, may, for example, equal the traditional IGF start frequency.
  • noise filling start frequency e.g. 2 kHz
  • noise filling end frequency e.g. 10 kHz
  • conventional IGF processing for audio bandwidth extension (ABE) may still be applied above 10 kHz, as an example, i.e., further IGF related whitening / flattening / energy data may be calculated for said IGF ABE region.
  • the HF energy value (or, for example, delta e.g. in case of differential entropy coding) may represent the original RMS energy of the spectro-temporally normalized spectral coefficients e.g. slightly below the noise filling end frequency (e.g., in the 8 -10 kHz frequency range) which may have been quantized to zero.
  • the energy value may, for example preferably be quantized like the scale factors in AAC, as an example i.e., logarithmically in steps of 1.51 dB. Beside the fine spectral envelope, the tonality based gap filling can, for example, therefore, accurately reconstruct also the coarse HF noise spectral envelope.
  • the energy value can, for example, be transmitted as a delta relative to the core coder's global gain and noise level product, as an example i.e., as a "noise gain normalized" value.
  • this may, for example be realized by transmitting a rounded scaled result e.g.
  • the above "noise gain normalized" HF energy delta (as an example, i.e., ratio EHF sf I (GG S f ⁇ L S f)) can, for example, be reconstructed in the decoder, for example according to: where ehf S f and L S f are the transmitted quantized HF energy delta and decoded noise level, respectively, and GG S f denotes the reconstructed global gain value as used for gain normalization in the encoder. Note that value A may be chosen as in the encoder and that the 1" (and the "1 +" in Sec. 4.4.1) may be omitted.
  • the inventive recovery of the desired HF spectral energy may, for example, be achieved e.g. simply by multiplying all generated gap-fill spectral coefficients substituted for zero-quantized coded coefficients in said spectral region, e.g. slightly below the noise filling end frequency (e.g., in the 8 -10 kHz frequency range, for example as stated above) by nrgFac S f for example prior to the application of an inverse (as an example i.e., frequency-to-time) transformation, e.g.
  • the original RMS energy of gap / noise filled spectral values (e.g. of gap and/or noise filled spectral values) slightly below the noise filling end frequency may, for example, be reconstructed closely.
  • the remainder of the decoder-side gap filling operation for example or namely, the application of either a tonality based filling or conventional noise filling, may, for example, depend on the presence of the audio tonality data mentioned earlier.
  • the audio tonality data mentioned earlier.
  • the side information that may for example be required to signal the subframe-wise /, t, and FD-LTP lag and gain or HF energy delta data may, for example, preferably, be of fixed bit length. This may simplify the bit allocation prior to spectral quantization in the encoder.
  • the given frame has more than one subframe (e.g., 2 subframes), then:
  • the FD-LTP lag value can, for example, due to the reduced transform length, be transmitted using 3 instead of 4 bits per subframe which, may, for example, effectively, save two bits for the affected frame.
  • the two bits can, for example, be used (and are, for example, preferably, be used) for other bit allocation control data, e.g. a 2-bit index for example defining how the bit budget available for coding of the spectral coefficient and gap fill data is distributed among the two subframes.
  • the sum of the inventive signaling overhead and for example if applicable, the 2-bit subframe bit distribution information may remain at a constant 12 bits per subframe, e.g. irrespective of the choice of the number of subframes. This may simplify the encoder-side bit allocation and/or quantization steps.
  • the spectral whitening flag can, e.g., be used to distinguish between mid and strong spectral flattening (e.g. if TD-LTP or HPF data is unavailable) and/or between no and mid spectral flattening (e.g. if said data is available) of the copy-up spectral content.
  • mid and strong spectral flattening e.g. if TD-LTP or HPF data is unavailable
  • no and mid spectral flattening e.g. if said data is available
  • the temporal flattening flag can, e.g., be used to signal the activation of TNS-like filtering of the copy- up spectral content e.g. in order to flatten its temporal envelope.
  • TNS-like filtering of the copy- up spectral content e.g. in order to flatten its temporal envelope.
  • Aspect 2 described in Sec. 4.2, introduced for example, inter alia, the signal adaptive choice, on a per-frame and/or a per-subframe basis, e.g. between different methods for generating "artificial" spectral gap-filling content, with the choice being signaled for example to said audio transform decoder e.g. by means of a frequency-domain long-term prediction (FD-LTP) parameter.
  • FD-LTP frequency-domain long-term prediction
  • this FD-LTP parameter preferably constitutes a transform-domain "lag" parameter p s t optionally transmitted in the audio bitstream for said frame or subframe sf.
  • the choice of spectral gap-filling method may depend, for example, on a different FD-LTP parameter instead, namely or for example, the FD-LTP "gain" parameter g S f.
  • the FD-LTP lag p S f and/or sign s s r data do not need to be transmitted (instead, a HF energy value may, for example, be transmitted), and the choice of whether or not to apply noise filling with FD- LTP post-processing (e.g. instead of traditional noise or gap filling without FD-LTP), may depend on the gain e.g. instead of the lag parameter.
  • the decoder-side step 1. e.g. as described in Sec. 4.2.2 would then or may, for example, be writen as follows (note the exchange of p S f and g S f):
  • Appendix 1 Application of FD-LTP Adaptive Filtering Aspect to Temporal Noise Shaping Filtering
  • the proposed strength-adaptive filtering operation defined by steps 4 and 5 in Sec. 4.3.2 can, for example, also be applied to Temporal Noise Shaping (TNS) synthesis filtering. Said two steps may, for example, effectively attenuate the filter (as an example i.e. , its strength) in a sample-index-wise (i) manner, for example if and only if
  • the TNS or FD-LTP filter may be attenuated, e.g. by multiplying each filter weight by l h, e.g. for each i, for example, if and only if
  • the spectral coefficient c(i) may be filtered with a TNS and/or FD-LTP filter, e.g. whose weights have been attenuated.
  • coefficient c(i) may, for example, be filtered with an unaltered TNS and/or FD-LTP filter.
  • the above generalized condition for strength adaptive filtering may, for example, apply to both TNS and FD-LTP.
  • the former may specify the spectrum before noise filling (e.g. used for marking in step 2 in Sec. 4.3.2) while the latter may specify the FD-LTP filtered spectrum after noise filling.
  • Appendix 2 Proposals for three [e.g. partially] independent decoders according to embodiments of the invention
  • Audio transform decoder performing substitution of zero-quantized spectral samples by noise samples, wherein a frame-wise or subframe-wise spectral tilt correction value, t S f, is read from a bit-stream, a frequency dependent tilt curve is derived from t s r, e.g., a line function in a logarithmic domain, and the noise samples substituted for the zero-quantized samples are multiplied 1 by the tilt curve.
  • Audio transform decoder performing or configured to perform a substitution of zero-quantized spectral samples by or using filled samples, wherein a frame-wise or subframe-wise spectral, e.g. LTP, distance value, p S f, is read from a bitstream, a first spectral substitution method, e.g. noise filling or some gap filling, is chosen if p S f indicates zero, and a further spectral substitution method, noise filling + FD-LTP of aspect 3, is chosen otherwise.
  • a frame-wise or subframe-wise spectral e.g. LTP, distance value, p S f
  • a first spectral substitution method e.g. noise filling or some gap filling
  • d (after noise filling) has been chosen here as symbol to differentiate from c (before noise filling), because c(/ - d s t) may, for example, always values before noise filling.
  • the following embodiments may, for example, address adaptive (sub)frame-wise selection of, and switching between, three types of spectral substitution (wherein this functionality may optionally be used in any of the embodiments disclosed herein).
  • the following embodiment may, for example, be an inventive further development, or, for example improvement, of the embodiment explained in section Aspect 2 (adaptive gap-fill choice, Sec. 4.2), but may optionally be used together with any other embodiment disclosed herein or independent from other embodiments.
  • Audio transform decoder e.g. configured to perform or performing substitution of zero-quantized samples, according to embodiment 2, wherein a frame or subframe-wise temporal, e.g. audio tonality, pitch info j S f is read from a bitstream, a first e.g. noise filing, or second, e.g. gap filling, spectral substitution method is chosen if p S f equals zero, the further spectral substitution method, e.g. noise filling + FD-LTP, aspect 3, is chosen otherwise, and the choice between the first and second spectral substitution method depends on pitch info J S f. j S f is not explicitly mentioned in the text as it is known in some conventional solutions (e.g.
  • Embodiments of the invention optionally address the scaling of the RF spectral values quantized (and for example substituted) to zero by 8-10 kHz (e.g. by means of "RF energy (delta) value") in the case of the first or second spectral substitution method (e.g. because only then the RF energy (delta) value is transmitted) . It is to be noted, that this concept (e.g.
  • the scaling of the RF spectral values quantized to zero may, for example be used in combination with the embodiments according to aspect 2, but may optionally be used in combination with any of the other embodiments of the invention or even independently. It is to be noted that, using this RF energy (delta) value may also be valid when legacy noise filling (first spectral substitution method) is selected; thus, this scaling in an 8-10 kHz range may, for example, not be bound to the "copy-up"-based filling method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Quality & Reliability (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

Des modes de réalisation selon l'invention concernent des procédés et des appareils pour un remplissage spectral amélioré par spectrographie dans un codage audio à l'aide d'un filtrage. Des modes de réalisation selon l'invention concernent des procédés et des appareils pour un remplissage d'espace spectral amélioré dans le temps dans un codage audio à l'aide de différents procédés de remplissage de bruit. Des modes de réalisation selon l'invention concernent des procédés et des appareils pour un remplissage spectral amélioré par spectrographie dans un codage audio à l'aide d'une inclinaison.
PCT/EP2022/052151 2021-12-23 2022-01-28 Procédé et appareil pour remplissage spectral amélioré dans le temps d'un espace spectral dans codage audio à l'aide d'un filtrage WO2023117146A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/EP2022/087818 WO2023118605A1 (fr) 2021-12-23 2022-12-23 Procédé et appareil pour un remplissage spectral amélioré dans le temps d'un espace spectral dans un codage audio à l'aide d'un filtrage
TW111149823A TW202333143A (zh) 2021-12-23 2022-12-23 在音訊寫碼中使用濾波用於頻譜時間改善頻譜間隙填充之方法及設備

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP21217661.4 2021-12-23
EP21217661 2021-12-23

Publications (1)

Publication Number Publication Date
WO2023117146A1 true WO2023117146A1 (fr) 2023-06-29

Family

ID=79231053

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2022/052151 WO2023117146A1 (fr) 2021-12-23 2022-01-28 Procédé et appareil pour remplissage spectral amélioré dans le temps d'un espace spectral dans codage audio à l'aide d'un filtrage

Country Status (1)

Country Link
WO (1) WO2023117146A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030233234A1 (en) * 2002-06-17 2003-12-18 Truman Michael Mead Audio coding system using spectral hole filling
US20130013321A1 (en) * 2009-11-12 2013-01-10 Lg Electronics Inc. Apparatus for processing an audio signal and method thereof

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030233234A1 (en) * 2002-06-17 2003-12-18 Truman Michael Mead Audio coding system using spectral hole filling
US20130013321A1 (en) * 2009-11-12 2013-01-10 Lg Electronics Inc. Apparatus for processing an audio signal and method thereof

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
C. R. HELMRICH, A. NIEDERMEIER, S. DISCH, AND F. GHIDO: "Spectral Envelope Reconstruction via IGF for Audio Transform Coding", CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP, 2015, pages 389 - 393, XP055261513, Retrieved from the Internet <URL:https://ieeexploreieee.or_q/document/7177997/> DOI: 10.1109/ICASSP.2015.7177997
C. R. HELMRICHA. NIEDERMEIERS. BAYERB. EDLER: "Low-Complexity Semi-Parametric Joint-Stereo Audio Transform Coding", PROC. EURASIP 23RD EUSIPCO, 2015, pages 799 - 803
C. R. HELMRICHG. MARKOVICB. EDLER: "Improved Low-Delay MDCT-Based Coding of Both Stationary and Transient Audio Signals", PROC. IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP, pages 6954 - 6958, XP032616622, Retrieved from the Internet <URL:https://ieeexplore.ieee.orq/document/6854948/> DOI: 10.1109/ICASSP.2014.6854948
G. FUCHSC. R. HELMRICHG. MARKOVICM. NEUSINGERE. RAVELLIT. MORIYA: "Low Delay LPC and MDCT Based Audio Coding in the EVS Codec", PROC. IEEE INT. CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP, 2015, pages 5723 - 5727, Retrieved from the Internet <URL:https://ieeexplore.ieee.org/document/7179068/>
K. SCHMIDTC. NEUKAM: "Low Complexity Tonality Control in the Intelligent Gap Filling Tool", PROC. IEEE ICASSP, 2016, pages 644 - 648, XP032900680, Retrieved from the Internet <URL:https://ieeexplore.ieee.orq/document/7471754/> DOI: 10.1109/ICASSP.2016.7471754

Similar Documents

Publication Publication Date Title
CN105264597B (zh) 感知转换音频编码中的噪声填充
AU2014211528B2 (en) Apparatus and method for generating a frequency enhanced signal using temporal smoothing of subbands
CN112086107B (zh) 用于辨别和衰减前回声的方法、设备、解码器和存储介质
CN107710324B (zh) 音频编码器和用于对音频信号进行编码的方法
CN111587456A (zh) 时域噪声整形
WO2023117146A1 (fr) Procédé et appareil pour remplissage spectral amélioré dans le temps d&#39;un espace spectral dans codage audio à l&#39;aide d&#39;un filtrage
WO2023117145A1 (fr) Procédé et appareil pour un remplissage d&#39;espace spectral amélioré spectro-temporel dans un codage audio à l&#39;aide de différents procédés de remplissage de bruit
WO2023118598A1 (fr) Procédé et appareil de remplissage d&#39;espace spectral amélioré de manière spectro-temporelle dans un codage audio à l&#39;aide d&#39;une inclinaison
WO2023117144A1 (fr) Procédé et appareil pour un remplissage d&#39;espace spectral amélioré de manière spectrotemporelle dans un codage audio à l&#39;aide d&#39;une inclinaison
WO2023118600A1 (fr) Procédé et appareil de remplissage d&#39;espace spectral amélioré de manière spectro-temporelle dans un codage audio à l&#39;aide de différents procédés de remplissage de bruit
WO2023118605A1 (fr) Procédé et appareil pour un remplissage spectral amélioré dans le temps d&#39;un espace spectral dans un codage audio à l&#39;aide d&#39;un filtrage

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22706004

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)