WO2009029037A1 - Adaptive transition frequency between noise fill and bandwidth extension - Google Patents

Adaptive transition frequency between noise fill and bandwidth extension Download PDF

Info

Publication number
WO2009029037A1
WO2009029037A1 PCT/SE2008/050969 SE2008050969W WO2009029037A1 WO 2009029037 A1 WO2009029037 A1 WO 2009029037A1 SE 2008050969 W SE2008050969 W SE 2008050969W WO 2009029037 A1 WO2009029037 A1 WO 2009029037A1
Authority
WO
WIPO (PCT)
Prior art keywords
frequency
spectral
transition
transition frequency
audio signal
Prior art date
Application number
PCT/SE2008/050969
Other languages
French (fr)
Inventor
Gustaf Ullberg
Manuel Briand
Anisse Taleb
Original Assignee
Telefonaktiebolaget Lm Ericsson (Publ)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=40387561&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=WO2009029037(A1) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Priority to CN200880105330XA priority Critical patent/CN101939782B/en
Priority to JP2010522869A priority patent/JP5183741B2/en
Priority to PL08828148T priority patent/PL2186086T3/en
Priority to ES08828148T priority patent/ES2403410T3/en
Priority to EP08828148A priority patent/EP2186086B1/en
Priority to MX2010001394A priority patent/MX2010001394A/en
Application filed by Telefonaktiebolaget Lm Ericsson (Publ) filed Critical Telefonaktiebolaget Lm Ericsson (Publ)
Priority to US12/674,341 priority patent/US9269372B2/en
Priority to BRPI0815972A priority patent/BRPI0815972B1/en
Publication of WO2009029037A1 publication Critical patent/WO2009029037A1/en
Priority to HK10109588.7A priority patent/HK1143239A1/en
Priority to US14/955,645 priority patent/US9711154B2/en
Priority to US15/639,347 priority patent/US10199049B2/en
Priority to US16/230,777 priority patent/US10878829B2/en
Priority to US17/128,665 priority patent/US11990147B2/en

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/038—Speech enhancement, e.g. noise reduction or echo cancellation using band spreading techniques
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/028—Noise substitution, i.e. substituting non-tonal spectral components by noisy source
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/032—Quantisation or dequantisation of spectral components
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/032—Quantisation or dequantisation of spectral components
    • G10L19/035—Scalar quantisation

Definitions

  • the present invention relates in general to methods and devices for coding and decoding of audio signals, and in particular to methods and devices for spectrum filling.
  • Transform based audio coders compress audio signals by quantizing the transform coefficients. For enabling low bitrates, quantizers might concentrate the available bits on the most energetic and perceptually relevant coefficients and transmit only those, leaving “spectral holes” of unquantized coefficients in the frequency spectrum.
  • SBR Spectrum Band Replication
  • the core codec is responsible for transmitting the lower part of the original spectrum while the SBR-decoder, which is mainly a post-process to the conventional waveform decoder, reconstructs the non-transmitted frequency range.
  • the spectral values of the high band are not transmitted directly as in conventional codecs.
  • the combined system offers a coding gain superior to the gain of the core codec alone.
  • the SBR methodology relies on the definition of a fixed transition frequency between a low band, encoded perceptually relevant low frequencies, and a high band, not encoded less relevant high frequencies.
  • this transition frequency relies on the audio content of the original signal. In other words, from one signal to another, the appropriate transition frequency can vary a lot. This is for instance the case when comparing clean speech and full-band music signals.
  • the "spectral holes" of the decoded spectrum can be divided in two kinds.
  • the first one is small holes at lower frequencies due to the effect of instantaneous masking, see e.g. J. D. Johnston, "Estimation of Perceptual Entropy Using Noise Masking Criteria", Proc. ICASSP, pp. 2524-2527, May 1988 [2].
  • the second one is larger holes at high frequencies resulting from the saturation by the absolute threshold of hearing and the addition of masking [2].
  • the SBR mainly concerns the second kind.
  • a typical audio codec based on such method which aims at filling the "spectral hole", i.e. not encoded coefficients, for the high frequencies, i.e. the second kind of "spectral holes”, should preferably be able to fill the spectral holes over the whole spectrum. Indeed, even if a SBR codec is able to deliver a full bandwidth audio signal, the reconstructed high frequencies will not mask the annoying artefacts introduced by the coding, i.e. quantization, of the low band, i.e. the perceptually relevant low frequencies.
  • a general object of the present invention is to provide methods and devices for enabling efficient suppression of perceptual artefacts caused by spectral holes over a fullband audio signal.
  • a method for spectrum recovery in spectral decoding of an audio signal comprises obtaining of an initial set of spectral coefficients representing the audio signal, and determining a transition frequency.
  • the transition frequency is adapted to a spectral content of the audio signal.
  • Spectral holes in the initial set of spectral coefficients below the transition frequency are noise filled and the initial set of spectral coefficients are bandwidth extended above the transition frequency.
  • a method for use in spectral coding of an audio signal comprises determining of a transition frequency for an initial set of spectral coefficients representing the audio signal.
  • the transition frequency is adapted to a spectral content of the audio signal.
  • the transition frequency defines a border between a frequency range, intended to be a subject for noise filling of spectral holes, and a frequency range, intended to be a subject for bandwidth extension.
  • a decoder for spectral decoding of an audio signal comprises an input for obtaining an initial set of spectral coefficients representing the audio signal and transition determining circuitry arranged for determining a transition frequency.
  • the transition frequency is adapted to a spectral content of the audio signal.
  • the decoder comprises a noise filler for noise filling of spectral holes in the initial set of spectral coefficients below the transition frequency and a bandwidth extender arranged for bandwidth extending the initial set of spectral coefficients above the transition frequency.
  • an encoder for spectral coding of an audio signal comprises transition determining circuitry arranged for determining a transition frequency for an initial set of spectral coefficients representing the audio signal.
  • the transition frequency is adapted to a spectral content of the audio signal.
  • the transition frequency defines a border between a frequency- range, intended to be a subject for noise filling of spectral holes, and a frequency range, intended to be a subject for bandwidth extension.
  • the present invention has a number of advantages.
  • One advantage is that a use of the transition frequency allows the use of a combined spectrum filling using both noise filling and bandwidth extension.
  • the transition frequency is defined adaptively, e.g. according to the coding scheme used, which makes the spectrum filling dependent on e.g. frequency resolution. Any speech and or audio codec using this method is able to deliver a high-quality, i.e. with reduced annoying artefacts, and full bandwidth audio signal.
  • the method is flexible in the sense it can be combined with any kind of frequency representation (DCT, MDCT, etc.) or filter banks, i.e. with any codec (perceptual, parametric, etc.).
  • FIG. 1 is a schematic block scheme of a codec system
  • FIG. 2 is a schematic block scheme of an embodiment of an embodiment of an audio signal encoder according to the present invention
  • FIG. 3 is a schematic illustration of spectral coefficients, groups thereof and frequency bands
  • FIG. 4 is a schematic block scheme of an embodiment of an embodiment of an audio signal decoder according to the present invention
  • FIGS. 5A-C are illustrations of embodiments of principles for finding a transition frequency
  • FIG. 6 is a flow diagram of steps of an embodiment of a method according to the present invention.
  • FIG. 7 is a flow diagram of a step of an embodiment of a signal handling method according to the present invention.
  • FIG. 1 An embodiment of a general codec system for audio signals is schematically illustrated in Fig. 1.
  • An audio source 10 gives rise to an audio signal 15.
  • the audio signal 15 is handled in an encoder 20, which produces a binary flux 25 comprising data representing the audio signal 15.
  • the binary flux 25 may be transmitted, as e.g. in the case of multimedia communication, by a transmission and/or storing arrangement 30.
  • the transmission and/or storing arrangement 30 optionally also may comprise some storing capacity.
  • the binary flux 25 may also only be stored in the transmission and/ or storing arrangement 30, just introducing a time delay in the utilization of the binary flux.
  • the transmission and/ or storing arrangement 30 is thus an arrangement introducing at least one of a spatial repositioning or time delay of the binary flux 25.
  • the binary flux 25 is handled in a decoder 40, which produces an audio output 35 from the data comprised in the binary flux.
  • the audio output 35 should resemble the original audio signal 15 as well as possible under certain constraints.
  • Perceptual audio coding has therefore become an important part for many multimedia services today.
  • the basic principle is to convert the audio signal into spectral coefficients in a frequency domain and using a perceptual model to determine a frequency and time dependent masking of the spectral coefficients.
  • Fig. 2 illustrates an embodiment of an audio encoder 20 according to the present invention.
  • the perceptual audio encoder 20 is a spectral encoder based on a perceptual transformer or a perceptual filter bank.
  • An audio source 15 is received, comprising frames of audio signals x[n].
  • a converter 21 is arranged for converting the time domain audio signal 15 into a set 24 of spectral coefficients X b [n] of a frequency domain.
  • the conversion can e.g. be performed by a Discrete Fourier Transform (DFT), a Discrete Cosine Transform (DCT) or a Modified Discrete Cosine Transform (MDCT).
  • DFT Discrete Fourier Transform
  • DCT Discrete Cosine Transform
  • MDCT Modified Discrete Cosine Transform
  • the converter 21 may thereby typically be constituted by a spectral transformer. The details of the actual transform are of no particular importance for the basic ideas of the present invention and are therefore not further discussed.
  • the set 24 of spectral coefficients i.e. a frequency representation of the input audio signal is provided to a quantizing and coding section 28, where the spectral coefficients are quantized and coded.
  • the quantization is operating for concentrate the available bits on the most energetic and perceptually relevant coefficients. This may be performed using e.g. different kinds of masking thresholds or bandwidth reductions.
  • the result will typically be "spectral holes" of unquantized coefficients in the frequency spectrum. In other words, some of the coefficients are left out on purpose, since they are perceptually less important, for not occupying transmission resources better needed for other purposes. Such spectral holes may then by different reconstructing strategies be corrected or reconstructed at the decoder side.
  • spectral holes of two kinds appear.
  • the first kind comprises spectral holes, single ones or a few neighbouring ones which occur at different places mainly in the low frequency region.
  • the second type is a more or less continuous group of spectral holes at the high-frequency- end of the spectrum.
  • the transition frequency is adapted to a spectral content of the audio signal.
  • the transition frequency is adapted to a spectral content of a present frame of the audio signal, however, the transition frequency may also depend on spectral contents of previous frames of the audio signal, and if there are no serious delay requirements, the transition frequency may also depend on spectral contents of future frames of the audio signal.
  • This adaptation can be performed at the encoder side by a transition determining circuitry 60, typically integrated with the quantizing and coding section 28.
  • the transition determining circuitry 60 can be provided as a separately operating section, whereby only a parameter representing the transition frequency is provided to the different functionalities of the encoder 20.
  • the transition frequency can be used at the encoder side e.g. for providing an appropriate envelope coding for the frequency intervals at the different sides of the transition frequency.
  • the quantizing and coding section 28 is further arranged for packing the coded spectral coefficients together with additional side information into a bitstream according to the transmission or storage standard that is going to be used.
  • a binary flux 25 having data representing the set of spectral coefficients is thereby outputted from the quantizing and coding section 28. Since the transition frequency is derivable directly from the spectral content of the audio signal, the same derivation can be performed on both sides of the transmission interface, i.e. both at the encoder and the decoder. This means that the value of the transition frequency itself not necessarily has to be transmitted among the additional side information. However, it is of course also possible to do that if there is available bit-rate capacity.
  • a MDCT transform is used. After the weighting performed by a psycho acoustic model, the MDCT coefficients are quantized using vector quantization. In vector quantization, VQ, the spectral coefficients are divided into small groups. Each group of coefficients can be seen as a single vector, and each vector is quantized individually.
  • the quantizer may focus the available bits on the most energetic and perceptually relevant groups, resulting in that some groups are set to zero. These groups form spectral holes in the quantized spectrum. This is illustrated in Fig. 3.
  • the groups 70 comprise the same number of spectral coefficients 71, in this case four. However, in alternative embodiments groups having different number of spectral coefficients may also be possible. In one particular embodiment, all groups comprise only one spectral coefficient each, i.e. the group is the same as the spectral coefficient itself.
  • Quantized groups 72 are illustrated in the figure by unfilled rectangles, while groups set to zero 73 are illustrated as black rectangles. It is typically only the quantized groups 72 that are transmitted to any end user.
  • the groups 70 of coefficients are in turn divided into different frequency bands 74. This division is preferably performed according to some psycho acoustical criterion. Groups having essentially similar psycho acoustical properties may thereby be treated collectively.
  • the number of members of each frequency band 74 i.e. the number of groups 70 associated with the frequency bands 74 may therefore differ. If large frequency portions have similar properties, a frequency band covering these frequencies may have a large frequency range. If the psycho acoustic properties change fast over frequencies, this instead calls for frequency bands of a small frequency- range.
  • the routines for spectrum fill may preferably depend on the frequency band to be filled, as discussed more in detail further below.
  • FIG. 4 an embodiment of an audio decoder 40 according to the present invention is illustrated.
  • a binary flux 25 is received, which has properties caused by the encoder described here above.
  • De-quantization and decoding of the received binary flux 25 e.g. a bitstream is performed in a spectral coefficient decoder 41.
  • the spectral coefficient decoder 41 is arranged for decoding spectral coefficients recovered from the binary flux into decoded spectral coefficients X ⁇ [ «] of an initial set of spectral coefficients 42, possible grouped in frequency groups Xf [n] .
  • the initial set of spectral coefficients 42 preferably resembles the set of spectral coefficients provided by the converter of the encoder side, possibly after postprocessing such as e.g. masking thresholds or bandwidth reductions.
  • the application of masking thresholds or bandwidth reductions at the encoder typically results in that the set of spectral coefficients 42 is incomplete in that sense that it typically comprises so-called “spectral holes”.
  • Spectral holes correspond to spectral coefficients that are not received in the binary flux.
  • the spectral holes are undefined or noncoded spectral coefficients X Q [n] or spectral coefficients automatically set to a predetermined value, typically zero, by the spectral coefficient decoder 41. To avoid audible artefacts, these coefficients have to be replaced by estimates (filled) at the decoder.
  • the spectral holes often come in two types. Small spectral holes are typically at the low frequencies, and one or a few big spectral holes typically occur at the high frequencies.
  • the decoder "fills" the spectrum by replacing the spectral holes in the spectrum with estimates of the coefficients. These estimates may be based on side-information transmitted by the decoder and/ or may be dependent on the signal itself. Examples of such useful side-information could be the power envelope of the spectrum and the tonality, i.e. spectral-flatness measure, of the missing coefficients.
  • the present invention relies on the definition of a transition frequency between low and high relevant parts of the spectrum. Based on this information, a typical coding algorithm relying on a high-quality "noise fill” procedure will be able to reduce coding artefacts occurring for low rates and also to regenerate a full bandwidth audio signal even at low rates and with a low complexity scheme based on "bandwidth extension". This will be discussed more in detail further below.
  • the initial set of spectral coefficients 42 from the spectral coefficient decoder 41 is provided to a transition determining circuitry 60.
  • the transition determining circuitry 60 is arranged for determining a transition frequency ft.
  • the initial set of spectral coefficients 42 from the spectral coefficient decoder 41 is also provided to a spectrum filler 43.
  • the spectrum filler 43 is arranged for spectrum filling the initial set of spectral coefficients 42, giving rise to a complete set 44 of reconstructed spectral coefficients X b [n].
  • the set 44 of reconstructed spectral coefficients have typically all spectral coefficients within a certain frequency range defined.
  • the spectrum filler 43 in turn comprises a noise filler 50.
  • the noise filler 50 is arranged for providing a process for noise filling of spectral holes, preferably in the low-frequency region, i.e. below the transition frequency ft.
  • a value is thereby assigned to spectral coefficients in the initial set of spectral coefficients below the transition frequency that are "missing", as a result of not being included in the received coded bitstream.
  • an output 65 from the transition determining circuitry 60 is connected to the noise filler 50, providing information associated with the transition frequency ft.
  • the spectrum filler 43 also comprises a bandwidth extender 55, arranged for bandwidth extending the initial set of spectral coefficients above the transition frequency in order to produce the set 44 of reconstructed spectral coefficients. Therefore, the output 65 from the transition determining circuitry 60 is also connected to the bandwidth extender 55.
  • the set 44 of reconstructed spectral coefficients is provided to a converter 45 connected to the spectrum filler 43.
  • the converter 45 is arranged for converting the set 44 of spectral coefficients of a frequency domain into an audio signal 46 of a time domain.
  • the converter 45 is in the present embodiment based on a perceptual transformer, corresponding to the transformation technique used in the encoder 20 (Fig. 2).
  • the signal is provided back into the time domain with an inverse transform, e.g. Inverse MDCT - IMDCT or Inverse DFT - IDFT, etc.
  • an inverse filter bank may be utilized.
  • the technique of the converter 45 as such is known in prior art, and will not be further discussed.
  • a final perceptually reconstructed audio signal 34 x'[n] is provided at an output 35 for the audio signal, possibly with further treatment steps.
  • the codec must decide in what frequency bands to use noise fill and in what frequency bands to use bandwidth extension. Noise fill gives the best result when most of the groups of the frequency band to be filled are quantized, and there are only minor spectral holes in the band. Bandwidth extension is preferable when a large part of the signal in the high frequencies is left unquantized.
  • One basic method would be to set a fixed transition frequency between the noise fill and bandwidth extension. Spectral holes in the frequency bands or groups under that frequency are filled by noise fill and spectral holes in groups or frequency bands over that frequency are filled by bandwidth extension.
  • the transition frequency is adaptively dependent on a distribution of spectral holes in said initial set of spectral coefficients.
  • a routine for finding a proper transition frequency could be to go through all the frequency bands, starting at the highest (BN) down to 1. If there are no quantized coefficients in the current band, it will be filled by bandwidth extension. If there are quantized coefficients in the band, the holes of this band as well as the following bands are filled using noise fill.
  • a transition frequency is set at the upper limit of the first frequency band seen from the high-frequency side that has a quantized coefficient in it. This is illustrated in Fig. 5A.
  • the spectral holes 77 in band N i.e. above the transition frequency ft are thus filled with 1 ⁇
  • the spectral holes 76 below the transition frequency ft are instead filled by noise filling.
  • Fig. 5B An alternative embodiment is illustrated in Fig. 5B.
  • the definition of the transition frequency is based directly on the groups 70, neglecting the frequency band division.
  • bandwidth extension is used for all groups from the highest frequencies down to the group immediately above the first quantized group 78.
  • the spectral holes 76 below the transition frequency t r are instead filled by noise filling.
  • the transition frequency ft is selected dependent on a proportion of spectral holes in the frequency bands.
  • the codec goes through the frequency bands, starting at the highest down to 1. For each frequency band, the number of coded spectral coefficients or groups is counted. If the number of quantized coefficients or groups divided by the total number of spectral coefficients or groups, i.e. the proportion of coded spectral coefficients, of the frequency band exceeds a certain threshold, the spectral holes of that frequency band and the following frequency bands are filled with noise fill. Otherwise bandwidth extension is used. Analogously, one may monitor the proportion of spectral holes in the frequency bands.
  • a transition frequency band is to be found, which is a highest frequency band in which a proportion of spectral holes is lower than a first threshold.
  • One possibility is to let the threshold itself depend on the frequency. In such a way, a certain proportion of spectral holes may be accepted in the high frequency parts for still using bandwidth expansion techniques, but not in the low frequency parts.
  • the transition frequency is set dependent on, and preferably equal to, an upper frequency limit of the transition frequency band.
  • One alternative is to search for the highest frequency coded spectral coefficient or group and setting the transition frequency at the high frequency side of that group.
  • the transition frequency does not vary too much between consecutive frames. Too large changes can be perceived as disturbing. Therefore, in an exemplary embodiment, the transition frequency is further dependent on a previously used transition frequency. It would for example J. O
  • transition frequency could be inputted as a value into a filter together with previous transition frequencies, giving a modified transition frequency having a more damped change behaviour. The transition frequency will then depend on more than one previous transition frequency.
  • routines are typically performed in the transition determining circuitry, i.e. preferably in the quantizing and coding section of the encoder and in the decoder, respectively.
  • Fig. 6 is a flow diagram illustrating steps of an embodiment of a method according to the present invention.
  • a method for spectrum recovery in spectral decoding of an audio signal starts in step 200.
  • step 210 an initial set of spectral coefficients representing the audio signal is obtained.
  • step 212 a transition frequency is determined. The transition frequency is adapted to a spectral content of the audio signal. Noise filling of spectral holes in the initial set of spectral coefficients below the transition frequency is performed in step 214 and bandwidth extending of the initial set of spectral coefficients above the transition frequency is performed in step 216.
  • the process ends in step 249.
  • Fig. 7 is a flow diagram illustrating a step of an embodiment of another method according to the present invention.
  • a method for use in spectral coding of an audio signal begins in step 200.
  • a transition frequency is determined.
  • the transition frequency for an initial set of spectral coefficients representing the audio signal is adapted to a spectral content of the audio signal.
  • the transition frequency defining a border between a frequency range, intended to be a subject for noise filling of spectral holes, and a frequency range, intended to be a subject for bandwidth extension.
  • the present invention acquires a number of advantages by the adaptive definition of the transition frequency according to the used coding scheme.
  • the adapted transition frequency allows the efficient use of a combined spectrum filling using both noise filling and bandwidth extension.
  • Any speech and or audio codec using this method is able to deliver a high-quality and full bandwidth audio signal with annoying artefacts reduced.
  • the method is flexible in the sense it can be combined with any kind of frequency representation (DCT, MDCT, etc.) or filter banks, i.e. with any codec (perceptual, parametric, etc.).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Quality & Reliability (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Abstract

A method for spectrum recovery in spectral decoding of an audio signal, comprises obtaining (210) of an initial set of spectral coefficients representing the audio signal, and determining (212) a transition frequency. The transition frequency is adapted to a spectral content of the audio signal. Spectral holes in the initial set of spectral coefficients below the transition frequency are noise filled (214) and the initial set of spectral coefficients are bandwidth extended (216) above the transition frequency. Decoders and encoders being arranged for performing part of or the entire method are also illustrated.

Description

ADAPTIVE TRANSITION FREQUENCY BETWEEN NOISE FILL
AND BANDWIDTH EXTENSION
TECHNICAL FIELD
The present invention relates in general to methods and devices for coding and decoding of audio signals, and in particular to methods and devices for spectrum filling.
BACKGROUND
When audio signals are to be stored and/ or transmitted, a standard approach today is to code the audio signals into a digital representation according to different schemes. In order to save storage and/or transmission capacity, it is e general wish to reduce the size of the digital representation needed to allow reconstruction of the audio signals with sufficient quality. The trade-off between size of the coded signal and signal quality depends on the actual application.
Transform based audio coders compress audio signals by quantizing the transform coefficients. For enabling low bitrates, quantizers might concentrate the available bits on the most energetic and perceptually relevant coefficients and transmit only those, leaving "spectral holes" of unquantized coefficients in the frequency spectrum.
The so-called SBR (Spectral Band Replication) technology, see e.g. 3GPP TS 26.404 V6.0.0 (2004-09), " Enhanced aacPlus general audio codec - encoder SBR part (Release 6)", 2004 [1], closes the gap between the band-limited signal of a conventional perceptual coder and the audible bandwidth of approximately 15 kHz. The general idea behind SBR is to recreate the missing high frequency contents of a decoded signal in a perceptually accurate manner. The frequencies above 15 kHz are less important from a psychoacoustic point of view, but may also be reconstructed. However, SBR cannot be used as a standalone codec. It always operates in conjunction with a conventional waveform codec, a so-called core codec. The core codec is responsible for transmitting the lower part of the original spectrum while the SBR-decoder, which is mainly a post-process to the conventional waveform decoder, reconstructs the non-transmitted frequency range. The spectral values of the high band are not transmitted directly as in conventional codecs. The combined system offers a coding gain superior to the gain of the core codec alone.
The SBR methodology relies on the definition of a fixed transition frequency between a low band, encoded perceptually relevant low frequencies, and a high band, not encoded less relevant high frequencies. However, in practice, this transition frequency relies on the audio content of the original signal. In other words, from one signal to another, the appropriate transition frequency can vary a lot. This is for instance the case when comparing clean speech and full-band music signals.
The "spectral holes" of the decoded spectrum can be divided in two kinds. The first one is small holes at lower frequencies due to the effect of instantaneous masking, see e.g. J. D. Johnston, "Estimation of Perceptual Entropy Using Noise Masking Criteria", Proc. ICASSP, pp. 2524-2527, May 1988 [2]. The second one is larger holes at high frequencies resulting from the saturation by the absolute threshold of hearing and the addition of masking [2]. The SBR mainly concerns the second kind.
Moreover, a typical audio codec based on such method which aims at filling the "spectral hole", i.e. not encoded coefficients, for the high frequencies, i.e. the second kind of "spectral holes", should preferably be able to fill the spectral holes over the whole spectrum. Indeed, even if a SBR codec is able to deliver a full bandwidth audio signal, the reconstructed high frequencies will not mask the annoying artefacts introduced by the coding, i.e. quantization, of the low band, i.e. the perceptually relevant low frequencies. SUMMARY
A general object of the present invention is to provide methods and devices for enabling efficient suppression of perceptual artefacts caused by spectral holes over a fullband audio signal.
The above objects are achieved by methods and devices according to the enclosed patent claims. In general words, according to a first aspect, a method for spectrum recovery in spectral decoding of an audio signal, comprises obtaining of an initial set of spectral coefficients representing the audio signal, and determining a transition frequency. The transition frequency is adapted to a spectral content of the audio signal. Spectral holes in the initial set of spectral coefficients below the transition frequency are noise filled and the initial set of spectral coefficients are bandwidth extended above the transition frequency.
According to a second aspect, a method for use in spectral coding of an audio signal comprises determining of a transition frequency for an initial set of spectral coefficients representing the audio signal. The transition frequency is adapted to a spectral content of the audio signal. The transition frequency defines a border between a frequency range, intended to be a subject for noise filling of spectral holes, and a frequency range, intended to be a subject for bandwidth extension.
According to a third aspect, a decoder for spectral decoding of an audio signal comprises an input for obtaining an initial set of spectral coefficients representing the audio signal and transition determining circuitry arranged for determining a transition frequency. The transition frequency is adapted to a spectral content of the audio signal. The decoder comprises a noise filler for noise filling of spectral holes in the initial set of spectral coefficients below the transition frequency and a bandwidth extender arranged for bandwidth extending the initial set of spectral coefficients above the transition frequency. „
According to a fourth aspect, an encoder for spectral coding of an audio signal comprises transition determining circuitry arranged for determining a transition frequency for an initial set of spectral coefficients representing the audio signal. The transition frequency is adapted to a spectral content of the audio signal. The transition frequency defines a border between a frequency- range, intended to be a subject for noise filling of spectral holes, and a frequency range, intended to be a subject for bandwidth extension.
The present invention has a number of advantages. One advantage is that a use of the transition frequency allows the use of a combined spectrum filling using both noise filling and bandwidth extension. Furthermore, the transition frequency is defined adaptively, e.g. according to the coding scheme used, which makes the spectrum filling dependent on e.g. frequency resolution. Any speech and or audio codec using this method is able to deliver a high-quality, i.e. with reduced annoying artefacts, and full bandwidth audio signal. The method is flexible in the sense it can be combined with any kind of frequency representation (DCT, MDCT, etc.) or filter banks, i.e. with any codec (perceptual, parametric, etc.).
BRIEF DESCRIPTION OF THE DRAWINGS
The invention, together with further objects and advantages thereof, may best be understood by making reference to the following description taken together with the accompanying drawings, in which:
FIG. 1 is a schematic block scheme of a codec system;
FIG. 2 is a schematic block scheme of an embodiment of an embodiment of an audio signal encoder according to the present invention;
FIG. 3 is a schematic illustration of spectral coefficients, groups thereof and frequency bands;
FIG. 4 is a schematic block scheme of an embodiment of an embodiment of an audio signal decoder according to the present invention; FIGS. 5A-C are illustrations of embodiments of principles for finding a transition frequency;
FIG. 6 is a flow diagram of steps of an embodiment of a method according to the present invention; and
FIG. 7 is a flow diagram of a step of an embodiment of a signal handling method according to the present invention.
DETAILED DESCRIPTION
Throughout the drawings, the same reference numbers are used for similar or corresponding elements.
An embodiment of a general codec system for audio signals is schematically illustrated in Fig. 1. An audio source 10 gives rise to an audio signal 15. The audio signal 15 is handled in an encoder 20, which produces a binary flux 25 comprising data representing the audio signal 15. The binary flux 25 may be transmitted, as e.g. in the case of multimedia communication, by a transmission and/or storing arrangement 30. The transmission and/or storing arrangement 30 optionally also may comprise some storing capacity. The binary flux 25 may also only be stored in the transmission and/ or storing arrangement 30, just introducing a time delay in the utilization of the binary flux. The transmission and/ or storing arrangement 30 is thus an arrangement introducing at least one of a spatial repositioning or time delay of the binary flux 25. When being used, the binary flux 25 is handled in a decoder 40, which produces an audio output 35 from the data comprised in the binary flux. Typically, the audio output 35 should resemble the original audio signal 15 as well as possible under certain constraints.
In many real-time applications, the time delay between the production of the original audio signal 15 and the produced audio output 35 is typically not allowed to exceed a certain time. If the transmission resources at the same time are limited, the available bit-rate is also typically low. In order to utilize the available bit-rate in a best possible manner, perceptual audio coding has been developed. Perceptual audio coding has therefore become an important part for many multimedia services today. The basic principle is to convert the audio signal into spectral coefficients in a frequency domain and using a perceptual model to determine a frequency and time dependent masking of the spectral coefficients.
Fig. 2 illustrates an embodiment of an audio encoder 20 according to the present invention. In this particular embodiment, the perceptual audio encoder 20 is a spectral encoder based on a perceptual transformer or a perceptual filter bank. An audio source 15 is received, comprising frames of audio signals x[n].
In a typical spectral encoder, a converter 21 is arranged for converting the time domain audio signal 15 into a set 24 of spectral coefficients Xb [n] of a frequency domain. In a typical transform encoder, the conversion can e.g. be performed by a Discrete Fourier Transform (DFT), a Discrete Cosine Transform (DCT) or a Modified Discrete Cosine Transform (MDCT). The converter 21 may thereby typically be constituted by a spectral transformer. The details of the actual transform are of no particular importance for the basic ideas of the present invention and are therefore not further discussed.
The set 24 of spectral coefficients, i.e. a frequency representation of the input audio signal is provided to a quantizing and coding section 28, where the spectral coefficients are quantized and coded. Typically, the quantization is operating for concentrate the available bits on the most energetic and perceptually relevant coefficients. This may be performed using e.g. different kinds of masking thresholds or bandwidth reductions. The result will typically be "spectral holes" of unquantized coefficients in the frequency spectrum. In other words, some of the coefficients are left out on purpose, since they are perceptually less important, for not occupying transmission resources better needed for other purposes. Such spectral holes may then by different reconstructing strategies be corrected or reconstructed at the decoder side. Typically, spectral holes of two kinds appear. The first kind comprises spectral holes, single ones or a few neighbouring ones which occur at different places mainly in the low frequency region. The second type is a more or less continuous group of spectral holes at the high-frequency- end of the spectrum.
According to the present invention, it is favourable to treat these two different kinds of spectral holes in different ways, in order to achieve an as efficient spectrum filling as possible. One parameter to determine is then a transition frequency, at which the different fill approaches meet, a so called transition frequency. Since the distribution of spectral holes differs between different kinds of audio signals, the optimum choice of transition frequency also differ, According to the present invention, the transition frequency is adapted to a spectral content of the audio signal. Typically, the transition frequency is adapted to a spectral content of a present frame of the audio signal, however, the transition frequency may also depend on spectral contents of previous frames of the audio signal, and if there are no serious delay requirements, the transition frequency may also depend on spectral contents of future frames of the audio signal. This adaptation can be performed at the encoder side by a transition determining circuitry 60, typically integrated with the quantizing and coding section 28. However, in alternative embodiments, the transition determining circuitry 60 can be provided as a separately operating section, whereby only a parameter representing the transition frequency is provided to the different functionalities of the encoder 20. The transition frequency can be used at the encoder side e.g. for providing an appropriate envelope coding for the frequency intervals at the different sides of the transition frequency.
The quantizing and coding section 28 is further arranged for packing the coded spectral coefficients together with additional side information into a bitstream according to the transmission or storage standard that is going to be used. A binary flux 25 having data representing the set of spectral coefficients is thereby outputted from the quantizing and coding section 28. Since the transition frequency is derivable directly from the spectral content of the audio signal, the same derivation can be performed on both sides of the transmission interface, i.e. both at the encoder and the decoder. This means that the value of the transition frequency itself not necessarily has to be transmitted among the additional side information. However, it is of course also possible to do that if there is available bit-rate capacity.
In a particular embodiment, a MDCT transform is used. After the weighting performed by a psycho acoustic model, the MDCT coefficients are quantized using vector quantization. In vector quantization, VQ, the spectral coefficients are divided into small groups. Each group of coefficients can be seen as a single vector, and each vector is quantized individually.
For instance, due to high restrictions on the bit rate, the quantizer may focus the available bits on the most energetic and perceptually relevant groups, resulting in that some groups are set to zero. These groups form spectral holes in the quantized spectrum. This is illustrated in Fig. 3. In the present embodiment, the groups 70 comprise the same number of spectral coefficients 71, in this case four. However, in alternative embodiments groups having different number of spectral coefficients may also be possible. In one particular embodiment, all groups comprise only one spectral coefficient each, i.e. the group is the same as the spectral coefficient itself. Quantized groups 72 are illustrated in the figure by unfilled rectangles, while groups set to zero 73 are illustrated as black rectangles. It is typically only the quantized groups 72 that are transmitted to any end user.
The groups 70 of coefficients are in turn divided into different frequency bands 74. This division is preferably performed according to some psycho acoustical criterion. Groups having essentially similar psycho acoustical properties may thereby be treated collectively. The number of members of each frequency band 74, i.e. the number of groups 70 associated with the frequency bands 74 may therefore differ. If large frequency portions have similar properties, a frequency band covering these frequencies may have a large frequency range. If the psycho acoustic properties change fast over frequencies, this instead calls for frequency bands of a small frequency- range. The routines for spectrum fill may preferably depend on the frequency band to be filled, as discussed more in detail further below.
At the decoding stage, the inverse operation is basically achieved. In Fig. 4, an embodiment of an audio decoder 40 according to the present invention is illustrated. A binary flux 25 is received, which has properties caused by the encoder described here above. De-quantization and decoding of the received binary flux 25 e.g. a bitstream is performed in a spectral coefficient decoder 41. The spectral coefficient decoder 41 is arranged for decoding spectral coefficients recovered from the binary flux into decoded spectral coefficients Xδ[«] of an initial set of spectral coefficients 42, possible grouped in frequency groups Xf [n] . The initial set of spectral coefficients 42 preferably resembles the set of spectral coefficients provided by the converter of the encoder side, possibly after postprocessing such as e.g. masking thresholds or bandwidth reductions.
As discussed further above, the application of masking thresholds or bandwidth reductions at the encoder typically results in that the set of spectral coefficients 42 is incomplete in that sense that it typically comprises so-called "spectral holes". "Spectral holes" correspond to spectral coefficients that are not received in the binary flux. In other words, the spectral holes are undefined or noncoded spectral coefficients XQ[n] or spectral coefficients automatically set to a predetermined value, typically zero, by the spectral coefficient decoder 41. To avoid audible artefacts, these coefficients have to be replaced by estimates (filled) at the decoder.
The spectral holes often come in two types. Small spectral holes are typically at the low frequencies, and one or a few big spectral holes typically occur at the high frequencies.
To minimize artefacts in the decoded audio signal, the decoder "fills" the spectrum by replacing the spectral holes in the spectrum with estimates of the coefficients. These estimates may be based on side-information transmitted by the decoder and/ or may be dependent on the signal itself. Examples of such useful side-information could be the power envelope of the spectrum and the tonality, i.e. spectral-flatness measure, of the missing coefficients.
Two different methods can be used to fill the different kinds of spectral holes. "Noise fill" works well for spectral holes in the lower frequencies, while "bandwidth extension" is more suitable at high frequencies. The present invention describes a method to decide where noise fill and bandwidth extension should be used, respectively.
The present invention relies on the definition of a transition frequency between low and high relevant parts of the spectrum. Based on this information, a typical coding algorithm relying on a high-quality "noise fill" procedure will be able to reduce coding artefacts occurring for low rates and also to regenerate a full bandwidth audio signal even at low rates and with a low complexity scheme based on "bandwidth extension". This will be discussed more in detail further below.
The initial set of spectral coefficients 42 from the spectral coefficient decoder 41, typically comprising a certain amount of spectral holes, is provided to a transition determining circuitry 60. The transition determining circuitry 60 is arranged for determining a transition frequency ft.
The initial set of spectral coefficients 42 from the spectral coefficient decoder 41 is also provided to a spectrum filler 43. The spectrum filler 43 is arranged for spectrum filling the initial set of spectral coefficients 42, giving rise to a complete set 44 of reconstructed spectral coefficients Xb [n]. The set 44 of reconstructed spectral coefficients have typically all spectral coefficients within a certain frequency range defined. The spectrum filler 43 in turn comprises a noise filler 50. The noise filler 50 is arranged for providing a process for noise filling of spectral holes, preferably in the low-frequency region, i.e. below the transition frequency ft. A value is thereby assigned to spectral coefficients in the initial set of spectral coefficients below the transition frequency that are "missing", as a result of not being included in the received coded bitstream. To this end, an output 65 from the transition determining circuitry 60 is connected to the noise filler 50, providing information associated with the transition frequency ft.
The spectrum filler 43 also comprises a bandwidth extender 55, arranged for bandwidth extending the initial set of spectral coefficients above the transition frequency in order to produce the set 44 of reconstructed spectral coefficients. Therefore, the output 65 from the transition determining circuitry 60 is also connected to the bandwidth extender 55.
As mentioned above, the result from the spectrum filler 43 is a complete set
44 of reconstructed spectral coefficients Xb [n], having all spectral coefficients within a certain frequency range defined.
The set 44 of reconstructed spectral coefficients is provided to a converter 45 connected to the spectrum filler 43. The converter 45 is arranged for converting the set 44 of spectral coefficients of a frequency domain into an audio signal 46 of a time domain. The converter 45 is in the present embodiment based on a perceptual transformer, corresponding to the transformation technique used in the encoder 20 (Fig. 2). In a particular embodiment, the signal is provided back into the time domain with an inverse transform, e.g. Inverse MDCT - IMDCT or Inverse DFT - IDFT, etc. In other embodiments an inverse filter bank may be utilized. As at the encoder side, the technique of the converter 45 as such, is known in prior art, and will not be further discussed. A final perceptually reconstructed audio signal 34 x'[n] is provided at an output 35 for the audio signal, possibly with further treatment steps. The codec must decide in what frequency bands to use noise fill and in what frequency bands to use bandwidth extension. Noise fill gives the best result when most of the groups of the frequency band to be filled are quantized, and there are only minor spectral holes in the band. Bandwidth extension is preferable when a large part of the signal in the high frequencies is left unquantized.
One basic method would be to set a fixed transition frequency between the noise fill and bandwidth extension. Spectral holes in the frequency bands or groups under that frequency are filled by noise fill and spectral holes in groups or frequency bands over that frequency are filled by bandwidth extension.
A problem with this approach is, however, that the optimal transition frequency is not the same for all audio signals. Some signals have most of the energy concentrated in the low frequencies and a big part of the signal could be subject to bandwidth extension. Other signals have their energy more evenly spread over the spectrum and these signals may benefit from using only noise fill.
According to one embodiment of a method according to the present invention the transition frequency is adaptively dependent on a distribution of spectral holes in said initial set of spectral coefficients. A routine for finding a proper transition frequency could be to go through all the frequency bands, starting at the highest (BN) down to 1. If there are no quantized coefficients in the current band, it will be filled by bandwidth extension. If there are quantized coefficients in the band, the holes of this band as well as the following bands are filled using noise fill. Thus a transition frequency is set at the upper limit of the first frequency band seen from the high-frequency side that has a quantized coefficient in it. This is illustrated in Fig. 5A. The spectral holes 77 in band N, i.e. above the transition frequency ft are thus filled with 1 ό
bandwidth extension approaches. The spectral holes 76 below the transition frequency ft are instead filled by noise filling.
An alternative embodiment is illustrated in Fig. 5B. Here the definition of the transition frequency is based directly on the groups 70, neglecting the frequency band division. Here, bandwidth extension is used for all groups from the highest frequencies down to the group immediately above the first quantized group 78. The spectral holes 76 below the transition frequency tr are instead filled by noise filling.
These methods are more adaptive to the audio signal and the quantizer, i.e. the coding scheme, but it may experience minor problems when the signal is quantized e.g. according to Fig. 5C. Here, a big part of the high frequencies of the signal is set to zero, and bandwidth extension should preferably be used from band B9 to B 12. However, since there is a single coded quantized group 79 in frequency band BI l, bandwidth extension will be completely disabled below this quantized group 79 and noise fill will be used at all bands up to this group 79.
To avoid also this problem, another embodiment is also proposed, where the transition frequency ft is selected dependent on a proportion of spectral holes in the frequency bands. Like in the previous embodiments, the codec goes through the frequency bands, starting at the highest down to 1. For each frequency band, the number of coded spectral coefficients or groups is counted. If the number of quantized coefficients or groups divided by the total number of spectral coefficients or groups, i.e. the proportion of coded spectral coefficients, of the frequency band exceeds a certain threshold, the spectral holes of that frequency band and the following frequency bands are filled with noise fill. Otherwise bandwidth extension is used. Analogously, one may monitor the proportion of spectral holes in the frequency bands. In other words, a transition frequency band is to be found, which is a highest frequency band in which a proportion of spectral holes is lower than a first threshold. There are also alternative criteria to select the transition frequency band. One possibility" is to let the threshold itself depend on the frequency. In such a way, a certain proportion of spectral holes may be accepted in the high frequency parts for still using bandwidth expansion techniques, but not in the low frequency parts. Anyone skilled in the art realizes that the details in selecting appropriate criteria can be varied in many ways, e.g. being dependent on other signal related properties or other side information.
In one embodiment, the transition frequency is set dependent on, and preferably equal to, an upper frequency limit of the transition frequency band. However, there are also various alternatives. One alternative is to search for the highest frequency coded spectral coefficient or group and setting the transition frequency at the high frequency side of that group.
The algorithm of the embodiment described above can also be described with the following pseudo code:
For currentBand = N to 1 ratio = numCodedCoefflnBand(currentBand) / numCoefflnBand(currentBand)
If ratio > threshold
Transition is between currentBand and currentBand + 1 Return
End if Next Transition is at the start of band 1
It is preferred if the transition frequency does not vary too much between consecutive frames. Too large changes can be perceived as disturbing. Therefore, in an exemplary embodiment, the transition frequency is further dependent on a previously used transition frequency. It would for example J. O
be possible to prohibit the transition frequency to change more than a predetermined absolute or relative amount between two consecutive frames. Alternatively, a provisional transition frequency could be inputted as a value into a filter together with previous transition frequencies, giving a modified transition frequency having a more damped change behaviour. The transition frequency will then depend on more than one previous transition frequency.
These routines are typically performed in the transition determining circuitry, i.e. preferably in the quantizing and coding section of the encoder and in the decoder, respectively.
Fig. 6 is a flow diagram illustrating steps of an embodiment of a method according to the present invention. A method for spectrum recovery in spectral decoding of an audio signal starts in step 200. In step 210, an initial set of spectral coefficients representing the audio signal is obtained. In step 212, a transition frequency is determined. The transition frequency is adapted to a spectral content of the audio signal. Noise filling of spectral holes in the initial set of spectral coefficients below the transition frequency is performed in step 214 and bandwidth extending of the initial set of spectral coefficients above the transition frequency is performed in step 216. The process ends in step 249.
Analogously, Fig. 7 is a flow diagram illustrating a step of an embodiment of another method according to the present invention. A method for use in spectral coding of an audio signal begins in step 200. In step 212, a transition frequency is determined. The transition frequency for an initial set of spectral coefficients representing the audio signal is adapted to a spectral content of the audio signal. The transition frequency defining a border between a frequency range, intended to be a subject for noise filling of spectral holes, and a frequency range, intended to be a subject for bandwidth extension. The present invention acquires a number of advantages by the adaptive definition of the transition frequency according to the used coding scheme. The adapted transition frequency allows the efficient use of a combined spectrum filling using both noise filling and bandwidth extension. Any speech and or audio codec using this method is able to deliver a high-quality and full bandwidth audio signal with annoying artefacts reduced. The method is flexible in the sense it can be combined with any kind of frequency representation (DCT, MDCT, etc.) or filter banks, i.e. with any codec (perceptual, parametric, etc.).
The embodiments described above are to be understood as a few illustrative examples of the present invention. It will be understood by those skilled in the art that various modifications, combinations and changes may be made to the embodiments without departing from the scope of the present invention. In particular, different part solutions in the different embodiments can be combined in other configurations, where technically possible. The scope of the present invention is, however, defined by the appended claims.
REFERENCES
[1] 3GPP TS 26.404 V6.0.0 (2004-09), " Enhanced aacPlus general audio codec - encoder SBR part (Release 6)", 2004
[2] J. D. Johnston, "Estimation of Perceptual Entropy Using Noise Masking Criteria", Proc. ICASSP, pp. 2524-2527, May 1988.

Claims

1. Method for spectrum recovery in spectral decoding of an audio signal, comprising the steps of: obtaining (210) an initial set (42) of spectral coefficients representing said audio signal; determining (212) a transition frequency (ft); noise filling (214) of spectral holes in said initial set (42) of spectral coefficients below said transition frequency (ft); and bandwidth extending (216) said initial set (42) of spectral coefficients above said transition frequency (ft); said transition frequency (ft) being adapted to a spectral content of said audio signal.
2. Method according to claim 1, wherein said transition frequency (ft) is adaptively dependent on a distribution of spectral holes in said initial set (42) of spectral coefficients.
3. Method according to claim 2, wherein said step of determining said transition frequency (ft) in turn comprises the steps of: dividing said spectral coefficients of said initial set (42) of spectral coefficients into a plurality of frequency bands (74); and selecting said transition frequency (ft) dependent on a proportion of spectral holes in said frequency bands (74) .
4. Method according to claim 3, wherein said frequency bands (74) have a constant frequency width.
5. Method according to claim 3, wherein at least two of said frequency bands (74) have different frequency widths.
6. Method according to any of the claims 3 to 5, wherein said step of selecting said transition frequency (ft) comprises: finding a transition frequency band, being a highest frequency band in which said proportion is lower than a first threshold.
7. Method according to claim 6, wherein said step of selecting said transition frequency (ft) further comprises: setting said transition frequency (ft) dependent on an upper frequency limit of said transition frequency band.
8. Method according to claim 6 or 7, wherein said step of setting said transition frequency (ft) is further dependent on a previously used transition frequency.
9. Method according to claim 8, wherein said step of setting said transition frequency (ft) is further dependent on more than one previously used transition frequency.
10. Method according to claim 8 or 9, wherein said transition frequency (ft) is prohibited to change more than a predetermined absolute or relative amount between two consecutive frames.
11. Method for use in spectral coding of an audio signal, comprising: determining (212) a transition frequency (ft) for an initial set (24; 42) of spectral coefficients representing said audio signal; said transition frequency (ft) defining a border between a frequency range, intended to be a subject for noise filling of spectral holes, and a frequency range, intended to be a subject for bandwidth extension; said transition frequency (ft) being adapted to a spectral content of said audio signal.
12. Decoder (40) for spectral decoding of an audio signal, comprising: input for obtaining an initial set (42) of spectral coefficients representing said audio signal; transition determining circuitry (60) arranged for determining a transition frequency (ft); a noise filler (50) for noise filling of spectral holes in said initial set (42)of spectral coefficients below said transition frequency (ft); and a bandwidth extender (55) arranged for bandwidth extending said initial set (42) of spectral coefficients above said transition frequency (ft) ; said transition frequency (ft) being adapted to a spectral content of said audio signal.
13. Decoder according to claim 12, wherein said transition determining circuitry (60) is arranged for adaptively determining said transition frequency (ft) dependent on a distribution of spectral holes in said initial set (42) of spectral coefficients.
14. Decoder according to claim 13, wherein said transition determining circuitry (60) is further arranged for dividing said spectral coefficients of said initial set of spectral coefficients into a plurality of frequency bands (74), and for selecting said transition frequency (ft) dependent on a proportion of spectral holes in said frequency bands (74) .
15. Decoder according to claim 14 wherein said frequency bands (74) have a constant frequency width.
16. Decoder according to claim 14, wherein at least two of said frequency bands (74) have different frequency widths.
17. Decoder according to any of the claims 14 to 16, wherein said transition determining circuitry (60) is further arranged for finding a transition frequency band, being a highest frequency band in which said proportion is lower than a first threshold.
18. Decoder according to claim 17, wherein said transition determining circuitry (60) is further arranged for setting said transition frequency (ft) dependent on an upper frequency limit of said transition frequency band.
19. Encoder (20) for spectral coding of an audio signal, comprising: transition determining circuitry (60) arranged for determining a transition frequency (ft) for an initial set (24) of spectral coefficients representing said audio signal; said transition frequency (ft) defining a border between a frequency range, intended to be a subject for noise filling of spectral holes, and a frequency range, intended to be a subject for bandwidth extension; said transition frequency (ft) being adapted to a spectral content of said audio signal.
PCT/SE2008/050969 2007-08-27 2008-08-26 Adaptive transition frequency between noise fill and bandwidth extension WO2009029037A1 (en)

Priority Applications (13)

Application Number Priority Date Filing Date Title
BRPI0815972A BRPI0815972B1 (en) 2007-08-27 2008-08-26 method for spectrum recovery in spectral decoding of an audio signal, method for use in spectral encoding of an audio signal, decoder, and encoder
MX2010001394A MX2010001394A (en) 2007-08-27 2008-08-26 Adaptive transition frequency between noise fill and bandwidth extension.
JP2010522869A JP5183741B2 (en) 2007-08-27 2008-08-26 Transition frequency adaptation between noise replenishment and band extension
PL08828148T PL2186086T3 (en) 2007-08-27 2008-08-26 Adaptive transition frequency between noise fill and bandwidth extension
ES08828148T ES2403410T3 (en) 2007-08-27 2008-08-26 Adaptive transition frequency between noise refilling and bandwidth extension
EP08828148A EP2186086B1 (en) 2007-08-27 2008-08-26 Adaptive transition frequency between noise fill and bandwidth extension
US12/674,341 US9269372B2 (en) 2007-08-27 2008-08-26 Adaptive transition frequency between noise fill and bandwidth extension
CN200880105330XA CN101939782B (en) 2007-08-27 2008-08-26 Adaptive transition frequency between noise fill and bandwidth extension
HK10109588.7A HK1143239A1 (en) 2007-08-27 2010-10-08 Adaptive transition frequency between noise fill and bandwidth extension
US14/955,645 US9711154B2 (en) 2007-08-27 2015-12-01 Adaptive transition frequency between noise fill and bandwidth extension
US15/639,347 US10199049B2 (en) 2007-08-27 2017-06-30 Adaptive transition frequency between noise fill and bandwidth extension
US16/230,777 US10878829B2 (en) 2007-08-27 2018-12-21 Adaptive transition frequency between noise fill and bandwidth extension
US17/128,665 US11990147B2 (en) 2007-08-27 2020-12-21 Adaptive transition frequency between noise fill and bandwidth extension

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US96813407P 2007-08-27 2007-08-27
US60/968,134 2007-08-27

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US12/674,341 A-371-Of-International US9269372B2 (en) 2007-08-27 2008-08-26 Adaptive transition frequency between noise fill and bandwidth extension
US14/955,645 Continuation US9711154B2 (en) 2007-08-27 2015-12-01 Adaptive transition frequency between noise fill and bandwidth extension

Publications (1)

Publication Number Publication Date
WO2009029037A1 true WO2009029037A1 (en) 2009-03-05

Family

ID=40387561

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SE2008/050969 WO2009029037A1 (en) 2007-08-27 2008-08-26 Adaptive transition frequency between noise fill and bandwidth extension

Country Status (12)

Country Link
US (5) US9269372B2 (en)
EP (2) EP2571024B1 (en)
JP (2) JP5183741B2 (en)
CN (1) CN101939782B (en)
BR (1) BRPI0815972B1 (en)
DK (1) DK2571024T3 (en)
ES (2) ES2526333T3 (en)
HK (1) HK1143239A1 (en)
MX (1) MX2010001394A (en)
PL (1) PL2186086T3 (en)
PT (1) PT2571024E (en)
WO (1) WO2009029037A1 (en)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102194457A (en) * 2010-03-02 2011-09-21 中兴通讯股份有限公司 Audio encoding and decoding method, system and noise level estimation method
WO2012017621A1 (en) 2010-08-03 2012-02-09 Sony Corporation Signal processing apparatus and method, and program
WO2012053150A1 (en) * 2010-10-18 2012-04-26 パナソニック株式会社 Audio encoding device and audio decoding device
CN102822890A (en) * 2010-03-30 2012-12-12 松下电器产业株式会社 Audio device
EP2613315A4 (en) * 2011-07-13 2013-07-10 Huawei Tech Co Ltd Method and device for coding/decoding audio signals
US9679580B2 (en) 2010-04-13 2017-06-13 Sony Corporation Signal processing apparatus and signal processing method, encoder and encoding method, decoder and decoding method, and program
US9691410B2 (en) 2009-10-07 2017-06-27 Sony Corporation Frequency band extending device and method, encoding device and method, decoding device and method, and program
US9704500B2 (en) 2013-01-29 2017-07-11 Huawei Technologies Co., Ltd. Method for predicting high frequency band signal, encoding device, and decoding device
US9767824B2 (en) 2010-10-15 2017-09-19 Sony Corporation Encoding device and method, decoding device and method, and program
US9875749B2 (en) 2013-01-29 2018-01-23 Huawei Technologies Co., Ltd. Method for predicting bandwidth extension frequency band signal, and decoding device
US9875746B2 (en) 2013-09-19 2018-01-23 Sony Corporation Encoding device and method, decoding device and method, and program
RU2719008C1 (en) * 2016-04-12 2020-04-16 Фраунхофер-Гезелльшафт Цур Фердерунг Дер Ангевандтен Форшунг Е.Ф. Audio encoder for encoding an audio signal, a method for encoding an audio signal and a computer program which take into account a detectable spectral region of peaks in the upper frequency range
US10692511B2 (en) 2013-12-27 2020-06-23 Sony Corporation Decoding apparatus and method, and program
US20210110836A1 (en) * 2007-08-27 2021-04-15 Telefonaktiebolaget Lm Ericsson (Publ) Adaptive transition frequency between noise fill and bandwidth extension
US11049508B2 (en) 2014-07-28 2021-06-29 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio encoder and decoder using a frequency domain processor with full-band gap filling and a time domain processor
US11410668B2 (en) 2014-07-28 2022-08-09 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio encoder and decoder using a frequency domain processor, a time domain processor, and a cross processing for continuous initialization

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2698031C (en) * 2007-08-27 2016-10-18 Telefonaktiebolaget Lm Ericsson (Publ) Method and device for noise filling
KR20090110244A (en) * 2008-04-17 2009-10-21 삼성전자주식회사 Method for encoding/decoding audio signals using audio semantic information and apparatus thereof
PL2304719T3 (en) 2008-07-11 2017-12-29 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio encoder, methods for providing an audio stream and computer program
JP4932917B2 (en) 2009-04-03 2012-05-16 株式会社エヌ・ティ・ティ・ドコモ Speech decoding apparatus, speech decoding method, and speech decoding program
JP5609737B2 (en) 2010-04-13 2014-10-22 ソニー株式会社 Signal processing apparatus and method, encoding apparatus and method, decoding apparatus and method, and program
CN103262409B (en) * 2010-09-10 2016-07-06 Dts（英属维尔京群岛）有限公司 The dynamic compensation of the unbalanced audio signal of frequency spectrum of the sensation for improving
WO2012037515A1 (en) 2010-09-17 2012-03-22 Xiph. Org. Methods and systems for adaptive time-frequency resolution in digital data coding
US8838442B2 (en) 2011-03-07 2014-09-16 Xiph.org Foundation Method and system for two-step spreading for tonal artifact avoidance in audio coding
WO2012122297A1 (en) * 2011-03-07 2012-09-13 Xiph. Org. Methods and systems for avoiding partial collapse in multi-block audio coding
WO2012122299A1 (en) 2011-03-07 2012-09-13 Xiph. Org. Bit allocation and partitioning in gain-shape vector quantization for audio coding
CN102800317B (en) * 2011-05-25 2014-09-17 华为技术有限公司 Signal classification method and equipment, and encoding and decoding methods and equipment
JP2013015598A (en) * 2011-06-30 2013-01-24 Zte Corp Audio coding/decoding method, system and noise level estimation method
BR122021019877B1 (en) 2011-06-30 2022-07-19 Samsung Electronics Co., Ltd DEVICE FOR GENERATING AN EXTENDED BANDWIDTH SIGNAL
JP5416173B2 (en) * 2011-07-07 2014-02-12 中興通訊股▲ふん▼有限公司 Frequency band copy method, apparatus, audio decoding method, and system
CN103368682B (en) 2012-03-29 2016-12-07 华为技术有限公司 Signal coding and the method and apparatus of decoding
EP2665208A1 (en) 2012-05-14 2013-11-20 Thomson Licensing Method and apparatus for compressing and decompressing a Higher Order Ambisonics signal representation
US9881616B2 (en) * 2012-06-06 2018-01-30 Qualcomm Incorporated Method and systems having improved speech recognition
US9633662B2 (en) * 2012-09-13 2017-04-25 Lg Electronics Inc. Frame loss recovering method, and audio decoding method and device using same
CN103778918B (en) * 2012-10-26 2016-09-07 华为技术有限公司 The method and apparatus of the bit distribution of audio signal
CN103854653B (en) 2012-12-06 2016-12-28 华为技术有限公司 The method and apparatus of signal decoding
MX343572B (en) * 2013-01-29 2016-11-09 Fraunhofer Ges Forschung Noise filling concept.
EP3528249A1 (en) 2013-04-05 2019-08-21 Dolby International AB Stereo audio encoder and decoder
CN117253498A (en) 2013-04-05 2023-12-19 杜比国际公司 Audio signal decoding method, audio signal decoder, audio signal medium, and audio signal encoding method
EP2830063A1 (en) * 2013-07-22 2015-01-28 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus, method and computer program for decoding an encoded audio signal
CA2927990C (en) * 2013-10-31 2018-08-14 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio bandwidth extension by insertion of temporal pre-shaped noise in frequency domain
EP2980792A1 (en) 2014-07-28 2016-02-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus and method for generating an enhanced signal using independent noise-filling
WO2016142002A1 (en) 2015-03-09 2016-09-15 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Audio encoder, audio decoder, method for encoding an audio signal and method for decoding an encoded audio signal
BR112018067944B1 (en) * 2016-03-07 2024-03-05 Fraunhofer - Gesellschaft Zur Förderung Der Angewandten Forschung E.V ERROR HIDDENING UNIT, ERROR HIDDENING METHOD, AUDIO DECODER, AUDIO ENCODER, METHOD FOR PROVIDING A CODED AUDIO REPRESENTATION AND SYSTEM
WO2018170626A1 (en) 2017-03-18 2018-09-27 华为技术有限公司 Connection recovery method, access and mobility management function entity, and user equipment
EP3382702A1 (en) * 2017-03-31 2018-10-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus and method for determining a predetermined characteristic related to an artificial bandwidth limitation processing of an audio signal
CN116157860A (en) * 2021-09-22 2023-05-23 京东方科技集团股份有限公司 Audio adjusting method, device, equipment and storage medium
WO2024050673A1 (en) * 2022-09-05 2024-03-14 北京小米移动软件有限公司 Audio signal frequency band extension method and apparatus, device, and storage medium

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005078706A1 (en) * 2004-02-18 2005-08-25 Voiceage Corporation Methods and devices for low-frequency emphasis during audio compression based on acelp/tcx
US20060265087A1 (en) * 2003-03-04 2006-11-23 France Telecom Sa Method and device for spectral reconstruction of an audio signal

Family Cites Families (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5583961A (en) * 1993-03-25 1996-12-10 British Telecommunications Public Limited Company Speaker recognition using spectral coefficients normalized with respect to unequal frequency bands
US5664057A (en) * 1993-07-07 1997-09-02 Picturetel Corporation Fixed bit rate speech encoder/decoder
SE9903553D0 (en) * 1999-01-27 1999-10-01 Lars Liljeryd Enhancing conceptual performance of SBR and related coding methods by adaptive noise addition (ANA) and noise substitution limiting (NSL)
US6226616B1 (en) * 1999-06-21 2001-05-01 Digital Theater Systems, Inc. Sound quality of established low bit-rate audio coding systems without loss of decoder compatibility
US7742927B2 (en) * 2000-04-18 2010-06-22 France Telecom Spectral enhancing method and device
SE0004163D0 (en) * 2000-11-14 2000-11-14 Coding Technologies Sweden Ab Enhancing perceptual performance or high frequency reconstruction coding methods by adaptive filtering
SE0004187D0 (en) * 2000-11-15 2000-11-15 Coding Technologies Sweden Ab Enhancing the performance of coding systems that use high frequency reconstruction methods
SE522553C2 (en) * 2001-04-23 2004-02-17 Ericsson Telefon Ab L M Bandwidth extension of acoustic signals
EP1395980B1 (en) * 2001-05-08 2006-03-15 Koninklijke Philips Electronics N.V. Audio coding
US6493668B1 (en) * 2001-06-15 2002-12-10 Yigal Brandman Speech feature extraction system
EP1351401B1 (en) * 2001-07-13 2009-01-14 Panasonic Corporation Audio signal decoding device and audio signal encoding device
US6988066B2 (en) * 2001-10-04 2006-01-17 At&T Corp. Method of bandwidth extension for narrow-band speech
US6895375B2 (en) * 2001-10-04 2005-05-17 At&T Corp. System for bandwidth extension of Narrow-band speech
DE60214027T2 (en) * 2001-11-14 2007-02-15 Matsushita Electric Industrial Co., Ltd., Kadoma CODING DEVICE AND DECODING DEVICE
ATE288617T1 (en) * 2001-11-29 2005-02-15 Coding Tech Ab RESTORATION OF HIGH FREQUENCY COMPONENTS
US20030187663A1 (en) * 2002-03-28 2003-10-02 Truman Michael Mead Broadband frequency translation for high frequency regeneration
GB2388502A (en) * 2002-05-10 2003-11-12 Chris Dunn Compression of frequency domain audio signals
US7447631B2 (en) * 2002-06-17 2008-11-04 Dolby Laboratories Licensing Corporation Audio coding system using spectral hole filling
US7330812B2 (en) * 2002-10-04 2008-02-12 National Research Council Of Canada Method and apparatus for transmitting an audio stream having additional payload in a hidden sub-channel
JP2004134900A (en) * 2002-10-09 2004-04-30 Matsushita Electric Ind Co Ltd Decoding apparatus and method for coded signal
KR101058062B1 (en) * 2003-06-30 2011-08-19 코닌클리케 필립스 일렉트로닉스 엔.브이. Improving Decoded Audio Quality by Adding Noise
JP2006087018A (en) * 2004-09-17 2006-03-30 Matsushita Electric Ind Co Ltd Sound processing unit
WO2006033058A1 (en) * 2004-09-23 2006-03-30 Koninklijke Philips Electronics N.V. A system and a method of processing audio data, a program element and a computer-readable medium
KR100707186B1 (en) * 2005-03-24 2007-04-13 삼성전자주식회사 Audio coding and decoding apparatus and method, and recoding medium thereof
US7885809B2 (en) * 2005-04-20 2011-02-08 Ntt Docomo, Inc. Quantization of speech and audio coding parameters using partial information on atypical subsequences
KR101171098B1 (en) * 2005-07-22 2012-08-20 삼성전자주식회사 Scalable speech coding/decoding methods and apparatus using mixed structure
US8332216B2 (en) * 2006-01-12 2012-12-11 Stmicroelectronics Asia Pacific Pte., Ltd. System and method for low power stereo perceptual audio coding using adaptive masking threshold
JP2009534713A (en) * 2006-04-24 2009-09-24 ネロ アーゲー Apparatus and method for encoding digital audio data having a reduced bit rate
KR20070115637A (en) * 2006-06-03 2007-12-06 삼성전자주식회사 Method and apparatus for bandwidth extension encoding and decoding
US20080109215A1 (en) * 2006-06-26 2008-05-08 Chi-Min Liu High frequency reconstruction by linear extrapolation
US8135047B2 (en) * 2006-07-31 2012-03-13 Qualcomm Incorporated Systems and methods for including an identifier with a packet associated with a speech signal
US20080208575A1 (en) * 2007-02-27 2008-08-28 Nokia Corporation Split-band encoding and decoding of an audio signal
US7761290B2 (en) * 2007-06-15 2010-07-20 Microsoft Corporation Flexible frequency and time partitioning in perceptual transform coding of audio
US7885819B2 (en) * 2007-06-29 2011-02-08 Microsoft Corporation Bitstream syntax for multi-process audio decoding
CA2697920C (en) * 2007-08-27 2018-01-02 Telefonaktiebolaget L M Ericsson (Publ) Transient detector and method for supporting encoding of an audio signal
CN103594090B (en) * 2007-08-27 2017-10-10 爱立信电话股份有限公司 Low complexity spectrum analysis/synthesis that use time resolution ratio can be selected
CN101939782B (en) * 2007-08-27 2012-12-05 爱立信电话股份有限公司 Adaptive transition frequency between noise fill and bandwidth extension
CA2698031C (en) * 2007-08-27 2016-10-18 Telefonaktiebolaget Lm Ericsson (Publ) Method and device for noise filling
US9117458B2 (en) * 2009-11-12 2015-08-25 Lg Electronics Inc. Apparatus for processing an audio signal and method thereof

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060265087A1 (en) * 2003-03-04 2006-11-23 France Telecom Sa Method and device for spectral reconstruction of an audio signal
WO2005078706A1 (en) * 2004-02-18 2005-08-25 Voiceage Corporation Methods and devices for low-frequency emphasis during audio compression based on acelp/tcx

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2186086A4 *

Cited By (63)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11990147B2 (en) * 2007-08-27 2024-05-21 Telefonaktiebolaget Lm Ericsson (Publ) Adaptive transition frequency between noise fill and bandwidth extension
US20210110836A1 (en) * 2007-08-27 2021-04-15 Telefonaktiebolaget Lm Ericsson (Publ) Adaptive transition frequency between noise fill and bandwidth extension
US9691410B2 (en) 2009-10-07 2017-06-27 Sony Corporation Frequency band extending device and method, encoding device and method, decoding device and method, and program
CN102194457A (en) * 2010-03-02 2011-09-21 中兴通讯股份有限公司 Audio encoding and decoding method, system and noise level estimation method
EP2555192A4 (en) * 2010-03-30 2013-09-25 Panasonic Corp Audio device
CN102822890A (en) * 2010-03-30 2012-12-12 松下电器产业株式会社 Audio device
EP2555192A1 (en) * 2010-03-30 2013-02-06 Panasonic Corporation Audio device
US9047876B2 (en) 2010-03-30 2015-06-02 Panasonic Intellectual Property Managment Co., Ltd. Audio device
US10297270B2 (en) 2010-04-13 2019-05-21 Sony Corporation Signal processing apparatus and signal processing method, encoder and encoding method, decoder and decoding method, and program
US10224054B2 (en) 2010-04-13 2019-03-05 Sony Corporation Signal processing apparatus and signal processing method, encoder and encoding method, decoder and decoding method, and program
US10381018B2 (en) 2010-04-13 2019-08-13 Sony Corporation Signal processing apparatus and signal processing method, encoder and encoding method, decoder and decoding method, and program
US10546594B2 (en) 2010-04-13 2020-01-28 Sony Corporation Signal processing apparatus and signal processing method, encoder and encoding method, decoder and decoding method, and program
US9679580B2 (en) 2010-04-13 2017-06-13 Sony Corporation Signal processing apparatus and signal processing method, encoder and encoding method, decoder and decoding method, and program
RU2666291C2 (en) * 2010-08-03 2018-09-06 Сони Корпорейшн Signal processing apparatus and method, and program
KR101967122B1 (en) * 2010-08-03 2019-04-08 소니 주식회사 Signal processing apparatus and method, and program
EP2471063A1 (en) * 2010-08-03 2012-07-04 Sony Corporation Signal processing apparatus and method, and program
EP3584793A1 (en) * 2010-08-03 2019-12-25 SONY Corporation Signal processing apparatus and method, and program
US9406306B2 (en) 2010-08-03 2016-08-02 Sony Corporation Signal processing apparatus and method, and program
KR102057015B1 (en) * 2010-08-03 2019-12-17 소니 주식회사 Signal processing apparatus and method, and program
CN104200808A (en) * 2010-08-03 2014-12-10 索尼公司 Signal processing apparatus and method
RU2550549C2 (en) * 2010-08-03 2015-05-10 Сони Корпорейшн Signal processing device and method and programme
WO2012017621A1 (en) 2010-08-03 2012-02-09 Sony Corporation Signal processing apparatus and method, and program
US9767814B2 (en) 2010-08-03 2017-09-19 Sony Corporation Signal processing apparatus and method, and program
CN102549658A (en) * 2010-08-03 2012-07-04 索尼公司 Signal processing apparatus and method, and program
EP4086901A1 (en) * 2010-08-03 2022-11-09 Sony Group Corporation Signal processing apparatus and method, and program
US10229690B2 (en) 2010-08-03 2019-03-12 Sony Corporation Signal processing apparatus and method, and program
KR101835156B1 (en) * 2010-08-03 2018-03-06 소니 주식회사 Signal processing apparatus and method, and program
KR20180026558A (en) * 2010-08-03 2018-03-12 소니 주식회사 Signal processing apparatus and method, and program
US11011179B2 (en) 2010-08-03 2021-05-18 Sony Corporation Signal processing apparatus and method, and program
EP3340244A1 (en) * 2010-08-03 2018-06-27 Sony Corporation Signal processing apparatus and method, and program
EP2471063A4 (en) * 2010-08-03 2014-01-22 Sony Corp Signal processing apparatus and method, and program
US10236015B2 (en) 2010-10-15 2019-03-19 Sony Corporation Encoding device and method, decoding device and method, and program
US9767824B2 (en) 2010-10-15 2017-09-19 Sony Corporation Encoding device and method, decoding device and method, and program
WO2012053150A1 (en) * 2010-10-18 2012-04-26 パナソニック株式会社 Audio encoding device and audio decoding device
JPWO2012053150A1 (en) * 2010-10-18 2014-02-24 パナソニック株式会社 Speech coding apparatus and speech decoding apparatus
JP5695074B2 (en) * 2010-10-18 2015-04-01 パナソニック インテレクチュアル プロパティ コーポレーション オブアメリカＰａｎａｓｏｎｉｃ Ｉｎｔｅｌｌｅｃｔｕａｌ Ｐｒｏｐｅｒｔｙ Ｃｏｒｐｏｒａｔｉｏｎ ｏｆ Ａｍｅｒｉｃａ Speech coding apparatus and speech decoding apparatus
US9984697B2 (en) 2011-07-13 2018-05-29 Huawei Technologies Co., Ltd. Audio signal coding and decoding method and device
EP2613315A1 (en) * 2011-07-13 2013-07-10 Huawei Technologies Co., Ltd. Method and device for coding/decoding audio signals
US11127409B2 (en) 2011-07-13 2021-09-21 Huawei Technologies Co., Ltd. Audio signal coding and decoding method and device
EP2613315A4 (en) * 2011-07-13 2013-07-10 Huawei Tech Co Ltd Method and device for coding/decoding audio signals
EP3174049A1 (en) * 2011-07-13 2017-05-31 Huawei Technologies Co., Ltd. Audio signal coding method and device
US9105263B2 (en) 2011-07-13 2015-08-11 Huawei Technologies Co., Ltd. Audio signal coding and decoding method and device
US10546592B2 (en) 2011-07-13 2020-01-28 Huawei Technologies Co., Ltd. Audio signal coding and decoding method and device
US10388295B2 (en) 2013-01-29 2019-08-20 Huawei Technologies Co., Ltd. Method for predicting bandwidth extension frequency band signal, and decoding device
US9875749B2 (en) 2013-01-29 2018-01-23 Huawei Technologies Co., Ltd. Method for predicting bandwidth extension frequency band signal, and decoding device
US10636432B2 (en) 2013-01-29 2020-04-28 Huawei Technologies Co., Ltd. Method for predicting high frequency band signal, encoding device, and decoding device
US9704500B2 (en) 2013-01-29 2017-07-11 Huawei Technologies Co., Ltd. Method for predicting high frequency band signal, encoding device, and decoding device
US10607621B2 (en) 2013-01-29 2020-03-31 Huawei Technologies Co., Ltd. Method for predicting bandwidth extension frequency band signal, and decoding device
US10089997B2 (en) 2013-01-29 2018-10-02 Huawei Technologies Co.,Ltd. Method for predicting high frequency band signal, encoding device, and decoding device
US9875746B2 (en) 2013-09-19 2018-01-23 Sony Corporation Encoding device and method, decoding device and method, and program
US10692511B2 (en) 2013-12-27 2020-06-23 Sony Corporation Decoding apparatus and method, and program
US11705140B2 (en) 2013-12-27 2023-07-18 Sony Corporation Decoding apparatus and method, and program
US11410668B2 (en) 2014-07-28 2022-08-09 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio encoder and decoder using a frequency domain processor, a time domain processor, and a cross processing for continuous initialization
US11049508B2 (en) 2014-07-28 2021-06-29 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio encoder and decoder using a frequency domain processor with full-band gap filling and a time domain processor
EP4239634A1 (en) * 2014-07-28 2023-09-06 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio coding using a frequency domain processor and a time domain processor
EP3511936B1 (en) * 2014-07-28 2023-09-06 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio coding using a frequency domain processor and a time domain processor
US11915712B2 (en) 2014-07-28 2024-02-27 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio encoder and decoder using a frequency domain processor, a time domain processor, and a cross processing for continuous initialization
US11929084B2 (en) 2014-07-28 2024-03-12 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio encoder and decoder using a frequency domain processor with full-band gap filling and a time domain processor
US12080310B2 (en) 2014-07-28 2024-09-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio encoder and decoder using a frequency domain processor with full-band gap filling and a time domain processor
RU2719008C1 (en) * 2016-04-12 2020-04-16 Фраунхофер-Гезелльшафт Цур Фердерунг Дер Ангевандтен Форшунг Е.Ф. Audio encoder for encoding an audio signal, a method for encoding an audio signal and a computer program which take into account a detectable spectral region of peaks in the upper frequency range
US11682409B2 (en) 2016-04-12 2023-06-20 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio encoder for encoding an audio signal, method for encoding an audio signal and computer program under consideration of a detected peak spectral region in an upper frequency band
US10825461B2 (en) 2016-04-12 2020-11-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio encoder for encoding an audio signal, method for encoding an audio signal and computer program under consideration of a detected peak spectral region in an upper frequency band
US12014747B2 (en) 2016-04-12 2024-06-18 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio encoder for encoding an audio signal, method for encoding an audio signal and computer program under consideration of a detected peak spectral region in an upper frequency band

Also Published As

Publication number Publication date
BRPI0815972B1 (en) 2020-02-04
MX2010001394A (en) 2010-03-10
PL2186086T3 (en) 2013-07-31
JP2013117730A (en) 2013-06-13
EP2571024B1 (en) 2014-10-22
EP2186086B1 (en) 2013-01-23
BRPI0815972A2 (en) 2015-09-29
CN101939782A (en) 2011-01-05
EP2186086A1 (en) 2010-05-19
US20170301358A1 (en) 2017-10-19
US9711154B2 (en) 2017-07-18
US9269372B2 (en) 2016-02-23
US11990147B2 (en) 2024-05-21
JP5458189B2 (en) 2014-04-02
US20190122680A1 (en) 2019-04-25
EP2186086A4 (en) 2012-01-25
US10878829B2 (en) 2020-12-29
JP5183741B2 (en) 2013-04-17
PT2571024E (en) 2014-12-23
DK2571024T3 (en) 2015-01-05
HK1143239A1 (en) 2010-12-24
ES2403410T3 (en) 2013-05-17
US20160086614A1 (en) 2016-03-24
EP2571024A1 (en) 2013-03-20
JP2010538318A (en) 2010-12-09
US20110264454A1 (en) 2011-10-27
US10199049B2 (en) 2019-02-05
CN101939782B (en) 2012-12-05
US20210110836A1 (en) 2021-04-15
ES2526333T3 (en) 2015-01-09
BRPI0815972A8 (en) 2017-11-14

Similar Documents

Publication Publication Date Title
US11990147B2 (en) Adaptive transition frequency between noise fill and bandwidth extension
US8370133B2 (en) Method and device for noise filling
EP2272063B1 (en) Method and apparatus for selective signal coding based on core encoder performance
US10311884B2 (en) Advanced quantizer
CN101836252A (en) Be used for generating the method and apparatus of enhancement layer in the Audiocode system
US20130197919A1 (en) "method and device for determining a number of bits for encoding an audio signal"

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200880105330.X

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08828148

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
WWE Wipo information: entry into national phase

Ref document number: 2008828148

Country of ref document: EP

Ref document number: MX/A/2010/001394

Country of ref document: MX

ENP Entry into the national phase

Ref document number: 2010522869

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 1759/DELNP/2010

Country of ref document: IN

WWE Wipo information: entry into national phase

Ref document number: 12674341

Country of ref document: US

ENP Entry into the national phase

Ref document number: PI0815972

Country of ref document: BR

Kind code of ref document: A2

Effective date: 20100225