EP2881943A1 - Apparatus and method for decoding an encoded audio signal with low computational resources - Google Patents

Apparatus and method for decoding an encoded audio signal with low computational resources Download PDF

Info

Publication number
EP2881943A1
EP2881943A1 EP13196305.0A EP13196305A EP2881943A1 EP 2881943 A1 EP2881943 A1 EP 2881943A1 EP 13196305 A EP13196305 A EP 13196305A EP 2881943 A1 EP2881943 A1 EP 2881943A1
Authority
EP
European Patent Office
Prior art keywords
bandwidth extension
harmonic
audio signal
extension mode
encoded audio
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP13196305.0A
Other languages
German (de)
English (en)
French (fr)
Inventor
Andreas NIEDERMEIER
Stephan Wilde
Daniel Fischer
Matthias Hildenbrand
Marc Gayer
Max Neuendorf
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Original Assignee
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Priority to EP13196305.0A priority Critical patent/EP2881943A1/en
Priority to KR1020167015028A priority patent/KR101854298B1/ko
Priority to MX2016007430A priority patent/MX353703B/es
Priority to CA2931958A priority patent/CA2931958C/en
Priority to JP2016536886A priority patent/JP6286554B2/ja
Priority to ES14808907.1T priority patent/ES2650941T3/es
Priority to PCT/EP2014/076000 priority patent/WO2015086351A1/en
Priority to RU2016127582A priority patent/RU2644135C2/ru
Priority to CN201480066827.0A priority patent/CN105981101B/zh
Priority to EP14808907.1A priority patent/EP3080803B1/en
Priority to BR112016012689-0A priority patent/BR112016012689B1/pt
Publication of EP2881943A1 publication Critical patent/EP2881943A1/en
Priority to US15/177,265 priority patent/US9799345B2/en
Priority to US15/621,938 priority patent/US10332536B2/en
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/167—Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/18—Vocoders using multiple modes
    • G10L19/24—Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/022—Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/18—Vocoders using multiple modes
    • G10L19/20—Vocoders using multiple modes using sound class specific coding, hybrid encoders or object based coding
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/18—Vocoders using multiple modes
    • G10L19/22—Mode decision, i.e. based on audio signal content versus external parameters
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/038—Speech enhancement, e.g. noise reduction or echo cancellation using band spreading techniques

Definitions

  • the present invention is related to audio processing and in particular to a concept for decoding an encoded audio signal using reduced computational resources.
  • the 'Unified speech and audio coding" (USAC) standard [1] standardizes a harmonic bandwidth extension tool, HBE, employing a harmonic transposer, and which is an extension of the spectral band replication (SBR) system, standardized in [1] and [2], respectively.
  • HBE harmonic bandwidth extension tool
  • SBR spectral band replication
  • SBR synthesizes high frequency content of bandwidth limited audio signals by using the given low frequency part together with given side information.
  • the SBR tool is described in [2]
  • enhanced SBR, eSBR is described in [1].
  • the harmonic bandwidth extension HBE which employs phase vocoders is part of eSBR and has been developed to avoid the auditory roughness which is often observed in signals subjected to copy-up patching, as it is carried out in the regular SBR processing.
  • the main scope of HBE is to preserve harmonic structures in the synthesized high frequency region of the given audio signal while applying eSBR.
  • a decoder which is conform to [1] shall provide decoding and applying HBE related data.
  • the HBE tool replaces the simple copy-up patching of the legacy SBR system by advanced signal processing routines. These require a considerable amount of processing power and memory for filter states and delay lines. On the contrary the complexity of the copy-up patching is negligible.
  • USAC-bitstreams are decoded as described in [1]. This implies necessarily the implementation of a HBE decoder tool, as described in [1], 7.5.3.
  • the tool can be signaled in all codec operating points which contain eSBR processing.
  • decoder devices which fulfill profile and conformance criteria of [1] this means that the overall worst case of computational workload and memory consumption increases significantly.
  • the actual increase in computational complexity is implementation and platform dependent.
  • the increase in memory consumption per audio channel is, in the current memory optimized implementation, at least 15 kwords for the actual HBE processing.
  • the present invention is based on the finding that an audio decoding concept requiring reduced memory resources is achieved when an audio signal consisting of portions to be decoded using an harmonic bandwidth extension mode and additionally containing portions to be decoded using a non-harmonic bandwidth extension mode is decoded, throughout the whole signal, with the non-harmonic bandwidth extension mode only.
  • a signal comprises portions or frames which are signaled to be decoded using a harmonic bandwidth extension mode, these portions or frames are nevertheless decoded using the non-harmonic bandwidth extension mode.
  • a processor for decoding the audio signal using the non-harmonic bandwidth extension mode is provided and additionally a controller is implemented within the apparatus or a controlling step is implemented within a method for decoding for controlling the processor to decode the audio signal using the second non-harmonic bandwidth extension mode even when the bandwidth extension control data included in the encoded audio signal indicates the first - i.e. harmonic - bandwidth extension mode for the audio signal.
  • the processor only has to be implemented with corresponding hardware resources such as memory and processing power to only cope with the computationally very efficient non-harmonic bandwidth extension mode.
  • the audio decoder is nevertheless in the position to accept and decode an encoded audio signal requiring a harmonic bandwidth extension mode with an acceptable quality.
  • the controller is configured for controlling the processor to decode the whole audio signal with the non-harmonic bandwidth extension mode, even though the encoded audio signal itself requires, due to the included bandwidth extension control data, that at least several portions of this signal are decoded using the harmonic bandwidth extension mode.
  • the present invention is advantageous due to the fact that it lowers the computational complexity and memory demand of particularly a USAC decoder.
  • the predetermined or standardized non-harmonic bandwidth extension mode is modified using harmonic bandwidth extension mode data transmitted in the bitstream in order to reuse bandwidth extension mode data which are basically not necessary for the non-harmonic bandwidth extension mode as far as possible in order to even improve the audio quality of the non-harmonic bandwidth extension mode.
  • an alternative decoding scheme is provided in this preferred embodiment, in order to mitigate the impairment of perceptual quality caused by omitting the harmonic bandwidth extension mode which is typically based on phase-vocoder processing as discussed in the USAC standard [1].
  • the processor has memory and processing resources being sufficient for decoding the encoded audio signal using the second non-harmonic bandwidth extension mode, wherein the memory or processing resources are not sufficient for decoding the encoded audio signal using the first harmonic bandwidth extension mode, when the encoded audio signal is an encoded stereo or multichannel audio signal.
  • the processor has memory and processing resources being sufficient for decoding the encoded audio signal using the second non-harmonic bandwidth extension mode and using the first harmonic bandwidth extension mode, when the encoded audio signal is an encoded mono signal, since the resources for mono decoding are reduced compared to the resources for stereo or multichannel decoding.
  • the available resources depend on the bit-stream configuration, i.e. combination of tools, sampling rate etc. For example it may be possible that resources are sufficient to decode a mono bit-stream using harmonic BWE but the processor lacks resources to decode a stereo bit-stream using harmonic BWE.
  • Fig. 1 a illustrates an embodiment of an apparatus for decoding an encoded audio signal.
  • the encoded audio signal comprises bandwidth extension control data indicating either a first harmonic bandwidth extension mode or a second non-harmonic bandwidth extension mode.
  • the encoded audio signal is input on a line 101 into an input interface 100.
  • the input interface is connected via line 108 to a limited resources processor 102.
  • a controller 104 is provided which is at least optionally connected to the input interface 100 via line 106 and which is additionally connected to the processor 102 via line 110.
  • the output of the processor 102 is a decoded audio signal as indicated at 112.
  • the input interface 100 is configured for receiving the encoded audio signal comprising the bandwidth extension control data indicating either a first harmonic bandwidth extension mode or a second non-harmonic bandwidth extension mode for an encoded portion such as a frame of the encoded audio signal.
  • the processor 102 is configured for decoding the audio signal using the second non-harmonic bandwidth extension mode only as indicated close to line 110 in Fig. 1a . This is made sure by the controller 104.
  • the controller 104 is configured for controlling the processor 102 to decode the audio signal using the second non-harmonic bandwidth extension mode, even when the bandwidth extension control data indicate the first harmonic bandwidth extension mode for the encoded audio signal.
  • Fig. 1b illustrates a preferred implementation of the encoded audio signal within a data stream or a bitstream.
  • the encoded audio signal comprises a header 114 for the whole audio item, and the whole audio item is organized into serial frames such as frame 1 116, frame 2 118 and frame 3 120. Each frame additionally has an associated header, such as header 1 116a for frame 1 and payload data 116b for frame 1. Furthermore, the second frame 118 again has header data 118a and payload data 118b. Analogously, the third frame 120 again has a header 120a and a payload data block 120b. In the USAC standard, the header 114 has a flag "harmonicSBR".
  • this flag harmonicSBR is zero, then the whole audio item is decoded using a non-harmonic bandwidth extension mode as defined in the USAC standard, which in this context refers back to the High Efficiency-AAC standard (HE-AAC), which is ISO/IEC 1449-3:2009, audio part.
  • HE-AAC High Efficiency-AAC standard
  • the harmonicSBR flag has a value of one, then the harmonic bandwidth extension mode is enabled, but can then be signaled, for each frame, by an individual flag sbrPatchingMode which can be zero or one.
  • Fig. 1c indicating the different values of the two flags.
  • the USAC standard decoder performs a harmonic bandwidth extension mode.
  • the controller 104 of Fig. 1 a is operative to nevertheless control the processor 102 to perform a non-harmonic bandwidth extension mode.
  • Fig. 2 illustrates a preferred implementation of the inventive procedure.
  • the input interface 100 or any other entity within the apparatus for decoding reads the bandwidth extension control data from the encoded audio signal, and this bandwidth extension control data can be one indication per frame or, if provided, an additional indication per item as discussed in the context of Fig. 1 b with respect to the USAC standard.
  • the processor 102 receives the bandwidth extension control data and stores the bandwidth extension control data in a specific control register implemented within the processor 102 of Fig. 1 a.
  • the controller 104 accesses this processor control register and, as indicated at 206, overwrites the control register with a value indicating the non-harmonic bandwidth extension.
  • the additional line in the high level syntax indicated at 600, 700, 702, 704 indicates that irrespective of the value sbrPatchingMode as read from the bitstream in 602, the sbrPatchingMode flag is nevertheless set to one, i.e. signaling, to the further process in the decoder, that a non-harmonic bandwidth extension mode is to be performed.
  • the syntax line 600 is placed subsequent to the decoder-side reading in of the specific harmonic bandwidth extension data consisting of sbrOversampllingFlag, sbrPitchInBinsFlag and sbrPitchInBins indicated at 604.
  • the encoded audio signal comprises common bandwidth extension payload data 606 for both bandwidth extension modes, i.e. the non-harmonic bandwidth extension mode and the harmonic bandwidth extension mode, and additionally data specific for the harmonic bandwidth extension mode illustrated at 604. This will be discussed later in the context of Fig. 3a .
  • the variable “lpHBE” illustrates the inventive procedure, i.e. the "low power harmonic bandwidth extension” mode which is a non-harmonic bandwidth extension mode, but with an additional modification which will be discussed later with respect to "the harmonic bandwidth extension”.
  • the processor 102 is a limited resources processor.
  • the limited resources processor 102 has processing resources and memory resources being sufficient for decoding the audio signal using the second non-harmonic bandwidth extension mode.
  • the memory or the processing resources are not sufficient for decoding the encoded audio signal using the first harmonic bandwidth extension mode.
  • a frame comprises a header 300, a common bandwidth extension payload data 302, additional harmonic bandwidth extension data 304 such as information on a pitch, a harmonic grid or so, and additionally, encoded core data 306.
  • the order of the data items can, however, be different from Fig. 3a .
  • the encoded core data are first. Then, the header 300 having the sbrPatchingMode flag/bit comes followed by the additional HBE data 304 and finally the common BW extension data 302.
  • the additional harmonic bandwidth extension data is, in the USAC example, as discussed in the context of Fig. 6 , item 604, the sbrPitchInBins information consisting of 7 bits.
  • the data sbrPitchInBins controls the addition of cross-product terms in the SBR harmonic transposer.
  • sbrPitchInBins is an integer value in the range between 0 and 127 and represents the distance measured in frequency bins for a 1536-DFT acting on the sampling frequency of the core coder.
  • the pitch or harmonic grid can be determined. This is illustrated in the formula (1) in Fig. 8b .
  • the values of sbrPitchInBins and sbrRatio are calculated where the SBR ratio can be as indicated in Fig. 8b above.
  • the pitch or the fundamental tone defining the harmonic grid can be included in the bitstream.
  • This data is used for controlling the first harmonic bandwidth extension mode and can, in one embodiment of the present invention, be discarded so that the non-harmonic bandwidth extension mode without any modifications is performed.
  • the straightforward non-harmonic bandwidth extension mode is modified using the control data for the harmonic bandwidth extension mode as illustrated in Fig. 3b and other figures.
  • the encoded audio signal comprises the common bandwidth extension payload data 302 for the first harmonic bandwidth extension and the second non-harmonic bandwidth extension mode and additional payload data 304 for the first harmonic bandwidth extension mode.
  • the processor 102 comprises a patching buffer as illustrated in Fig. 3b , and the specific implementation of the buffer is exemplarily explained with respect to Fig. 8d .
  • the additional payload data 304 for the first harmonic bandwidth extension mode comprises information on a harmonic characteristic of the encoded audio signal, and this harmonic characteristic can be sbrPitchInBins data, other harmonic grid data, fundamental tone data or any other data, from which a harmonic grid or a fundamental tone or a pitch of the corresponding portion of the encoded audio signal can be derived.
  • the controller 104 is configured for modifying a patching buffer content of a patching buffer used by the processor 102 to perform a patching operation in decoding the encoded audio signal so that a harmonic characteristic of a patch signal is closer to the harmonic characteristic than a signal patched without modifying the patching buffer.
  • item 902 indicates a decoded core spectrum before patching.
  • the crossover frequency x0 is indicated at 16 and a patch source is indicated to extend from frequency line 4 to frequency line 10.
  • the patch source start and/or stop frequency is preferably signaled within the encoded audio signal typically as data within the common bandwidth extension payload data 302 of Fig. 3a .
  • Item 904 indicates the same situation as in item 902, but with an additionally calculated harmonic grid k ⁇ f 0 at 906.
  • a patch destination 908 is indicated.
  • This patch destination is preferably additionally included in the common bandwidth extension payload data 302 of Fig. 3a .
  • the patch source indicates the lower frequency of the source range as indicated at 903 and the patch destination indicates the lower border of the patch destination. If the typically non-harmonic patching would be applied as indicated 910, then it would be seen that there would be a mismatch between the tonal lines or harmonic lines of the patched data and the calculated harmonic grid 906.
  • the legacy SBR patching or the straightforward USAC or High Efficiency AAC non-harmonic patching mode inserts a patch with a false harmonic grid. In order to address this issue, the modification of this straightforward non-harmonic patch is performed by the processor.
  • One way to modify is to rotate the content of the patching buffer or, stated differently, to move the harmonic lines within the patching band, but without changing the distance in frequency of the harmonic lines.
  • Other ways to match the harmonic grid of the patch to the calculated harmonic grid of the decoded spectrum before patching are clear for those skilled in the art.
  • the additional harmonic bandwidth extension data included in the encoded audio signal together with the common bandwidth extension payload data are not simply discarded, but are reused to even improve the audio quality by modifying the non-harmonic bandwidth extension mode typically signaled within the bitstream.
  • the modified non-harmonic bandwidth extension mode is still a non-harmonic bandwidth extension mode relying on a copy-up operation of a set of adjacent frequency bins into a set of adjacent frequency bins, this procedure does not result in an additional amount of memory resources compared to performing the straightforward non-harmonic bandwidth extension mode but significantly enhances audio quality of the reconstructed signal due to the matching harmonic grids as indicating in Fig. 9 at 912.
  • Fig. 3c illustrates a preferred implementation performed by the controller 104 of Fig. 3b .
  • the controller 104 calculates a harmonic grid from the additional harmonic bandwidth extension data and to this end, any calculation can be performed, but in the context of USAC the formula (1) in Fig. 8b is performed.
  • a patching source band and a patching target band are determined, i.e. this may comprise basically reading the patch source data 903 and the patch destination data 908 from the common bandwidth extension data. In other embodiments, however, this data can be predefined and therefore can already be known to the decoder and does not necessarily have to be transmitted.
  • the patching source band is modified within the frequency borders, i.e. the patch borders of the patch source are not changed compared to the transmitted data. This can be done either before patching, i.e. when the patch data is with respect to the core or decoded spectrum before patching indicated at 902 or when the patch content has already been transposed into the higher frequency range, i.e. as illustrated in Fig. 9 at 910 and 912, where the rotation is performed subsequent to patching, where patching is symbolized by arrow 914.
  • This patching 914 or "copy-up" is a non-harmonic patching which can be seen in Fig. 9 by comparing the broadness of the patch source comprising six frequency increments, and the same six frequency increments in the target range, i.e. at 910 or 912.
  • the modification is performed in such a way that a frequency portion in the patching source band coinciding with the harmonic grid is located, after patching, in a target frequency portion coinciding with the harmonic grid.
  • the patching buffer indicated at three different states 828, 830, 832 is provided within the processor 102.
  • the processor is configured to load the patching buffer as indicated at 400 in Fig. 4 .
  • the controller is configured to calculate 402 a buffer shift value using the additional bandwidth extension data and the common bandwidth extension data.
  • the buffer content is shifted by the calculated buffer shift value.
  • Item 830 indicates when the shift value has been calculated to be "-2”
  • item 832 indicates a buffer state in which a shift value of 2 has been calculated in step 404 and a shift by +2 has been performed in step 404.
  • a patching is performed using the shifted patching buffer content and the patch is nevertheless performed in a non-harmonic way.
  • the patch result is modified using common bandwidth extension data.
  • common bandwidth extension data can be, as known from High Efficiency AAC or from USAC, spectral envelope data, noise data, data on specific harmonic lines, inverse filtering data, etc.
  • the processor typically comprises a core decoder 500, a patcher 502 with the patching buffer, a patch modifier 504 and a combiner 506.
  • the core decoder is configured to decode the encoded audio signal to obtain a decoded spectrum before patching as illustrated in 902 in Fig. 9 .
  • the patcher with the patching buffer 502 performs the operation 914 in Fig. 9 .
  • the patcher 502 performs the modification of the patching buffer either before or after patching as discussed in the context of Fig. 9 .
  • the patch modifier 504 finally uses additional bandwidth extension data to modify the patch result as outlined at 408 in Fig.
  • the combiner 506 which can be, for example, a frequency domain combiner in the form of a synthesis filterbank, combines the output of the patch modifier 504 and the output of the core decoder 500, i.e. the low band signal, in order to finally obtain the bandwidth extended audio signal as output at line 112 in Fig. 1 a.
  • the bandwidth extension control data may comprise a first control data entity for an audio item, such as harmonicSBR illustrated in Fig. 1 b , where this audio item comprises a plurality of audio frames 116, 118, 120.
  • the first control data entity indicates whether the first harmonic bandwidth extension mode is active or not for the plurality of frames.
  • a second control data entity is provided corresponded to SBR patching mode exemplarily in the USAC standard which is provided in each of the headers 116a, 118a, 120a for the individual frames.
  • the input interface 100 of Fig. 1 a is configured to read the first control data for the audio item and the second control data entity for each frame of the plurality of frames, and the controller 104 of Fig. 1 a is configured for controlling the processor 102 to decode the audio signal using the second non-harmonic bandwidth extension mode irrespective of a value of the first control data entity and irrespective of a value of the second control data entity.
  • the USAC decoder is forced to skip the relatively high complex harmonic bandwidth extension calculation.
  • bandwidth extension or "low power HBE” is engaged, if the flag IpHBE indicated at 600 and 700, 702, 704 is set to a non-zero value.
  • the IpHBE flag may be set by a decoder individually, depending on the available hardware resources. A zero value means the decoder will act fully standard compliant, i.e. as instructed by the first and second control data entities of Fig. 1 b. However, if the value is one, then the non-harmonic bandwidth extension mode will be performed by the processor even when the harmonic bandwidth extension mode is signaled.
  • the present invention provides a lower computational complexity and lower memory consumption requiring processor together with a new decoding procedure.
  • the bitstream syntax of eSBR as defined in [1] shares a common base for both HBE [1] and legacy SBR decoding [2].
  • additional information is encoded into the bitstream.
  • the "low complexity HBE" decoder in a preferred embodiment of the present invention decodes the USAC encoded data according to [1] and discards all HBE specific information. Remaining eSBR data is then fed to and interpreted by the legacy SBR [2] algorithm, i.e. the data is used to apply copy-up patching [2] instead of harmonic transposition.
  • the modification of the eSBR decoding mechanics is, with respect to the syntax changes, illustrated in Figs. 6 and 7a , 7b .
  • the specific HBE information such as sbrPitchInBins information carried by the bitstream is reused.
  • the sbrPitchInBins value might be transmitted within a USAC frame. This value reflects a frequency value which was determined by an encoder to transmit information describing the harmonic structure of the current USAC frame. In order to exploit this value without using the standard HBE functionality, the following inventive method should be applied step by step:
  • Fig. 8a gives a detailed description of the inventive algorithm how to calculate the distance of start and stop patch to the harmonic grid harmonicGrid (hg) Harmonic grid according to (1) source_band QMF patch source band 903 of Fig. 9 dest_band QMF patch destination band 908 of Fig. 9 p_mod_x source_band mod hg k_mod_x dest_band mod hg mod Modulo operation NINT Round to nearest integer sbrRatio SBR ratio, i.e. 1 2 , 3 8 or 1 4 pitchInBins Pitch information transmitted in the bitstream
  • Fig. 8a is discussed in more detail.
  • this control i.e. the whole calculation is performed in the controller 104 of Fig. 1a .
  • the harmonic grid is calculated according to formula (1) as illustrated in Fig. 8b .
  • step 804 determines whether the source-band value is even. If this is the case, then the harmonic grid is determined to be 2, but if this is not the case, then the harmonic grid is determined to be equal to 3.
  • step 810 the modulo calculations are performed.
  • step 812 it is determined whether both modulo-calculation differ. If the results are identical, the procedure ends, and if the results differ, the shift value is calculated as indicated in block 814 as the difference between both mod-calculation results. Then, as also illustrated in step 814, the buffer shift with wraparound is performed. It is worth noting that phase relations are preferably be considered when applying the shift.
  • the whole procedure comprises the step of extracting the sbrPitchInBins information from the bitstream as indicated at 820. Then, the controller calculates the harmonic grid as indicated at 822. Then, in step 824, both the distance of the source start sub-band and the destination start sub-band to the harmonic grid is calculated which corresponds, in the preferred embodiment, to step 810. Finally, as indicated in block 826, the QMF buffer shift, i.e. the wraparound shift within the QMF domain of the High Efficiency AAC non-harmonic bandwidth extension is performed.
  • the harmonic structure of the signal is reconstructed according to the transmitted sbrPitchInBins information even though a non-harmonic bandwidth extension procedure has been performed.
  • aspects have been described in the context of an apparatus for encoding or decoding, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.
  • Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.
  • embodiments of the invention can be implemented in hardware or in software.
  • the implementation can be performed using a non-transitory storage medium such as a digital storage medium, for example a floppy disc, a Hard Disk Drive (HDD), a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.
  • a digital storage medium for example a floppy disc, a Hard Disk Drive (HDD), a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be
  • Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
  • embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer.
  • the program code may, for example, be stored on a machine readable carrier.
  • inventions comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
  • an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
  • a further embodiment of the inventive method is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.
  • the data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitory.
  • a further embodiment of the invention method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein.
  • the data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example, via the internet.
  • a further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein.
  • a processing means for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein.
  • a further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
  • a further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver.
  • the receiver may, for example, be a computer, a mobile device, a memory device or the like.
  • the apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.
  • a programmable logic device for example, a field programmable gate array
  • a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein.
  • the methods are preferably performed by any hardware apparatus.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Quality & Reliability (AREA)
  • Mathematical Physics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
EP13196305.0A 2013-12-09 2013-12-09 Apparatus and method for decoding an encoded audio signal with low computational resources Withdrawn EP2881943A1 (en)

Priority Applications (13)

Application Number Priority Date Filing Date Title
EP13196305.0A EP2881943A1 (en) 2013-12-09 2013-12-09 Apparatus and method for decoding an encoded audio signal with low computational resources
RU2016127582A RU2644135C2 (ru) 2013-12-09 2014-11-28 Устройство и способ декодирования кодированного аудиосигнала с низкими вычислительными ресурсами
CN201480066827.0A CN105981101B (zh) 2013-12-09 2014-11-28 对编码音频信号进行解码的装置、方法和计算机存储介质
CA2931958A CA2931958C (en) 2013-12-09 2014-11-28 Apparatus and method for decoding an encoded audio signal with low computational resources
JP2016536886A JP6286554B2 (ja) 2013-12-09 2014-11-28 低演算資源を用いて符号化済みオーディオ信号を復号化する装置及び方法
ES14808907.1T ES2650941T3 (es) 2013-12-09 2014-11-28 Método y aparato para decodificar una señal de audio codificada con bajos recursos computacionales
PCT/EP2014/076000 WO2015086351A1 (en) 2013-12-09 2014-11-28 Apparatus and method for decoding an encoded audio signal with low computational resources
KR1020167015028A KR101854298B1 (ko) 2013-12-09 2014-11-28 낮은 계산 자원들로 인코딩된 오디오 신호를 디코딩하기 위한 장치 및 방법
MX2016007430A MX353703B (es) 2013-12-09 2014-11-28 Método y aparato para decodificar una señal de audio codificada con bajos recursos computacionales.
EP14808907.1A EP3080803B1 (en) 2013-12-09 2014-11-28 Apparatus and method for decoding an encoded audio signal with low computational resources
BR112016012689-0A BR112016012689B1 (pt) 2013-12-09 2014-11-28 aparelho e método para decodificar um sinal de áudio codificado com baixos recursos computacionais
US15/177,265 US9799345B2 (en) 2013-12-09 2016-06-08 Apparatus and method for decoding an encoded audio signal with low computational resources
US15/621,938 US10332536B2 (en) 2013-12-09 2017-06-13 Apparatus and method for decoding an encoded audio signal with low computational resources

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP13196305.0A EP2881943A1 (en) 2013-12-09 2013-12-09 Apparatus and method for decoding an encoded audio signal with low computational resources

Publications (1)

Publication Number Publication Date
EP2881943A1 true EP2881943A1 (en) 2015-06-10

Family

ID=49725065

Family Applications (2)

Application Number Title Priority Date Filing Date
EP13196305.0A Withdrawn EP2881943A1 (en) 2013-12-09 2013-12-09 Apparatus and method for decoding an encoded audio signal with low computational resources
EP14808907.1A Active EP3080803B1 (en) 2013-12-09 2014-11-28 Apparatus and method for decoding an encoded audio signal with low computational resources

Family Applications After (1)

Application Number Title Priority Date Filing Date
EP14808907.1A Active EP3080803B1 (en) 2013-12-09 2014-11-28 Apparatus and method for decoding an encoded audio signal with low computational resources

Country Status (11)

Country Link
US (2) US9799345B2 (pt)
EP (2) EP2881943A1 (pt)
JP (1) JP6286554B2 (pt)
KR (1) KR101854298B1 (pt)
CN (1) CN105981101B (pt)
BR (1) BR112016012689B1 (pt)
CA (1) CA2931958C (pt)
ES (1) ES2650941T3 (pt)
MX (1) MX353703B (pt)
RU (1) RU2644135C2 (pt)
WO (1) WO2015086351A1 (pt)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112863527A (zh) * 2017-03-23 2021-05-28 杜比国际公司 用于音频信号的高频重建的谐波转置器的后向兼容集成
US20230137053A1 (en) * 2020-05-30 2023-05-04 Huawei Technologies Co., Ltd. Audio Coding Method and Apparatus

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI758146B (zh) * 2015-03-13 2022-03-11 瑞典商杜比國際公司 解碼具有增強頻譜帶複製元資料在至少一填充元素中的音訊位元流
TWI834582B (zh) * 2018-01-26 2024-03-01 瑞典商都比國際公司 用於執行一音訊信號之高頻重建之方法、音訊處理單元及非暫時性電腦可讀媒體
CA3152262A1 (en) 2018-04-25 2019-10-31 Dolby International Ab Integration of high frequency reconstruction techniques with reduced post-processing delay
US11527256B2 (en) * 2018-04-25 2022-12-13 Dolby International Ab Integration of high frequency audio reconstruction techniques

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2169670A2 (en) * 2008-09-25 2010-03-31 LG Electronics Inc. An apparatus for processing an audio signal and method thereof

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
SE9700772D0 (sv) * 1997-03-03 1997-03-03 Ericsson Telefon Ab L M A high resolution post processing method for a speech decoder
US6850884B2 (en) * 2000-09-15 2005-02-01 Mindspeed Technologies, Inc. Selection of coding parameters based on spectral content of a speech signal
AU2004319555A1 (en) * 2004-05-17 2005-11-24 Nokia Corporation Audio encoding with different coding models
US8880410B2 (en) * 2008-07-11 2014-11-04 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating a bandwidth extended signal
EP2239732A1 (en) * 2009-04-09 2010-10-13 Fraunhofer-Gesellschaft zur Förderung der Angewandten Forschung e.V. Apparatus and method for generating a synthesis audio signal and for encoding an audio signal
ES2400661T3 (es) 2009-06-29 2013-04-11 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Codificación y decodificación de extensión de ancho de banda
KR101826331B1 (ko) * 2010-09-15 2018-03-22 삼성전자주식회사 고주파수 대역폭 확장을 위한 부호화/복호화 장치 및 방법
CN102208188B (zh) * 2011-07-13 2013-04-17 华为技术有限公司 音频信号编解码方法和设备

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2169670A2 (en) * 2008-09-25 2010-03-31 LG Electronics Inc. An apparatus for processing an audio signal and method thereof

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ANONYMOUS: "Study on ISO/IEC 23003-3:201x/DIS of Unified Speech and Audio Coding", IEEE, LIS, SOPHIA ANTIPOLIS CEDEX, FRANCE, no. N12013, 22 April 2011 (2011-04-22), XP030018506, ISSN: 0000-0001 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112863527A (zh) * 2017-03-23 2021-05-28 杜比国际公司 用于音频信号的高频重建的谐波转置器的后向兼容集成
US12094480B2 (en) 2017-03-23 2024-09-17 Dolby International Ab Backward-compatible integration of harmonic transposer for high frequency reconstruction of audio signals
US20230137053A1 (en) * 2020-05-30 2023-05-04 Huawei Technologies Co., Ltd. Audio Coding Method and Apparatus
US12062379B2 (en) * 2020-05-30 2024-08-13 Huawei Technologies Co., Ltd. Audio coding of tonal components with a spectrum reservation flag

Also Published As

Publication number Publication date
JP6286554B2 (ja) 2018-02-28
EP3080803A1 (en) 2016-10-19
CA2931958C (en) 2018-10-02
US20170278522A1 (en) 2017-09-28
US20160284359A1 (en) 2016-09-29
WO2015086351A1 (en) 2015-06-18
CN105981101A (zh) 2016-09-28
JP2016539377A (ja) 2016-12-15
US10332536B2 (en) 2019-06-25
MX353703B (es) 2018-01-24
BR112016012689B1 (pt) 2021-02-09
EP3080803B1 (en) 2017-10-04
CA2931958A1 (en) 2015-06-18
ES2650941T3 (es) 2018-01-23
US9799345B2 (en) 2017-10-24
KR20160079878A (ko) 2016-07-06
CN105981101B (zh) 2020-04-10
RU2644135C2 (ru) 2018-02-07
MX2016007430A (es) 2016-08-19
KR101854298B1 (ko) 2018-05-03

Similar Documents

Publication Publication Date Title
US10332536B2 (en) Apparatus and method for decoding an encoded audio signal with low computational resources
RU2665887C1 (ru) Декодирование битовых аудиопотоков с метаданными расширенного копирования спектральной полосы по меньшей мере в одном заполняющем элементе
RU2740688C1 (ru) Обратно совместимая интеграция методов высокочастотного восстановления для аудиосигналов
CN112204659B (zh) 具有减少后处理延迟的高频重建技术的集成
US12094480B2 (en) Backward-compatible integration of harmonic transposer for high frequency reconstruction of audio signals
CN112189231A (zh) 高频音频重建技术的集成

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20131209

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN

18W Application withdrawn

Effective date: 20151113