EP2884490B1 - Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system - Google Patents

Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system Download PDF

Info

Publication number
EP2884490B1
EP2884490B1 EP14199155.4A EP14199155A EP2884490B1 EP 2884490 B1 EP2884490 B1 EP 2884490B1 EP 14199155 A EP14199155 A EP 14199155A EP 2884490 B1 EP2884490 B1 EP 2884490B1
Authority
EP
European Patent Office
Prior art keywords
frame
windowed
samples
output
input
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP14199155.4A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP2884490A1 (en
Inventor
Bernhard Grill
Markus Schnell
Ralf Geiger
Gerald Schuller
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Original Assignee
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Publication of EP2884490A1 publication Critical patent/EP2884490A1/en
Application granted granted Critical
Publication of EP2884490B1 publication Critical patent/EP2884490B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/022—Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/0212—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using orthogonal transformation
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/08—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters
    • G10L19/12—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters the excitation function being a code excitation, e.g. in code excited linear prediction [CELP] vocoders
    • G10L19/135—Vector sum excited linear prediction [VSELP]

Definitions

  • the present invention relates to an analysis filterbank, a synthesis filterbank and systems comprising any of the aforementioned filterbanks, which can, for instance, be implemented in the field of modern audio encoding, audio decoding or other audio transmission-related applications. Moreover, the present invention also relates to a mixer and a conferencing system.
  • Modern digital audio processing is typically based on coding schemes which enable a significant reduction in terms of bitrates, transmission bandwidths and storage space, compared to a direct transmission or storage of the respective audio data. This is achieved by encoding the audio data on the sender site and decoding the encoded data on the receiver site before, for instance, providing the decoded audio data to a listener.
  • Such digital audio processing systems can be implemented with respect to a wide range of parameters comprising a typical storage space for a typical potentailly standardized stream of audio data, bitrates, computational complexity especially in terms of an efficiency of an implementation, achievable qualities suitable for different applications and in terms of the delay caused during both, the encoding and the decoding of the audio data and the encoded audio data, respectively.
  • digital audio systems can be applied in many different fields of applications ranging from an ultra-low quality transmission to a high-end-transmission and storage of audio data (e.g. for a high-quality music listening experience).
  • a digital audio system comprising a low delay may require a higher bitrate of a transmission bandwidth compared to an audio system with a higher delay at a comparable quality level.
  • the WO 98/02971 A1 relates to a method for coding and decoding audio signals.
  • the method of encoding time-discrete audio signals comprises the steps of weighting the time-discrete audio signal by means of window functions overlapping each other so as to form blocks, the window functions producing blocks of a first length for signals varying weakly with time and blocks of a second length for signals varying strongly with time.
  • a start window sequence is selected for the transition from windowing with blocks of the first length to windowing with blocks of the second length, whereas a stop window sequence is selected for the opposite transition.
  • the start window sequence is selected from at least two different start window sequences having different lengths, whereas the stop window sequence is selected from at least two different stop window sequences having different lengths.
  • the method of decoding blocks of encoded audio signals selects a suitable inverse transformation as well as a suitable synthesis window as a reaction to side information associated with each block.
  • An embodiment of a synthesis filterbank according to claim 1, for filtering a plurality of input frames, wherein each input frame comprises a number of ordered input values, comprises a frequency/time converter configured to providing a plurality of output frames, wherein an output frame comprises a number of ordered output samples, wherein an output frame is a time representation of an input frame, a windower configured to generating a plurality of windowed frames.
  • a windowed frame comprises a plurality of windowed samples.
  • the windower is furthermore configured to providing the plural-ity of windowed samples for a processing in an overlapping manner based on a sample advance value.
  • the embodiment of the synthesis filterbank further comprises an overlap/adder configured to providing an added frame comprising a start section and a remainder section, wherein an added frame comprises a plurality of added samples by adding at least three windowed samples from at least three windowed frames for an added sample in the remainder section of an added frame and by adding at least two windowed samples from at least two different windowed frames for an added sample in the start section.
  • the number of windowed samples added to obtain an added sample in the remainder section is at least one sample higher compared to the number of windowed samples added to obtain an added sample in the start section,
  • the windower is configured to disregarding at least the earliest output value according to the order of the ordered output samples or to setting the corresponding windowed samples to a predetermined value or to at least a value in a predetermined range for each windowed frame of the plureality of windowed frames.
  • the overlap/adder (230) is configured to providing the added sample in the remainder section of an added frame based on at least three windowed samples from at least three different windowed frames and an added sample in the start section based on at least two windowed samples from at least two different windowed frames.
  • Figs. 1 to 24 show block diagrams and further diagrams describing the functional properties and features of different embodiments of an analysis filterbank, a synthesis filterbank, an encoder, a decoder, a mixer, a conferencing system and other embodiments of the present invention.
  • an embodiment of an analysis filterbank and a schematic representation of input frames being processed by an embodiment of an analysis filterbank will be described in more detail.
  • Fig. 1 shows a first embodiment of an analysis filterbank 100 comprising a windower 110 and time/frequency converter 120.
  • the windower 110 is configured to receiving a plurality of time-domain input frames, each input frame comprising a number of ordered input samples at an input 110i.
  • the windower 110 is furthermore adapted to generating a plurality of windowed frames, which are provided by the windower at the output 110o of the windower 110.
  • Each of the windowed frames comprises a plurality of windowed samples, wherein the windower 110 is furthermore configured to processing the plurality of windowed frames in an overlapping manner using a sample advance value as will be explained in more detail in the context of Fig. 2 .
  • the time/frequency converter 120 is capable of receiving the windowed frames as output by the windower 110 and configured to providing an output frame comprising a number of output values, such that an output frame is a spectral representation of a windowed frame.
  • FIG. 2 shows a schematic representation of five input frames 130-(k-3), 130-(k-2), 130-(k-1), 130-k and 130-(k+1) as a function of time, as indicated by an arrow 140 at the bottom of Fig. 2 .
  • the operation of an embodiment of an analysis filterbank 100 will be described in more detail with reference to the input frame 130-k, as indicated by the dashed line in Fig. 2 .
  • the input frame 103-(k+1) is a future input frame
  • the three input frames 130-(k-1), 130-(k-2), and 130-(k-3) are past input frames.
  • k is an integer indicating a frame index, such that the larger the frame index is, the farther the respective input frame is located "in the future". Accordingly, the smaller the index k is, the farther the input frame is located "in the past”.
  • Each of the input frames 130 comprises at least two subsections 150, which are equally long.
  • the input frame 130-k as well as the other input frames 130 comprise subsections 150-2, 150-3 and 150-4 which are equal in length in terms of input samples.
  • Each of these subsections 150 of the input frame 130 comprises M input samples, wherein M is a positive integer.
  • the input frame 130 also comprises a first subsection 150-1 which may comprise also M input frames.
  • the first subsection 150-1 comprises an initial section 160 of the input frame 130, which may comprise input samples or other values, as will be explain in more detail at a later stage.
  • the first subsection 150-1 is not required to comprise an initial section 160 at all.
  • the first subsection 150-1 may in principle comprise a lower number of input samples compared to the other subsections 150-2, 150-3, 150-4. Examples for this case will also be illustrated later on.
  • the other subsections 150-2, 150-3, 150-4 comprise typically the same number of input samples M, which is equal to the socalled sample advance value 170, which indicates a number of input samples by which two consecutive input frames 130 are moved with respect to time and each other.
  • sample advance value M as indicated by an arrow 170 is, in the case of an embodiment of an analysis filterbank 100, as illustrated in Figs. 1 and 2 equal to the length of the subsections 150-2, 150-3, 150-4, the input frames 130 are generated and processed by the windower 110 in an overlapping manner.
  • the sample advance value M (arrow 170) is also identical with the length of the subsections 150-2 to 150-4.
  • the input frames 130-k and 130-(k+1) are, hence, in terms of a significant number of input samples, equal in the sense that both input frames comprise these input samples, while they are shifted with respect to the individual subsections 150 of the two input frames 130.
  • the third subsection 150-3 of the input frame 130-k is equal to the fourth subsection 150-4 of the input frame 130-(k+1).
  • the second subsection 150-2 of the input frame 130-k is identical to the third subsection 150-3 of the input frame 130-(k+1).
  • the two input frames 130-k, 130-(k+1) corresponding to the frame indices k and (k+1) are in terms of two subsections 150 in the case of the embodiments shown in Fig. 2 , identical, apart from the fact that in terms of the input frame with the index frame (k+1), the samples are moved.
  • the two aforementioned input frames 130-k and 130-(k+1) furthermore share at least one sample from the first subsection 150-1 of the input frame 130-k.
  • all input samples in the first subsection 150-1 of the input frame 130-k, which are not part of the initial section 160 appear as part of the second subsection 150-2 of the input frame 130-(k+1).
  • the input samples of the second subsection 150-2 corresponding to the initial section 160 of the input frame 130-k before may or may not be based on the input values or input samples of the initial section 160 of the respective input frame 130, depending on the concrete implementation of an embodiment of an analysis filterbank.
  • the initial section 160 comprises "meaningful" encoded input samples in the sense that the input samples in the initial section 160 do represent an audio signal in the time-domain, these input samples will also be part of the subsection 150-2 of the following input frame 130-(k+1). This case, is however, in many applications of an embodiment of an analysis filterbank, not an optimal implementation, as this option might cause additional delay.
  • the corresponding input values of the initial section 160 may comprise random values, a predetermined, fixed, adaptable or programmable value, which can for instance be provided in terms of an algorithmic calculation, determination or other fixing by a unit or module, which may be coupled to the input 110i of the windower 110 of the embodiment of the analysis filterbank.
  • this module is typically required to provide as the input frame 130-(k+1), an input frame which comprises in the second subsection 150-2 in the area corresponding to the initial section 160 of the input frame before "meaningful" input samples, which do correspond to the corresponding audio signal.
  • the unit or module coupled to the input 110i of the windower 110 is typically also required to provide meaningful input samples corresponding to the audio signal in the framework of the first subsection 150-1 of the input frame 130-(k+1).
  • the input frame 130-k corresponding to the frame index k is provided to the embodiment of an analysis filterbank 100 after sufficient input samples are gathered, such that the subsection 150-1 of this input frame can be filled with these input samples.
  • the rest of the first subsection 150-1, namely the initial section 160 is then filled up with input samples or input values, which may comprise random values or any other values such as a predetermined, fixed, adaptable or programmable value or any other combination of values.
  • a typical sampling frequency such as a sampling frequency in the range between a few kHz and up to several 100 kHz.
  • the unit or module continues collecting input samples based on the audio signal to incorporate these input samples into the following input frame 130-(k+1) corresponding to the frame index k+1.
  • the module or unit did not finish collecting sufficient input samples to provide the input frame 130-k in terms of the first subsection 150-1 with sufficient input samples to completely fill up the first subsection 150-1 of this input frame, but provides this input frame to the embodiment of the analysis filterbank 100 as soon as enough input samples are available, such that the first subsection 150-1 can be filled up with input samples without the initial section 160.
  • the following input samples will be used to fill up the remaining input samples of the second subsection 150-2 of the following input frame 130-(k+1) until enough input samples are gathered, such that the first subsection 150-1 of this next input frame can also be filled until the initial section 160 of this frame begins.
  • the initial section 160 will be filled up with random numbers or other "meaningless" input samples or input values.
  • sample advance value 170 which is equal to the length of the subsection 150-2 to 150-4 in the case of the embodiment shown in Fig. 2 is indicated in Fig. 2 and the error representing the sample advance value 170 is shown in Fig. 2 from the beginning of the initial section 160 of the input frame 130-k until the beginning of the initial section 160 of the following input frame 130-(k+1).
  • an input sample corresponding to an event in the audio signal corresponding to the initial section 160 will in the last two cases will not be present in the respective input frames 130-k, but in the following input frame 130-(k+1) in the framework of the second subsection 150-2.
  • an embodiment of an analysis filterbank 100 may provide an output frame with a reduced delay as the input samples corresponding to the initial section 160 are not part of the respective input frame 130-k but will only be influencing the later input frame 130-(k+1).
  • an embodiment of an analysis filterbank may offer in many applications and implementations the advantage of providing the output frame based on the input frame sooner, as the first subsection 150-1 is not required to comprise the same number of input samples as the other subsection 150-2 to 150-4.
  • the information comprised in the "missing section" is comprised in the next input frame 130 in the framework of the second subsection 150-2 of that respective input frame 130.
  • the length of each of the input frames 130 is no longer an integer multiple of the sample advance value 170 or the length of the subsection 150-2 to 150-4.
  • the length of each of the input frames 130 differs from the corresponding integer multiples of the sample advance value by the number of input samples, which the module or unit providing the windower 110 with the respective input frames stops short of providing the full first subsection 150-1.
  • the overall length of such an input frame 130 differs from the respective integer number of sample advance values by the difference between the lengths of the first subsection 150-1 compared to the length of the other subsections 150-2 to 150-4.
  • the module or unit which can for instance comprise a sampler, a sample-and-hold-stage, a sample-and-holder or a quantizer, may start providing the corresponding input frame 130 short of a predetermined number of input samples, such that each of the input frames 130 can be provided to the embodiment of an analysis filterbank 100 with a shorter delay as compared to the case in which the complete first subsection 150-1 is filled with corresponding input samples.
  • such a unit or module which can be coupled to the input 110i of the windower 110 may for instance comprise a sampler and/or a quantizer such as an analog/digital converter (A/D converter).
  • a module or unit may further comprise some memory or registers to store the input samples corresponding to the audio signal.
  • such a unit or module may provide each of the input frames in an overlapping manner, based on a sample advanced value M.
  • an input frame comprises more than twice the number of input samples compared to the number of samples gathered per frame or block.
  • Such a unit or module is in many embodiments adapted such that two consecutively generated input frames are based on a plurality of samples which are shifted with respect to time by the sample advance value.
  • the later input frame of the two consecutively generated input frames is based on at least one fresh output sample as the earliest output sample and the aforementioned plurality of samples is shifted later by the sample advance value in the earlier input frame of the two input frames.
  • an input frame 130 may comprise in principle, an arbitrary number of input samples, which is larger than twice the size of the sample advance value M (arrow 170), wherein the number of input values of the initial section 160, if present, are required to be included in this number, as it might be helpful considering some implementations of an embodiment based on a system utilizing frames, wherein each frame comprises a number of samples which is identical to the sample advance value.
  • any number of subsections, each having a length identical to the sample advance value M (arrow 170) can be used in the framework of an embodiment of an analysis filterbank 100, which is greater or equal to three in the case of a frame based system. If this is not the case, in principle, any number of input samples per input frame 130 can be utilized being greater than twice the sample advance value.
  • the windower 110 of an embodiment of an analysis filterbank 100 is configured to generating a plurality of windowed frames based on the corresponding input frames 130 on the basis of the sample advance value M (arrow 170) in an overlapping manner as previously explained.
  • the windower 110 is configured to generating the windowed frame, based on a weighing function, which may for instance comprise a logarithmic dependence to model the hearing characteristics of the human ear.
  • a weighing function may for instance comprise a logarithmic dependence to model the hearing characteristics of the human ear.
  • other weighing functions may also be implemented, such as a weighing function modeling, the psycho-acoustic characteristics of the human ear.
  • the windower function is implemented in an embodiment of an analysis filterbank, can, for instance, also be implemented such that each of the input samples of an input frame is multiplied by a real-valued windower function comprising real-valued sample-specific window coefficients.
  • FIG. 2 shows a schematical crude representation of a possible window function or a windowing function 180, by which the windower 110, as shown in Fig. 1 generates the windowed frames, based on the corresponding input frames 130.
  • the windower 110 can furthermore provide windowed frames to the time/frequency converter 120 in a different way.
  • the windower 110 is configured to generating a windowed frame, wherein each of the windowed frames comprises a plurality of windowed samples.
  • the windower 110 can be configured in different ways. Depending on the length of an input frame 130 and depending on the length of the windowed frame to be provided to the time/frequency provider 120, several possibilities of how the windower 110 is implemented to generate the windowed frames can be realized.
  • an input frame 130 comprises an initial section 160, so that in a case of an embodiment shown in Fig. 2 the first subsection 150-1 of each of the input frames 130 comprises as many input values or input samples as the other subsections 150-2 to 150-4, the windower 110 can for instance be configured such that the windowed frame also comprises the same number of windowed samples as the input frame 130 comprises input samples of input values.
  • all the input samples of the input frame apart from the input values of the input frames 130 in the initial section 160 may be processed by the windower 110 based on the windowing function or the window function as previously described.
  • the input values of the initial section 160 may, in this case, be set to a predetermined value or to at least one value in a predetermined range.
  • the predetermined value may for instance be an embodiment of some analysis filterbank 100 equal to the value 0 (zero), whereas in other embodiments, different values may be desirable. For instance, it is possible to use, in principle, any value with respect to the initial section 160 of the input frames 130, which indicates that the corresponding values are of no significance in terms of the audio signal. For instance, the predetermined value may be a value which is outside of a typical range of input samples of an audio signal. For instance, windowed samples inside a section of the windowed frame corresponding to the initial section 160 of the input frame 130 may be set to a value of twice or more the maximum amplitude of an input audio signal indicating that these values do not correspond to signals to be processed further. Other values, for instance negative values of an implementation-specific absolute value, may also be used.
  • windowed samples of the windowed frames corresponding to the initial section 160 of an input frame 130 can also be set to one or more values in a predetermined range.
  • a predetermined range may for instance be a range of small values, which are in terms of an audio experience meaningless, so that the outcome is audibly indistinguishable or so that the listening experience is not significantly disturbed.
  • the predetermined range may for instance be expressed as a set of values having an absolute value, which is smaller than or equal to a predetermined, programmable, adaptable or fixed maximum threshold.
  • a threshold may for instance be expressed as a power of 10 or a power or two as 10 s or 2 s , where the s is an integer value depending on the concrete implementation.
  • the predetermined range may also comprise values, which are larger than some meaningful values.
  • the predetermined range may also comprise values, which comprise an absolute value, which is larger than or equal to a programmable, predetermined or fixed minimum threshold.
  • a minimum threshold may in principle be expressed once again in terms of a power of two or a power of ten, as 2 s or 10 s , wherein s is once again an integer depending on the concrete implementation of an embodiment of an analysis filterbank.
  • the predetermined range can for instance comprise values which are expressible by setting or not setting the least significant bit or plurality of least significant bits in the case of a predetermined range comprising small values.
  • the predetermined range may comprise values, representable by setting or not setting the most significant bit or a plurality of most significant bits.
  • the predetermined value as well as the predetermined ranges may also comprise other values, which can for instance be created based on the aforementioned values and thresholds by multiplying these with a factor.
  • the windower 110 may also be adapted such that the windowed frames provided at the output 110o do not comprise windowed samples corresponding to input frames of the initial sections 160 of the input frames 130.
  • the length of the windowed frame and the length of the corresponding input frames 130 may for instance differ by the length of the initial section 160.
  • the windower 110 may be configured or adapted to disregarding at least a latest input sample according to the order of the input samples as previously described in terms of time.
  • the windower 110 may be configured such that one or more or even all input values or input samples of the initial section 160 of an input frame 130 are disregarded.
  • the length of the windowed frame is equal to the difference between the lengths of the input frame 130 and the length of the initial section 160 of the input frame 130.
  • each of the input frames 130 may not comprise an initial section 160 at all, as indicated before.
  • the first subsection 150-1 differs in terms of the length of the respective subsection 150, or in terms of the number of input samples from the other subsections 150-2 to 150-4.
  • the windowed frame may or may not, comprise windowed samples or windowed values such that a similar first subsection of the windowed frame corresponding to the first subsection 150-1 of the input frame 130 comprises the same number as windowed samples or windowed values as the other subsections corresponding to the subsections 150 of the input frame 130.
  • the additional windowed samples or windowed values can be set to a predetermined value or at least one value in the predetermined range, as indicated earlier.
  • the windower 110 may be configured in embodiments of an analysis filterbank 100 such that both, the input frame 130 and the resulting windowed frame comprise the same number of values or samples and wherein both, the input frame 130 and the resulting windowed frames do not comprise the initial section 160 or samples corresponding to the initial section 160.
  • the first subsection 150-1 of the input frame 130 as well as the corresponding subsection of the windowed frame comprise less values or samples compared to the other subsections 150-2 to 150-4 of the input frame 130 of the corresponding subsections of the windowed frame.
  • the windowed frame is not required to correspond either to a length of an input frame 130 comprising an initial section 160, or to an input frame 130 not comprising an initial section 160.
  • the windower 110 may also be adapted such that the windowed frame comprises one or more values or samples corresponding to values of the initial section 160 of an input frame 130.
  • the initial section 160 represents or at least comprises a connected subset of sample indices n corresponding to a connected subset of input values or input samples of an input frame 130.
  • the windowed frames comprising a corresponding initial section comprises a connected subset of sample indices n of windowed samples corresponding to the respective initial section of the windowed frame, which is also referred to as the starting section or start section of the windowed frame.
  • the rest of the windowed frame without the initial section or starting section which is sometimes also referred to as the remainder section.
  • the windower 110 can in embodiments of an analysis filterbank 100 be adapted to generating the windowed samples of windowed values of a windowed frame not corresponding to the initial section 160 of an input frame 130, if present at all, based on a window function which may incorporate psycho-acoustic models, for instance, in terms of generating the windowed samples based on a logarithmic calculation based on the corresponding input samples.
  • the windower 110 can also be adapted in different embodiments of an analysis filterbank 100, such that each of the windowed samples is generated by multiplying a corresponding input sample with a sample-specific windowed coefficient of the window function defined over a definition set.
  • the corresponding windower 110 is adapted such that the window function, as for instance, described by the window coefficients, is asymmetric over the definition set with respect to a midpoint of the definition set.
  • the window coefficients of the window function comprise an absolute value of more than 10%, 20% or 30%, 50% of a maximum absolute value of all window coefficients of the window function in the first half of the definition set with respect to the midpoint, wherein the window function comprises less window coefficients having an absolute value of more than the aforementioned percentage of the maximum absolute value of the window coefficients in the second half of the definition set, with respect to the midpoint.
  • a window function is schematically shown in context of each of the input frames 130 in Fig.
  • window function 180 More examples of window functions will be described in the context of the Figs. 5 to 11 , including a brief discussion of spectral and other properties and opportunities offered by some embodiments of an analysis filterbank as well as a synthesis filterbank implementing window functions as shown in these figures and described in passages.
  • an embodiment of an analysis filterbank 100 also comprises the time/frequency converter 120, which is provided with the windowed frames from the windower 110.
  • the time/frequency converter 120 is in turn adapted to generating an output frame or a plurality of output frames for each of the windowed frames such that the output frame is a spectral representation of the corresponding windowed frame.
  • the time/frequency converter 120 is adapted such that the output frame comprises less than half the number of output values compared to the number of input samples of an input frame, or compared to half the number of windowed samples of a windowed frame.
  • time/frequency converter 120 may be implemented such that it is based on a discrete cosine transform and/or a discrete sine transform such that the number of output samples of an output frame is less than half the number of input samples of an input frame.
  • a discrete cosine transform and/or a discrete sine transform such that the number of output samples of an output frame is less than half the number of input samples of an input frame.
  • a time/frequency converter 120 is configured such that it outputs a number of output samples, which is equal to the number of input samples of a starting section 150-2, 150-3, 150-4, which is not the starting section of the first subsection 150-1 of the input frame 130, or which is identical to the sample advance value 170.
  • the number of output samples is equal to the integer M representing the sample advance value of a length of the aforementioned subsection 150 of the input frame 130.
  • Typical values of the sample advance value or M are in many embodiments 480 or 512.
  • M 360.
  • the initial section 160 of an input frame 130 or the difference between the number of samples in the other subsections 150-2, 150-3, 150-4 and the first subsection 150-1 of an input frame 130 is equal to M/4.
  • M 480
  • different lengths can also be implemented and do not represent a limit in terms of an embodiment of an analysis filterbank 100.
  • the time/frequency converter 120 can for instance be based on a discrete cosine transform or a discrete sine transform
  • MDCT modified discrete cosine transform
  • an embodiment of an analysis filterbank 100 may offer as an advantage a lower delay of a digital audio processing without reducing the audio quality at all, or somehow significantly.
  • the input frames of embodiments of an analysis filterbank 100 are not required to comprise the four subsections 150-1 to 150-4 as illustrated in Fig. 2 .
  • the windower is not required to be adapted such that the windowed frames also comprise four corresponding subsections or the time/frequency converter 120 to be adapted such that it is capable of providing the output frame based on a windowed frame comprising four subsections.
  • statements in the context of the input frame in terms of the length of the input frames 130 can also be transferred to the length of the windowed frames as explained in the context of the different options concerning the initial section 160 and its presence in the input frames 130.
  • an embodiment of an analysis filterbank in view of an error resilient advanced audio codec low delay implementation (ER AAC LD) will be explained with respect to modifications in order to adapt the analysis filterbank of the ER AAC LD to arrive at an embodiment of an analysis filterbank 100 which is also sometimes referred to as a low-delay (analysis filterbank).
  • ER AAC LD error resilient advanced audio codec low delay implementation
  • the synthesis window function w is used as the analysis window function by inverting the order, as can be seen by comparing the argument of the window function w(n-1-n).
  • Fig. 5 shows a plot of the low-delay window functions, wherein the analysis window is simply a time-reverse replica of the synthesis window.
  • x' i,n represents an input sample or input value corresponding to the block index i and the sample index n.
  • k is an integer indicating the spectral coefficient index and N is an integer indictating twice the number of output values of an output frame, or as previously explained, the window length of one transform window based on the windows_sequence value as implemented in the ER AAC LD codec.
  • the time/frequency converter may be implemented based on a windowed frame comprising windowed samples corresponding to the initial section 160 of the input frames 130.
  • the equations above are based on windowed frames comprising a length of 1920 windowed samples.
  • the equations given above can be adapted such that the corresponding equations are carried out.
  • this can for instance lead to the sample index n running from the -N,..., 7N/8-1 in the case of M/4 N/8 windowed samples missing in the first subsection, compared to the other subsections of the windowed frame as previously explained.
  • the equation given above can easily be adapted by modifying the summation indices accordingly to not incorporate the windowed samples of the initial section or starting section of the windowed frame.
  • further modifications can easily be obtained accordingly in the case of a different length of the initial section 160 of the input frames 130 or in the case of the difference between the length of the first subsection and the other subsections of the windowed frame, as also previously explained.
  • an analysis filterbank 100 may also comprise an implementation in which the number of calculations can be even more reduced, in principle, leading to a higher computational efficiency.
  • An example in the case of the synthesis filterbank will be described in the context of Fig. 19 .
  • an embodiment of an analysis filterbank 100 can be implemented in the framework of a socalled error resilient advanced audio codec enhanced low-delay (ER AAC ELD) which is derived from the aforementioned ER AAC LD codec.
  • ER AAC ELD error resilient advanced audio codec enhanced low-delay
  • the analysis filterbank of the ER AAC LD codec is modified to arrive at an embodiment of an analysis filterbank 100 in order to adopt the low-delay filterbank as an embodiment of an analysis filterbank 100.
  • the ER AAC ELD codec comprising an embodiment of an analysis filterbank 100 and/or an embodiment of a synthesis filterbank, which will be explained in more detail later on, provides the ability to extend the usage of generic low bitrate audio coding to applications requiring a very low delay of the encoding/decoding chain. Examples come for instance from the field of full-duplex real-time communications, in which different embodiments can be incorporated, such as embodiments of an analysis filterbank, a synthesis filterbank, a decoder, and encoder, a mixer and a conferencing system.
  • a first component which is coupled to a second component can be directly connected or connected via a further circuitry or further component to the second component.
  • two components being close to each other comprise the two alternatives of the components being directly connected to each other or via a further circuitry of a further component.
  • Fig. 3 shows an embodiment of a synthesis filterbank 200 for filtering a plurality of input frames, wherein each input frame comprises a number of ordered input values.
  • the embodiment of the synthesis filterbank 200 comprises a frequency/time converter 210, a windower 220 and an overlap/adder 230 coupled in series.
  • a plurality of input frames provided to the embodiment of the synthesis filter bank 200 will be processed first by the frequency/time converter 210. It is capable of generating a plurality of output frames based on the input frames so that each output frame is a time representation of the corresponding input frame. In other words, the frequency/time converter 210 performs a transition for each input frame from the frequency-domain to the time-domain.
  • the windower 220 which is coupled to the frequency/time converter 210, is then capable of processing each output frame as provided by the frequency/time converter 210 to generate a windowed frame based on this output frame.
  • the windower 220 is capable of generating the windowed frames by processing each of the output samples of each of the output frames, wherein each windowed frame comprises a plurality of windowed samples.
  • the windower 220 is capable of generating the windowed frames based on the output frames by weighing the output samples based on a weighing function.
  • the weighing function may, for instance, be based on a psycho-acoustic model incorporating the hearing capabilities or properties of the human ear, such as the logarithmic dependency of the loudness of an audio signal.
  • the windower 220 may also generate the windowed frame based on the output frame by multiplying each output sample of an output frame with a sample-specific value of a window, windowing function or window function. These values are also referred to as window coefficients or windowing coefficients.
  • the windower 220 may be adapted in at least some embodiments of a synthesis filterbank 200 to generate the windowed samples of a windowed frame by multiplying these with a window function attributing a real-valued window coefficient to each of a set of elements of a definition set.
  • window function may be asymmetric or non-symmetric with respect to a midpoint of the definition set, which in turn is not required to be an element of the definition set itself.
  • the windower 220 generates the plurality of windowed samples for a further processing in an overlapping manner based on a sample advance value by the overlap/adder 230, as will be explained in more detail in the context of Fig. 4 .
  • each of the windowed frames comprises more than twice the number of windowed samples compared to a number of added samples as provided by the overlap/adder 230 coupled to an output of the windower 220.
  • the overlap/adder is than capable of generating an added frame in an overlapping manner by adding up at least three windowed samples from at least three different windowed frames for at least some of the added samples in embodiments of a synthesis filterbank 200.
  • the overlap/adder 230 coupled to the windower 220 is then capable of generating or providing an added frame for each newly received windowed frame.
  • the overlap/adder 230 operates the windowed frames in an overlapping manner to generate a single added frame.
  • Each added frame comprises a start section and a remainder section, as will be explained in more detail in the context of Fig. 4 , and comprises furthermore a plurality of added samples by adding at least three windowed samples from at least three different windowed frames for an added in the remainder section of an added frame and by adding at least two windowed samples from at least two different windowed frames for an added samples in the starting section.
  • the number of windowed samples added to obtain an added sample in the remainder section may be at least one sample higher compared to the number of windowed samples added to obtain an added sample in the start section.
  • the windower 220 may also be configured to disregarding the earliest output value according to the order of the ordered output samples, to setting the corresponding windowed samples to a predetermined value or to at least a value in the predetermined range for each windowed frame of the plurality of windowed frames.
  • the overlap/adder 230 may in this case be capable of providing the added sample in the remainder section of an added frame, based on at least three windowed samples from at least three different windowed frames and an added sample in the starting section based on at least two windowed samples from at least two different windowed frames, as will be explained in the context of Fig. 4 .
  • Fig. 4 shows a schematic representation of five output frames 240 corresponding to the frame indices k, k-1, k-2, k-3 and k+1, which are labeled accordingly. Similar to the schematic representation shown in Fig. 2 , the five output frames 240 shown in Fig. 4 are arranged according to their order with respect to time as indicated by an arrow 250. With reference to the output frame 240-k, the output frames 240-(k-1), 240-(k-2) and 240-(k-3) refer to past output frames 240. Accordingly, the output frame 240-(k+1) is with respect to the output frame 240-k a following or future output frame.
  • the output frames 240 shown in Fig. 4 comprise, in the case of the embodiment shown in Fig. 4 , four subsets 260-1, 260-2, 260-3 and 260-4 each.
  • the first subsection 260-1 of each of the output frames 240 may or may not, comprise an initial section 270, as was already discussed in the framework of Fig. 2 in the context of the initial section 160 of the input frames 130.
  • the first subsection 260-1 may be shorter compared to the other subsections 260-2, 260-3 and 260-4 in the embodiment illustrated in Fig. 4 .
  • the other subsections 260-2, 260-3 and 260-4 comprise each a number of output samples equal to the aforementioned sample advance value M.
  • the frequency/time converter 210 is in the embodiment shown in Fig. 3 provided with a plurality of input frames on the basis of which the frequency/time converter 210 generates a plurality of output frames.
  • the length of each of each of the input frames is identical to the sample advance value M, wherein M is once again a positive integer.
  • the output frames generated by the frequency/time converter 210 however do comprise at least more than twice the number of input values of an input frame. To be more precise, in an embodiment in accordance with the situation shown in Fig.
  • the output frames 240 comprise even more than three times the number of output samples compared to the number of input values, each of which also comprises in embodiments related to the shown situation M input values.
  • the output frames can be divided into subsections 260, wherein each of the subsections 260 of the output frames 240 (optionally without the first subsection 260-1, as discussed earlies) comprise M output samples.
  • the sample advance value M is also identical to the lengths of the subsections 260-2, 260-3 and 260-4 of the output frames 240.
  • the first subsection 260-1 of the output frame 240 can comprise M output samples. If, however, the initial section 270 of the output frame 240 does not exist, the first subsection 260-1 of each of the output frames 240 is shorter than the remaining subsections 260-2 to 260-4 of the output frames 240.
  • the frequency/time converter 210 provides to the windower 220 a plurality of the output frames 240, wherein each of the output frames comprises a number of output samples being larger than twice the sample advance value M.
  • the windower 220 is then capable of generating windowed frames, based on the current output frame 240, as provided by the frequency/time converter 210. More explicitly, each of the windowed frames corresponding to an output frame 240 is generated based on the weighing function, as previously mentioned. In an embodiment based on the situation shown in Fig. 4 , the weighing function is in turn based upon a window function 280, which is schematically shown over each of the output frames 240. In this context, it should also be noted that the window function 280 does not yield any contribution for output samples in the initial section 270 of the output frame 240, if present.
  • the windower 220 may be adapted or configured quite differently.
  • the windower 220 can be adapted such that it may or may not generate windowed frames based on the output frames comprising the same number of windowed samples.
  • the windower 220 can be implemented such that it generates windowed frames also comprising an initial section 270, which can be implemented, for instance, by setting the corresponding windowed samples to a predetermined value (e.g. 0, twice a maximum allowable signal amplitude, etc.) or to at least one value in a predetermined range, as previously discussed in the context of Figs. 1 and 2 .
  • a predetermined value e.g. 0, twice a maximum allowable signal amplitude, etc.
  • both, the output frame 240 as well as the windowed frame based upon the output frame 240 may comprise the same number of samples or values.
  • the windowed samples in the initial section 270 of the windowed frame do not necessarily depend on the corresponding output samples of the output frame 240.
  • the first subsection 260-1 of the windowed frame is, however, with respect to the samples not in the initial section 270 based upon the output frame 240 as provided by the frequency/time converter 210.
  • the corresponding windowed sample may be set to a predetermined value, or to a value in a predetermined range, as was explained in the context of the embodiment of an analysis filterbank illustrated in Figs. 1 and 2 .
  • the initial section 270 comprises more than one windowed sample, the same may also be true for this or these other windowed samples or values of the initial section 270.
  • the windower 220 may be adapted such that the windowed frames do not comprise an initial section 270 at all.
  • the windower 220 can be configured to disregarding the output samples of the output frames 240 in the initial section 270 of the output frame 240.
  • the first subsection 260-1 of a windowed frame may or may not comprise the initial section 270. If an initial section of the windowed frame exists, the windowed samples or values of this section are not required to depend on the corresponding output samples of the respective output frame at all.
  • the windower 220 may also be configured to generating a windowed frame based on the output frame 240 comprising or not comprising an initial section 270 itself. If the number of output samples of the first subsection 260-1 is smaller than the sample advance value M, the windower 220 may in some embodiments of a synthesis filterbank 200 be capable of setting the windowed samples corresponding to the "missing output samples" of the initial section 270 of the windowed frame to the predetermined value or to at least one value in the predetermined range.
  • the windower 220 may in this case be capable of filling up the windowed frame with the predetermined value or at least one value in the predetermined range so that the resulting windowed frame comprises a number of windowed samples, which is an integer multiple of the sample advance value M, the size of an input frame or the length of an added frame.
  • both the output frames 240 and the windowed frames might not comprise an initial section 270 at all.
  • the windower 220 may be configured to simply weighing at least some of the output samples of the output frame to obtain the windowed frame.
  • the windower 220 might employ a window function 280 or the like.
  • the initial section 270 of the output frames 240 corresponds to the earliest samples in the output frame 250 in the sense that these values correspond to the "freshest" samples having the smallest sample index.
  • these samples refer to samples corresponding to a smallest amount of time will have elapsed when playing back a corresponding added sample as provided by the overlap/adder 230 compared to the other output samples of the output frame 240.
  • the freshest output samples correspond to a position left in the respective output frame 240 or subsection 260.
  • the time as indicated by the arrow 250 corresponds to the sequence of output frames 240 and not to the sequence of output samples inside each of the output frames 240.
  • the frequency/time converter 210 and/or the windower 220 are adapted such that the initial section 270 of the output frame 240 and the windowed frame are either completely present, or not present at all.
  • the number of output or windowed samples in the first subsection 260-1 is accordingly equal to the number of output samples in an output frame, which is equal to M.
  • a synthesis filterbank 200 can also be implemented, in which the either or both of the frequency/time converter 210 and the windower 220 may be configured such that the initial section 270 is present, but the number of samples in the first subsection 260-1 is yet smaller than the number of output samples in an output frame of a frequency/time converter 210.
  • all samples or values of any of the frames are treated as such, although of course only a single or a fraction of the corresponding values or samples may be utilized.
  • the overlap/adder 230 coupled to the windower 220 is capable of providing an added frame 290, as shown at the bottom of Fig. 4 , which comprises a start section 300 and a remainder section 310.
  • the overlap/adder 230 can be implemented such that an added sample as comprised in the added frame in the start section is obtained by adding at least two windowed samples of at least two different windowed frames.
  • an added sample in the start section 300 is based upon three or four windowed samples or values from at least three or four different windowed frames, respectively, as indicated by an arrow 320.
  • the question, whether three or four windowed samples will be used in the case of the embodiment used in Fig. 4 depends on the concrete implementation of the embodiment in terms of the initial section 270 of the windowed frame based on the corresponding output frame 240-k.
  • the output frames 240 as shown in Fig. 4 as the windowed frames provided by the windower 220 based on the respective output frames 240, as the windowed frames are obtained in the situation illustrated in Fig. 4 by multiplying at least the output samples of the output frames 240 outside the initial section 270 with values derived from the window function 280.
  • the reference sign 240 may also be used for a windowed frame.
  • the windowed sample or windowed value in the initial section 270 may be utilized in adding up the remaining three added samples from the second subsection of the windowed frame 240-(k-1) (corresponding to the output frame 240-(k-1)), the third subsection from the windowed frame 240-(k-2) (corresponding to the output frame 240-(k-2)) and the fourth subsection of the windowed frame 240-(k-3) (corresponding to the output frame 240-(k-3)), if the predetermined value or the predetermined range are such that summing up the windowed sample from the initial section 270 of the windowed frame 240-k (corresponding to the output frame 240-k) does not significantly disturb or alter the outcome.
  • the corresponding added sample in the start section 300 is normally obtained by adding the at least two windowed samples from the at least two windowed frames.
  • the added sample in the start section of the added frame 290 is obtained by adding up the aforementioned three windowed samples from the windowed frames 240-(k-1), 240-(k-2) and 240-(k-3).
  • This case can, for instance, be caused by the windower 220 being adapted such that a corresponding output sample of an output frame is disregarded by the windower 220.
  • the overlap/adder 230 may be configured such that the corresponding windowed sample is not taken into consideration for adding up the respective windowed sample to obtain the added sample.
  • windowed samples in the initial section 270 may also be considered to be disregarded by the overlap/adder, as the corresponding windowed samples will not be used to obtain the added sample in the start section 300.
  • the overlap/adder 230 is adapted to adding up at least three windowed samples from at least three different windowed frames 240 (corresponding to three different output frames 240).
  • a windowed frame 240 in the embodiment shown in Fig. 4 comprises four subsections 260
  • an added sample in the remainder section 310 will be generated by the overlap/adder 230 by adding up four windowed samples from four different windowed frames 240.
  • an added sample in the remainder section 310 of the added frame 290 is obtained by the overlap/adder 230 by adding up the corresponding windowed sample from the first section 260-1 of the windowed frame 240-k, from the second subsection 260-2 of the windowed frame 240-(k-1) of the third subsection 260-3 from the windowed frame 240-(k-2) and from the fourth subsection 260-4 from the windowed frame 240-(k-3).
  • the sample advance value M is equal to the length of the added frame 290.
  • the length of an input frame is, as mentioned before, equal to the sample advance value M.
  • each of the output/windowed frames 240 comprises four starting sections 260-1 to 260-4.
  • an embodiment of the synthesis filterbank can easily be implemented in which an output or windowed frame only comprises one windowed sample more than twice the number of added samples of an added frame 290.
  • an embodiment of a synthesis filterbank 200 can be adapted such that each windowed frame only comprises 2M+1 windowed samples.
  • SBR Spectral Bank Replication
  • IMDCT Inverse Modified Discrete Cosine Transform
  • the frequency/time converter 210 can be implemented with a longer window function, such that the sample index n is now running up to 2N-1, rather than up to N-1.
  • n 0 ⁇ N 2 + 1 2 , wherein spec[i][k] is an input value corresponding to the spectral coefficient index k and the window index I of the input frame.
  • the parameter N is equal to 960 or 1024.
  • the parameter N can also acquire any value.
  • the windower 220 and the overlap/adder 230 may also be modified compared to the windowing and overlap/adds implemented in the framework of an ER AAC LD codec
  • the length N of a window function is replaced by a length 2N window function with more overlap in the past and less overlap in the future.
  • these window coefficients correspond to the initial sections 160, 270 of the respective frames. As previously explained, this section is not required to be implemented at all.
  • the corresponding modules may be constructed such that multiplying with a value zero is not required.
  • the windowed samples may be set to zero or disregarded, to mention only two possible implementation-related differences of embodiments.
  • a windowed frame does not necessarily comprise an initial section
  • the equations and expressions given above might, for instance, be altered in terms of the borders of the summing indices to exclude windowed samples of the initial section in the case an initial section is not present or comprises trivial windowed samples (e.g. zero-valued samples).
  • an ER AAC LD codec optionally with an appropriate SBR tool can be implemented to obtain an ER AAC ELD codec, which can, for instance, be used to achieve a low bitrate and/or low delay audio coding and decoding system.
  • An overview of an end coder and a decoder will be given in the framework of Figs. 12 and 13 , respectively.
  • both embodiments of an analysis filterbank 100 and of a synthesis filterbank 200 may offer the advantage of enabling an enhanced low delay coding mode by implementing a low delay window function in the framework of an analysis/synthesis filterbank 100, 200 as well as in the framework of embodiments of an encoder and decoder.
  • an embodiment of an analysis filterbank or a synthesis filterbank which may comprise one of the window functions, which will be described in more detail in the context of Figs. 5 to 11 , several advantages may be achieved depending on the concrete implementation of an embodiment of a filterbank comprising a low delay window function. Referring to the context of Fig.
  • the pre-echo behavior is similar to the low-overlap window, so that an embodiment of a synthesis filterbank and/or of an analysis filterbank can represent an excellent trade-off between quality and low delay depending on the concrete implementation of an embodiment of the filterbanks.
  • a further advantage which may, for instance, be employed in the framework of an embodiment of a conferencing system, is that only one window function can be used to process all kinds of signals.
  • Fig. 5 shows a graphical representation of a possible window function, which can, for instance, be employed in the framework of a windower 110, 220 in the case of an embodiment of an analysis filterbank 100 and in the case of a synthesis filterbank 200.
  • the lower graph of Fig. 5 shows the corresponding synthesis window function for an embodiment of a synthesis filterbank.
  • both window functions comprise a significant higher number of window coefficients in one half of the definition set with respect to the aforementioned midpoint having absolute values of the window coefficients, which are larger than 10%, 20%, 30% or 50% of the maximum absolute value of all window coefficients.
  • the analysis window function and the synthesis window function are in terms of the indices an inverse of each other.
  • the window function shown in the two graphs in Fig. 5 is that in the case of the analysis window shown in the upper graph, the last 120 windowing coefficients and in the case of the synthesis window function in the bottom graph in Fig. 5 , the first 120 window coefficients are set to zero or comprise an absolute value so that they can be considered to be equal to 0 within a reasonable accuracy.
  • the aforementioned 120 windowing coefficients of the two window functions can therefore be considered to cause an appropriate number of samples to be set to at least one value in a predetermined range by multiplying the 120 window coefficients with the respective samples.
  • the 120 zero-valued windowed coefficients will result in creating the initial section 160, 270 of the windowed frames in embodiments of an analysis filterbank and a synthesis filterbank, if applicable, as previously explained.
  • the 120 zero-valued window coefficients can be interpreted by the windower 110 by the time/frequency converter 120, by the windower 220 and by the overlap/adder 230 in embodiments of an analysis filterbank 100 and a synthesis filterbank 200 to treat or process the different frames accordingly, even in the case that the initial sections 160, 270 of the appropriate frames are not present at all.
  • appropriate embodiments of an analysis filterbank 100 and a synthesis filterbank 200 will be established in which the initial sections 160, 270 of the corresponding frames comprise M/4 samples or the corresponding first subsections 150-1, 260-1 comprise M/4 values or samples less than the other subsections, to put it in more general terms.
  • the analysis window function shown in the upper graph of Fig. 5 and the synthesis window function shown in the lower graph of Fig. 5 represents low-delay window functions for both an analysis filterbank and a synthesis filterbank.
  • both the analysis window function and the synthesis window function as shown in Fig. 5 are mirrored versions of each other with respect to the aforementioned midpoint of the definition set of which both window functions are defined.
  • window coefficients as well as lifting coefficients, which will be subsequently introduced
  • the Figs. given are not required to be implemented as precisely as given.
  • other window functions may be implemented, which are filter coefficients, window coefficients and other coefficients, such as lifting coefficients, which are different from the coefficients given below in the annex, as long as the variations are within the third digit following the comma or in higher digits, such as the fourth, fifth, etc. digits.
  • Fig. 6 shows a comparison of the window function as shown in Fig. 5 in the case of an analysis window function in the upper graph of Fig. 6 , and in the case of a synthesis window function in the lower graph of Fig. 6 .
  • two graphs also comprise the socalled sine window function, which is for instance, employed in the aforementioned ER AAC codecs AAC LC and AAC LD.
  • the direct comparison of the sine window and the low-delay window function as shown in the two graphs of Fig. 6 illustrate the different time objects of the time window as explained in the context of Fig. 5 .
  • the sine window frame function is symmetric about its respective midpoint of the shortened definition set and comprises in the first 120 elements of the definition set (mostly) window coefficients being larger than zero.
  • the low-delay window comprises 120 (ideally) zero-valued windowed coefficients and is significantly asymmetric with respect its respective midpoint of the prolonged definition set compared to the definition set of the sine window.
  • Fig. 7 comprises in three graphs, three different window functions.
  • the upper graph of Fig. 7 shows the aforementioned sine window
  • the middle graph shows the socalled low-overlap window
  • the bottom graph shows the low-delay window.
  • the sine window as well as the low-overlap window in the two topmost graphs in Fig. 7 are defined only over limited or shortened definition sets comprising 1024 sample indices as compared to the low delay window function as shown in the bottom graph of Fig. 7 , which is defined over 2048 sample indices.
  • the plots of the window shapes of a sine window, the low-overlap window and the low-delay window in Fig. 7 comprise more of less the same characteristics as previously discussed in terms of the sine window and the low delay window.
  • the sine window (top graph in Fig. 7 ) is once again symmetric with regard to the appropriate midpoint of the definition set lying between indices 511 and 512.
  • window coefficients may differ from the values given in table 4, as long as they hold the relations given in table 3 in the annex.
  • variations with respect to the window coefficients can easily be implemented, as long as the variations are within the third digit following the comma, or in higher digits such as the fourth, fifth, etc. digits, as previously explained.
  • the low-overlap window has not been described so far.
  • the low delay window also comprises a definition set comprising 1024 elements.
  • the low-overlap window also comprises at the beginning of a definition set and at the end of a definition set, a connected subset in which the low-overlap window vanishes.
  • a steep rise or decay follows, which comprises only a little over 100 sample indices each.
  • the symmetric low-overlap window does not comprise values larger than 1 and may comprise a lesser stop-band attenuation compared to window functions as employed in some embodiments.
  • the low-overlap window comprises a significant lower definition set while having the same sample advance value, as the low delay window and does not acquire values larger than one.
  • both the sine window and the low-overlap window are with respect to their respective midpoints of the definition sets orthogonal or symmetric, while the low-delay window is asymmetric in the described manner over the midpoint of its definition set.
  • the low overlap window was introduced in order to eliminate pre-echo artifacts for transients.
  • the lower overlap avoids spreading of the quantization noise before the signal attack, as illustrated in Fig. 8 .
  • the new low-delay window has the same property, but offers a better frequency response, as will be apparent by comparing the frequency responses shown in Figs. 10 and 11 . Therefore, the low delay window is capable of replacing both traditional AAC LD windows, i.e. the sign window at the low-overlap window, so that a dynamic window shape adaptation is not required to be implemented anymore.
  • Fig. 8 shows for the same window functions shown in Fig. 7 in the same order of graphs an example of quantization noise spreading for the different window shapes of the sine window or the low-overlap window and the low-delay window.
  • a low-delay window in an embodiment of a synthesis filterbank or an analysis filterbank, may result in an advantage concerning an improved pre-echo behavior.
  • an analysis window the path that accesses future input values and, thus would require a delay, are reduced by more than a sample and preferably by 120/128 samples in the case of a block length or sample advance value of 480/512 samples, such that it reduces the delay in comparison to the MDCT (Modified Discrete Cosine Transform).
  • MDCT Modified Discrete Cosine Transform
  • FIG. 9 shows a schematic sketch of the masking behavior of the human ear.
  • Fig. 9 shows a schematic representation of the hearing threshold level of the human ear, as a function of time, when a sound or a tone having a specific frequency is present during a period of time of approximately 200 ms.
  • a pre-masking is present for a short period of time of approximately 20 ms, therefore, enabling a smooth transition between no masking and the masking during the presence of the tone or sound, which is sometimes referred to as simultaneous masking.
  • the masking is on.
  • the tone or sound disappears, as indicated by the arrow 360 in Fig. 9 , the masking is not immediately lifted, but during a period of time or approximately 150 ms, the masking is slowly reduced, which is also sometimes referred to as post-masking.
  • Fig. 9 shows a general temporal masking property of human hearing, which comprises a phase of pre-masking as well as a phase of post-masking before and after a sound or tone being present. Due to the reduction of the pre-echo behavior by incorporating a low-delay window in an embodiment of an analysis filterbank 100 and/or a synthesis filterbank 200, audible distortions will be severely limited in many cases as the audible pre-echoes will, at least to some extent, fall into the pre-masking period of the temporal masking effect of the human ear as shown in Fig. 9 .
  • a low-delay window function as illustrated in Figs. 5 to 7 , described in more detail with respect to relations and values in tables 1 to 4 in the annex, offers a frequency response, which is similar to that of a sine window.
  • Fig. 10 shows a comparison of the frequency response between the sine window (dashed line) and an example of a low-delay window (solid line).
  • the low-delay window is comparable in terms of the frequency selectivity to the sine window.
  • the frequency response of the low-delay window is similar or comparable to the frequency response of the sine window, and much better than the frequency response of the low-overlap window, as in comparison with the frequency responses shown in Fig. 11 illustrate.
  • Fig. 11 shows a comparison of the frequency responses between the sine window (dashed line) and the low-overlap window (solid line).
  • the solid line of the frequency response of the low-overlap window is significantly larger than the corresponding frequency response of the sine window.
  • the low-delay window and the sine window show comparable frequency responses, which can be seen by comparing the two frequency responses shown in Fig. 10
  • a comparison between the low-overlap window and the low-delay window can easily be drawn, as the plot shown in Figs. 10 and 11 both show the frequency response of the sine window and comprise the same scales with respect to the frequency axis and the intensity axis (db). Accordingly, it can easily be concluded that the sine window which can easily implemented in an embodiment of a synthesis filterbank as well as in an embodiment of an analysis filterbank offers compared to the low-overlap window a significantly better frequency response.
  • the low-delay window offers a considerable advantage compared to pre-echo behavior, while the pre-echo behavior of the low-delay window is comparable to that of a low-overlap window, the low-delay window represents an excellent tradeoff between the two aforementioned windows.
  • the low-delay window which can be implemented in the framework of an embodiment of an analysis filterbank as well as an embodiment of a synthesis filterbank and related embodiments, due to this trade-off, the same window function can be used for transient signals, as well as tonal signals, so that no switching between different block lengths or between different windows is necessary.
  • embodiments of an analysis filterbank, a synthesis filterbank and related embodiments offer the possibility of building an encoder, a decoder and further systems that do not require switching between different sets of operational parameters such as different block sizes, or block lengths, or different windows or window shapes.
  • a longer overlap is introduced without creating an additional delay.
  • a window of about twice the length of the corresponding sine window with twice the amount of overlap and according benefits of the frequency selectivity as outlined before an implementation can be obtained with only minor additional complexity, due to a possible increase size of block length multiplications and memory elements.
  • further details on such an implementation will be explained in the context of Figs. 19 to 24 .
  • Fig. 12 shows a schematic block diagram of an embodiment of an encoder 400.
  • the encoder 400 comprises an embodiment of an analysis filterbank 100 and, as an optional component, an entropy encoder 410, which is configured to encoding the plurality of output frames provided by the analysis filterbank 100 and configured to outputting a plurality of encoded frames based on the output frames.
  • the entropy encoder 410 may be implemented as a Huffman encoder or another entropy encoder utilizing an entropy-efficient coding scheme, such as the arithmetic coding-scheme.
  • the encoder offers an output of the number of bands N while having a reconstructional delay of less than 2N or 2N-1.
  • an in principle an embodiment of an encoder also represents a filter, an embodiment of an encoder 400 offers a finite impulse response of more than 2N samples. That is, an embodiment of an encoder 400 represents an encoder which is capable of processing (audio) data in a delay-efficient way.
  • such an embodiment may also comprise a quantizer, filter or further components to pre-process the input frames provided to the embodiment of the analysis filterbank 100 or to process the output frames before entropy encoding the respective frames.
  • a quantizer can be provided to an embodiment of an encoder 400 before the analysis filterbank 100 to quantize the data or to requantize the data, depending on the concrete implementation and field of application.
  • an equalization or another gain adjustment in terms of the output frames in the frequency-domain can be implemented.
  • Fig. 13 shows an embodiment of a decoder 450 comprising an entropy decoder 460 as well as an embodiment of a synthesis filterbank 200, as previously described.
  • the entropy decoder 460 of the embodiment of the decoder 450 represents an optional component, which can, for instance, be configured for decoding a plurality of encoded frames, which might, for instance, be provided by an embodiment of an encoder 400. Accordingly, the entropy decoder 460 might by a Huffman or algorithmic decoder or another entropy decoder based on an entropy-encoding/decoding scheme, which is suitable for the application of the decoder 450 at hand.
  • the entropy decoder 460 can be configured to provide a plurality of input frames to the synthesis filterbank 200, which, in turn, provides a plurality of added frames at an output of the synthesis filterbank 200 or at an output of the decoder 450.
  • the decoder 450 may also comprise additional components, such as a dequantizer or other components such as a gain adjuster.
  • a gain adjuster can be implemented as an optional component to allow a gain adjustment or equalization in the frequency-domain before the audio data will be transferred by the synthesis filterbank 200 into the time-domain.
  • an additional quantizer may be implemented in a decoder 450 after the synthesis filterbank 200 to offer the opportunity of requantizing the added frames prior to providing the optionally requantized added frames to an external component of the decoder 450.
  • Embodiments of an encoder 400 as shown in Fig. 12 and embodiments of a decoder 450 as shown in Fig. 13 can be applied in many fields of audio encoding/decoding as well as audio processing. Such embodiments of an encoder 400 and a decoder 450 can, for instance, be employed in the field of high-quality communications.
  • an embodiment of an encoder or coder as well as an embodiment for a decoder offer the opportunity of operating the said embodiment without having to implement a change of parameter such as switching the block length or switching between different windows.
  • an embodiment of the present invention in the form of a synthesis filterbank, an analysis filterbank and related embodiments is by far not required to implement different block lengths and/or different window functions.
  • a low-delay AAC coder (AAC LD) has, over time, increasing adaptation as a full-bandwidth high-quality communications coder, which is not subjected to limitations that usual speech coders have, such as focusing on single-speakers, speech material, bad performance for music signals, and so on.
  • This particular codec is widely used for video/teleconferencing in other communication applications, which, for instance, have triggered the creation of a low-delay AAC profile due to industry demand.
  • an enhancement of the coders' coding efficiency is of wide interest to the user community and is the topic of the contribution, which some embodiments of the present invention are capable of providing.
  • the enhanced AAC ELD coder or AAC EL decoder comprising embodiments of low-delay filterbanks, exhibit a delay comparable to that of a plane AAC LD coder, but is capable of saving a significant amount of the bitrate at the same level of quality, depending on the concrete implementation.
  • an AAC ELD coder may be capable of saving up to 25% or even up to 33% of the bitrate at the same level of quality compared to an AAC LD coder.
  • Embodiments of a synthesis filterbank or an analysis filterbank can be implemented in a socalled enhanced low-delay AAC codec (AAC ELD), which is capable of extending the range of operation down to 24 kbit/s per channel, depending on the concrete implementation and application specification.
  • AAC ELD enhanced low-delay AAC codec
  • embodiments of the present invention can be implemented in the framework of a coding as an extension of the AAC LD scheme utilizing optionally additional coding tools.
  • Such an optional coding tool is the spectral band replication (SBR) tool, which can be integrated or additionally be employed in the framework of both an embodiment of an encoder as well as an embodiment of a decoder.
  • SBR spectral band replication
  • SBR is an attractive enhancement, as it enables an implementation of a dual rate coder, at which the sampling frequency for a lower part of the frequency spectrum is encoded with only half of the sampling frequency of the original sampler.
  • SBR is capable of encoding a higher spectral range of frequencies based on the lower part, such that the overall sampling frequency can, in principle, be reduced by a factor of 2.
  • the delay saved may, in principle, reduce the overall delay of the system by a factor of 2 of the saved delay.
  • an AAC ELD coder may exhibit the delay well within the acceptable range for bi-directional communication, while saving of up to 25% to 33% of the rate compared to a regular AAC LD coder, while maintaining the level of audio quality.
  • the present application describes a description of possible technical modifications along with an evaluation of an achievable coder performance, at least in terms of some of the embodiments of the present invention.
  • a low-delay filterbank is capable of achieving a substantial delay reduction by utilizing a different window function, as previously explained, with multiple overlaps instead of employing a MDCT or IMDCT, while at the same time offering the possibility of perfect reconstruction, depending on the concrete implementation.
  • An embodiment of such a low-delay filterbank is capable of reducing the reconstruction delay without reducing the filter length, but still maintaining the perfect reconstruction property under some circumstances in the case of some embodiments.
  • the resulting filterbanks have the same cosine modulation function as a traditional MDCT, but can have longer window functions, which can be non-symmetric or asymmetric with a generalized or low reconstruction delay.
  • an embodiment of the filterbank may be capable of reducing the delay of 2M to (2M - M/2) samples by implementing M/4 zero-valued window coefficients or by adapting the appropriate components, as previously explained, accordingly such that the first subsections 150-1, 260-1 of the corresponding frames comprise M/4 samples less than the other subsections.
  • a dual rate system is used to achieve a higher coding gain compared to a single rate system, as explained earlier on.
  • a more energy efficient encoding as possible having lesser frequency bands will be provided by the corresponding coder, which leads to a bitwise reduction due to some extent, removing redundant information from the frames provided by the coder.
  • an embodiment of a low-delay filterbank as previously described is used in the framework of the AAC LD core coder to arrive at an overall delay that is acceptable for communication applications. In other words, in the following, the delay will be described in terms of both the AAC LD core and the AAC ELD core coder.
  • a delay reduction can be achieved by implementing a modified MDCT window/filterbank.
  • Substantial delay reduction is achieved by utilizing the aforementioned and described different window functions with multiple overlap to extend the MDCT and the IMDCT to obtain a low-delay filterbank.
  • the technique of low-delay filterbanks allows utilizing a non-orthogonal window with multiple overlap. In this way, it is possible to obtain a delay, which is lower than the window length. Hence, a low delay with a still long impulse response resulting in good frequency selectivity can be achieved.
  • an embodiment of an encoder and an embodiment of a decoder 450 may under certain circumstances be capable of producing a good audio quality at a very small bit range. While the aforementioned ER AAC LD codec produces good audio quality as a bit range of 64 kb/sec to 48 kb/sec per channel, the embodiments of the encoder 400 and the decoder 450, as described in the present document, can be capable of providing an audio coder and decoder, which is under some circumstances able to produce at an equal audio quality at even lower bitrates of about 32 kb/sec per channel. Moreover, embodiments of an encoder and decoder have an algorithmic delay small enough to be utilized for two-way communication systems, which can be implemented in existing technology by using only minimum modifications.
  • Embodiments of the present invention achieve this by combining existing MPEG-4 audio technology with a minimum number adaptation necessary for low-delay operations necessary for low-delay operation to arrive at embodiments of the present invention.
  • the MPEG-4 ER AAC low-delay coder can be combined with a MPEG-4 spectral band replication (SPR) tool to implement embodiments of an encoder 400 and a decoder 450 by considering the described modifications.
  • SPR spectral band replication
  • the resulting increase in algorithmic delay is alleviated by minor modifications of the SPR tool, which will not be described in the present application, and the use of an embodiment of a low-delay core coder filterbank and an embodiment of an analysis filterbank or a synthesis filterbank.
  • such an enhanced AAC LD coder is capable of saving up to 33% of the bitrate at the same level of quality compared to a plain ACC LD coder while retaining low enough delay for a two-way communication application.
  • a coding system comprising a SBR tool is described.
  • all components of a coding system 500 shown in Fig. 14a are analyzed with respect to their contribution to the overall system delay.
  • Fig. 14a gives a detailed overview of the complete system, wherein Fig. 14b puts emphasis on the sources of delay.
  • the system shown in Fig. 14a comprises an encoder 500, which, in turn, comprises an MDCT time/frequency converter, operates in the dual rate approach as a dual rate coder.
  • the encoder 500 also comprises a QMF-analysis filterbank 520, which is part of the SBR tool.
  • both the MDCT converter 510 as well as the QMF-analysis filterbank 520 is provided with the same input data.
  • the QMF-analysis filterbank 520 provides the SBR data. Both data are combined into a bit stream and provided to a decoder 530.
  • the decoder 530 comprises an IMDCT frequency/time converter 540, which is capable of decoding the bit stream to obtain, at least in terms of the low band parts, a time-domain signal, which will be provided to an output of the decoder via a delayer 550.
  • an output of the IMDCT converter 540 is coupled to a further QMF-analysis filterbank 560, which is part of a SBR tool of the decoder 530.
  • the SBR tool comprises a HF generator 570, which is coupled to an output of the QMF-analysis filterbank 560 and capable of generating the higher frequency components based on the SBR data of the QMF-analysis filterbank 520 of the encoder 500.
  • An output of the HF generator 570 is coupled to a QMF-synthesis filterbank 580, which transforms the signals in the QMF-domain back into the time domain in which the delayed low band signals are combined with the high band signals, as provided by the SBR tool of the decoder 530. The resulting data will then be provided as the output data of thee decoder 530.
  • Fig. 14b emphasizes the delay sources of the system shown in Fig. 14a .
  • Fig. 14b illustrates the delay sources of the MPEG-4 ER AAC LD system comprising a SBR tool.
  • the appropriate coder of this audio system utilizes a MDCT/IMDCT filterbank for a time/frequency/time transformation or conversion with a frame size of 512 or 480 samples.
  • the results in reconstruction delays, therefore, which are equal to 1024 are 960 samples, depending on the concrete implementation.
  • the delay value has to be doubled due to the sampling rate conversion.
  • FIG. 15 comprises a table, which gives an overview of the delay produced by the different components assuming a sampling rate of 48 kHz and the core coder frame size of 480 samples, wherein the core coder effectively runs at a sampling rate of 24 kHz due to the dual rate approach.
  • Fig. 15 shows that in the case of an AAC LD codec along with a SBR tool, an overall algorithmic delay of 16 ms would result, which is substantially higher than what is permissible for telecommunication applications.
  • This evaluation comprises the standard combination of the AAC LD coder along with the SBR tool, which includes the delay contributions from the MDCT/IMDCT dual rate components, the QMF components and the SBR overlap components.
  • an overall delay of only 42 ms is achievable, which includes the delay contributions from the embodiments of the low-delay filterbanks in the dual rate mode (ELD MDCT + IMDCT) and the QMF components.
  • the algorithmic delay of the AAC LD core can be described as being 2M samples, wherein, once again, M is the basic frame length of the core coder.
  • the low-delay filterbank reduces the number of samples by M/2 due to introducing the initial sections 160, 270 or by introducing an appropriate number of zero-valued or other values in the framework of the appropriate window functions.
  • the delay is doubled due to the sampling rate conversion of a dual rate system.
  • the QMF components comprise a filterbank's reconstruction delay of 640 samples.
  • the SBR HF reconstruction causes an additional delay with a standard SBR tool of 6 QMF slots due to the variable time grid. Accordingly, the delay is in the standard SBR, six times 64 samples of 384 samples.
  • a delay saving of 18 ms can be achieved by not implementing a straightforward combination of a AAC LD coder along with a SBR tool having an overall delay of 60 ms, but an overall delay of 42 ms is achievable.
  • the overlap delay which is a second important aspect in terms of delay optimization, can be significantly reduced by introducing an embodiment of a synthesis filterbank or an analysis filterbank to achieve a low bitrate and a low-delay audio coding system.
  • Embodiments of the present invention can be implemented in many fields of application, such as conferencing systems and other bi-directional communication systems.
  • this codec such as teleconferencing, employ a sampling rate of 32 kHz and, thus, work with a delay of 30 ms.
  • the delay requirements of modern ITU telecommunication codec allow delay of, roughly speaking, 40 ms.
  • Different examples include the recent G.722.1 annex C coder with an algorithmic delay of 40 ms and the G.729.1 coder with an algorithmic delay of 48 ms.
  • the overall delay achieved by an enhanced AAC LD coder or AAC ELD coder comprising an embodiment of a low-delay filterbank can be operated to fully lie within the delay range of common telecommunication coders.
  • Fig. 16 shows a block diagram of an embodiment of a mixer 600 for mixing a plurality of input frames, wherein each input frame is a spectral representation of a corresponding time-domain frame being provided from a different source.
  • each input frame for the mixer 600 can be provided by an embodiment of an encoder 400 or another appropriate system or component.
  • the mixer 600 is adapted to receive input frames from three different sources. However, this does not represent any limitation.
  • an embodiment of a mixer 600 can be adapted or configured to process and receive an arbitrary number of input frames, each input frame provided by a different source, such as a different encoder 400.
  • the embodiment of the mixer 600 shown in Fig. 16 comprises an entropy decoder 610, which is capable of entropy decoding the plurality of input frames provided by the different sources.
  • the entropy decoder 610 can for instance be implemented as a Huffman entropy decoder or as an entropy decoder employing another entropy decoding algorithm such as the socalled Arithmetic Coding, Unary Coding, Elias Gamma Coding, Fibonacci Coding, Golomb Coding or Rice Coding.
  • the entropy decoded input frames are then provided to an optional dequantizer 620, which can be adapted such that the entropy decoded input frames can be dequantized to accommodate for application-specific circumstances, such as the loudness characteristic of the human ear.
  • the entropy decoded and optionally dequantized input frames are then provided to a scaler 630, which is capable of scaling the plurality of entropy frames in the frequency domain.
  • the scaler 630 can for instance, scale each of the optionally dequantized and entropy decoded input frames by multiplying each of the values by a constant factor 1/P, wherein P is an integer indicating the number of different sources or encoders 400.
  • the scaler 630 is in this case capable of scaling down the frames provided by the dequantizer 620 or the entropy decoder 610 to scale them down to prevent the corresponding signals from becoming too large in order to prevent an overflow or another computational error, or to prevent audible distortions like clipping.
  • Different implementations of the scaler 630 can also be implemented, such as a scaler which is capable of scaling the provided frame in an energy conserving manner, by for instance, evaluating the energy of each of the input frames, depending on one or more spectral frequency bands. In such a case, in each of these spectral frequency bands, the corresponding values in the frequency domain can be multiplied with a constant factor, such that the overall energy with respect to all frequency ranges is identical.
  • the scaler 630 may also be adapted such that the energy of each of the spectral subgroups is identical with respect to all input frames of all different sources, or that the overall energy of each of the input frames is constant.
  • the scaler 630 is then coupled to an adder 640,which is capable of adding up the frames provided by the scaler, which are also referred to as scaled frames in the frequency domain to generate an added frame also in the frequency domain. This can for instance be accomplished by adding up all values corresponding to the same sample index from all scaled frames provided by the scaler 630.
  • the adder 640 is capable of adding up the frames provided by the scaler 6340 in the frequency domain to obtain an added frame, which comprises the information of all sources as provided by the scaler 630.
  • an embodiment of a mixer 600 may also comprise a quantizer 650 to which the added frame of the adder 640 may be provided to.
  • the optional quantizer 650 can for instance be used to adapt the added frame to fulfill some conditions.
  • the quantizer 650 may be adapted such that the tact of the dequantizer 620 may be reversed.
  • the quantizer 650 may then be adapted to provide these special requirements of conditions to the added frame.
  • the quantizer 650 may for instance be adapted to accommodate for the characteristics of the human ear.
  • the embodiment of the mixer 600 may further comprise an entropy encoder 660, which is capable of entropy encoding the optionally quantized added frame and to provide a mixed frame to one or more receivers, for instance, comprising an embodiment of an encoder 450.
  • the entropy encoder 660 may be adapted to entropy encoding the added frame based on the Huffman algorithm or another of the aforementioned algorithms.
  • a mixer By employing an embodiment of an analysis filterbank, a synthesis filterbank or another related embodiment in the framework of an encoder and a decoder, a mixer can be established and implemented which is capable of mixing signals in the frequency-domain.
  • a mixer can be implemented, which is capable of directly mixing a plurality of input frames in the frequency domain, without having to transform the respective input frames into the time-domain to accommodate for the possible switching of parameters, which are implemented in state-of-the-art-codecs for speech communications.
  • these embodiments enable an operation without switching parameters, like switching the block lengths or switching between different windows.
  • Fig. 17 shows an embodiment of a conferencing system 700 in the form of a MCU (Media Control Unit), which, can for instance be implemented in the framework of a server.
  • the conferencing system 700 or MCU 700 comprises for a plurality of bit streams, of which in Fig. 17 , two are shown.
  • a combined entropy decoder and dequantizer 610, 620 as well as a combined unit 630, 640 which are labeled in Fig. 17 as "mixer".
  • the output of the combined unit 630, 640 is provided to the combined unit comprising a quantizer 650 and the entropy encoder 660, which provides as the mixed frames an outgoing bit stream.
  • Fig. 17 shows an embodiment of a conferencing system 700 which is capable of mixing a plurality of incoming bit streams in the frequency domain, as the incoming bit stream as well as the outgoing bit streams have been created using a low-delay window on the encoder side, whereas the outgoing bit streams are intended and capable of being processed, based on the same low-delay window on the decoder side.
  • the MCU 700 shown in Fig. 17 is based on the use of one universal low-delay window only.
  • An embodiment of a mixer 600 as well as an embodiment of a conferencing system 700 is therefore suitable to be applied in the framework of embodiments of the present invention in the form of an analysis filterbank, a synthesis filterbank and the other related embodiments.
  • a technical application of an embodiment of a low-delay codec with only one window allows a mixing in the frequency-domain. For instance, in (tele-) conferencing scenarios with more than two participants or sources, it might often be desirable to receive several codec signals, mix them up to one signal and further transmit the resulting encoded signal.
  • the implementational method can be reduced compared to a straightforward manner of decoding the incoming signals, mixing the decoded signals in the time-domain and re-encoding the mixed signal again into the frequency-domain.
  • the implementation of such a straightforward mixer in the form of a MCU is shown in Fig. 18 as a conferencing system 750.
  • the conferencing system 750 also comprises a combined module 760 for each of the incoming bit streams operating in the frequency domain and capable of entropy decoding and dequantization of the incoming bit streams.
  • the modules 760 are coupled to the IMDCT converter 770 each, of which one is operating in the sine window mode of operation, whereas the other one is currently operating in the low-overlap window mode of operation.
  • the two IMDCT converters 770 transform the incoming bit streams from the frequency-domain into the time-domain, which is necessary in the case of a conferencing system 750 as the incoming bit streams are based on an encoder, which uses both, the sine window and the low-overlap window, depending on the audio signal to encode the respective signals.
  • the conferencing system 750 furthermore comprises a mixer 780, which mixes in the time-domain the two incoming signals from the two IMDCT converters 770 and provides a mixed time-domain signal to a MDCT converter 790, which transfers the signal from the time-domain into the frequency-domain.
  • a mixer 780 which mixes in the time-domain the two incoming signals from the two IMDCT converters 770 and provides a mixed time-domain signal to a MDCT converter 790, which transfers the signal from the time-domain into the frequency-domain.
  • the mixed signal in the frequency domain as provided by the MDCT 790 is then provided to a combined module 795, which is then capable of quantizing an entropy encoding the signal to form the outgoing bit stream.
  • the approach according to the conferencing system 750 has two disadvantages. Due to the complete decoding and encoding done by the two IMDCT converters 770 and the MDCT 790, the high computational cost is to be paid by implementing the conferencing system 750. Moreover, due to the introduction of the decoding and encoding, an additional delay is introduced which can be high under certain circumstances.
  • Fig. 19 shows an embodiment of an efficient implementation of a low-delay filterbank.
  • a synthesis filterbank 800 will be described in more detail, which can for instance be implemented in an embodiment of a decoder.
  • the embodiment of a low-delay analysis filterbank 800 hence, symbolizes a reverse of an embodiment of a synthesis filterbank or an encoder.
  • the synthesis filterbank 800 comprises an inverse type-iv discrete cosine transform frequency/time converter 810, which is capable of providing a plurality of output frames to a combined module 820 comprising a windower and an overlap/adder.
  • the time/frequency 810 is an inverse type-iv discrete cosine transform converter, which is provided with an input frame comprising M ordered input values y k (0) , ..., y k (M-1), wherein M is once again a positive integer and wherein k is an integer indicating a frame index.
  • the time/frequency converter 810 provides 2M ordered output samples x k (0),...,x k (2M-1) based on the input values and provides these output samples to the module 820 which in turn comprises the windower and the overlap/adder mentioned before.
  • the embodiment of the computationally efficient implementation of a low-delay filterbank 800 comprises in the framework of the lifter 830, a plurality of combined delayers and multipliers 840 as well as a plurality of adders 850 to carry out the aforementioned calculations in the framework of the lifter 830.
  • an embodiment of a low-delay filterbank 800 can be implemented as sufficiently as a regular MDCT converter.
  • the general structure of such an embodiment is illustrated in Fig. 19 .
  • the inverse DCT-IV and the inverse windowing-overlap/add are performed in the same way as the traditional windows, however, employing the aforementioned windowing coefficients, depending on the concrete implementation of the embodiment.
  • M/4 window coefficients are zero-valued windowed coefficients, which thus do not, in principle, involve any operation.
  • M additional multiplier-add operations are required, as can be seen in the framework of the lifter 830. These additional operations are sometimes also referred to as "zero-delay matrices". Sometimes these operations are also known as "lifting steps".
  • the efficient implementation shown in Fig. 19 may under some circumstances be more efficient as a straightforward implementation of a synthesis filterbank 200. To be more precise, depending on the concrete implementation, such a more efficient implementation might result in saving M operations, as in the case of a straightforward implementation for M operations, it might be advisable to implement, as the implementation shown in Fig. 19 , requires in principle, 2M operations in the framework of the module 820 and M operations in the framework of the lifter 830.
  • the table in Fig. 20 comprises an estimate of the resulting overall number of operations in the case of an (modified) IMDCT converter along with a windowing in the case of a low-delay window function. The overall number of operations is 9600.
  • the arithmetic complexity of this IMDCT converter along with the windowing for the sine window is 9216 operations, which is of the same order of magnitude as the resulting overall number of operations in the case of the embodiment of the synthesis filterbank 800 shown in Fig. 19 .
  • Fig. 22 comprises a table for an AAC LC codec, which is also known as the advance audio codec with low complexity.
  • the complexity of the core coder comprising an embodiment of an enhanced low-delay filterbank is essentially comparable to that of a core coder, using a regular MDCT-IMDCT filterbank.
  • the number of operations is roughly speaking half the number of operations of an AAC LC codec.
  • Fig. 23 comprises two tables, wherein Fig. 23a comprises a comparison of the memory requirements of different codecs, whereas Fig. 23b comprises the same estimate with respect to the ROM requirement.
  • the tables in both Figs. 23a and 23b each comprise for the aforementioned codecs AAC LD, AAC ELD and AAC LC information concerning the frame length, the working buffer and concerning the state buffer in terms of the RAM-requirement ( Fig. 23a ) and information concerning the frame length, the number of window coefficients and the sum, in terms of the ROM-memory requirements ( Fig. 23b ).
  • AAC LD codec
  • AAC ELD AAC ELD
  • AAC LC information concerning the frame length
  • the working buffer and concerning the state buffer in terms of the RAM-requirement
  • Fig. 23b information concerning the frame length, the number of window coefficients and the sum, in terms of the ROM-memory requirements
  • the abbreviation AAC, ELD refer to an embodiment of a synthesis filterbank, analysis filterbank, encoder, decoder or a later embodiment.
  • the described efficient implementation according to Fig. 19 of an embodiment of the low-delay filterbank requires an additional state memory of length M and M additional coefficients, the lifting coefficients l(0),...,l(M-1).
  • a frame length of the AAC LD is half the frame length of the AAC LC
  • the resulting memory requirement is in the range of that of the AAC LC.
  • Fig. 24 comprises a list of used codecs for a MUSHRA test used in the framework of a performance assessment.
  • the abbreviation AOT stands for Audio Object Type, wherein the entry "X” stands for the audio object tape ER AAC ELD which can also be set to 39.
  • the AOT, X or AOT 39 identifies an embodiment of a synthesis filterbank or an analysis filterbank.
  • the abbreviation AOT stands in this context for "audio object type”.
  • the AAC ELD decoder at 32 kbit/s per channel performs significantly better than the original AAC L decoder at 32 kb/s. Moreover, the AAC ELD decoder at 32 kb/s per channel performs statistically indistinguishable from the original AAC LD decoder at 48 kb/s per channel.
  • binding AAC LD and the low-delay filterbank performs statistically indistinguishable from an original AAC LD coder both running at 48 kb/s. This confirms the appropriateness of a low-delay filterbank.
  • the overall coder performance remains comparable, while a significant saving in codec delay is achieved. Moreover, it was possible to retain the coder pressure performance.
  • AAC ELD AAC ELD codec
  • an enhanced AAC ELD decoder which may optionally be combined with a spectral band replication (SBR) tool.
  • SBR spectral band replication
  • minor modifications in terms of a real, live implementation may become necessary in the SBR tool and the core coder modules.
  • the performance of the resulting enhanced low-delay audio decoding based on the aforementioned technology is significantly increased, compared to what is currently delivered by the MPEG-4 audio standard. Complexity of the core coding scheme remains, however, essentially identical.
  • embodiments of the present invention comprise an analysis filterbank or synthesis filterbank including a low-delay analysis window or a low-delay synthesis filter.
  • an embodiment of a method of analyzing a signal or synthesizing a signal having a low-delay analysis filtering step or a low-delay synthesis filtering step Embodiments of a low-delay analysis filter or low-delay synthesis filter are also described.
  • computer programs having a program code for implementing one of the above methods when running on a computer are disclosed.
  • An embodiment of the present invention comprises also an encoder having a low delay analysis filter, or decoder having a low delay synthesis filter, or one of the corresponding methods.
  • embodiments of the inventive methods can be implemented in hardware, or in software.
  • the implementation can be performed using a digital storage medium, in particular, a disc a CD, or a DVD having electronically readable control signals stored thereon, which cooperate with the programmable computer or a processor such that an embodiment of the inventive methods is performed.
  • an embodiment of the present invention is, therefore, a computer program product with program code stored on a machine-readable carrier, the program code being operative for performing an embodiment of the inventive methods when the computer program product runs on the computer or processor.
  • embodiments of the inventive methods are therefore, a computer program having a program code for performing at least one of the embodiments of the inventive methods, when the computer program runs of the computer or processor.
  • processors comprise CPUs (Central Processing Unit), ASICs (Application Specific Integrated Circuits) or further integrated circuits (IC).
  • an analysis filterbank for filtering a plurality of time-domain input frames, wherein an input frame comprises a number of ordered input samples.
  • the analysis filterbank comprises a windower which is configured to generating a plurality of windowed frames, a windowed frame comprising a plurality of windowed samples, wherein the windower is configured to processing the plurality of input frames in an overlapping manner using a sample advance value, wherein the sample advance value is less than the number of ordered input samples of an input frame divided by 2; and a time/frequency converter which is configured to providing an output frame comprising a number of output values, an output frame being a spectral representation of a windowed frame.
  • the windower may be configured such that N is equal to 960 and the window coefficients w(0) to w(2N-1) obey the relations given in table 1 in the annex.
  • the analysis filterbank may further be configured such that the windower is configured such that the window coefficients w(0) to w(2N-1) comprise the values given in table 2 in the annex.
  • N may be equal to 1024 and the window coefficients w(0) to w(2N-1) may obey the relations given in table 3 in the annex.
  • the window coefficients w(0) to w(2N-1) may even comprise the values as given in table 4 in the annex.
  • the windower may be configured to generating a windowed frame based on an input frame by weighing at least a plurality of input samples of the input frame with a window function
  • the analysis filterbank may further be configured such that the windower is configured such that the window function attributes real-valued window coefficients to a definition set.
  • the definition set may comprise at least a number of elements being greater than or equal to the difference between the number of the ordered input samples of an input frame and the number of input samples to be disregarded or the number of windowed samples of a windowed frame set to the predetermined value or set to at least a value in the predetermined range by the windower or greater than or equal to the number of ordered input samples.
  • the windower may be configured to generating a windowed frame based on an input frame by weighing at least a plurality of input samples of the input frame with a window function, and the analysis filterbank may further be configured such that the windower is configured such that the window function is asymmetric over the definition set with respect to a midpoint of the definition set.
  • the window function may comprise more window coefficients with an absolute value of more than 10% of a maximum absolute value of the window coefficients of the window function in a first half of the definition set than in a second half of the definition set with respect to the midpoint of the definition set, wherein the first half corresponds to the latest half of the input samples.
  • the windower may be configured such that the predetermined value is 0.
  • the analysis filterbank may further be configured such that the windower is configured to disregarding or setting to the predetermined value or to a value in the predetermined range 128 or 120 windowed samples.
  • the analysis filterbank may be comprised in an encoder.
  • the encoder may further comprise an entropy encoder, wherein the entropy encoder is configured to encoding the plurality of output frames provided by the analysis filterbank, and wherein the entropy encoder is configured to outputting a plurality of encoded frames based on the output frames
  • a synthesis filterbank for filtering a plurality of input frames, wherein each input frame comprises a number of ordered input values, comprises a frequency/time converter configured to providing a plurality of output frames, an output frame comprising a number of ordered output samples, an output frame being a time representation of an input frame; a windower configured to generating a plurality of windowed frames, a windowed frame comprising a plurality of windowed samples; and wherein the windower is configured to providing the plurality of windowed samples for a processing in an overlapping manner based on a sample advance value; an overlap/adder configured to providing an added frame comprising a start section and a remainder section, an added frame comprising a plurality of added samples by adding at least three windowed samples from at least three windowed frames for an added sample in the remainder section of an added frame and by adding at least two windowed samples from at least two different windowed frames for an added sample in the start section, wherein the number of
  • the synthesis filterbank may further be configured such that the overlap/adder is configured such that an added sample in the remainder section of an added frame corresponds to output samples which are not disregarded, windowed samples set to the predetermined value or set to a value in the predetermined range by the windower, and wherein an added sample in the start section of an added frame corresponds to an output sample which is disregarded or to a windowed sample set to the predetermined value or set to a value in the predetermined range by the windower.
  • the synthesis filterbank may further be configured such that the frequency/time converter is configured to providing output frames comprising more than twice the number of output samples compared to the number of input values of an input frame.
  • the synthesis filterbank may further be configured such that the frequency/time converter is configured to providing output frames comprising a number of output samples, which is equal to a number of input values of an input frame multiplied by an integer number greater than 2.
  • the synthesis filterbank may further be configured such that the frequency/time converter is configured to providing an output frame comprising a number of output samples, which is equal to the number of input values of an input frame multiplied by 4.
  • the synthesis filterbank may further be configured such that the frequency/time converter is based on at least one of a discrete cosine transform and a discrete sine transform.
  • the synthesis filterbank may further be configured such that the frequency/time converter is configured such that N is equal to 960 or 1024.
  • the synthesis filterbank may further be configured such that the windower is configured to disregarding a plurality of output samples of an output frame or setting a plurality of windowed samples to the predetermined value or to at least a value in the predetermined range,
  • the synthesis filterbank may further be configured such that the windower is configured such that plurality of disregarded output samples comprises a connected subset of output samples comprising the earliest output sample according to the order of the ordered output samples, or wherein the plurality of windowed samples, which are set to the predetermined value or to at least a value in the predetermined range, comprises a connected subset of windowed samples comprising at least a windowed sample corresponding to the earliest output sample.
  • the synthesis filterbank may further be configured such that the windower is configured to generating a windowed frame based on an output frame and a weighing function by weighing at least an output sample of the output frame based on the weighing function.
  • the synthesis filterbank may further be configured such that the windower is configured to generating a windowed frame based on an output frame by multiplying an output sample of the output frame with a value based on a window function.
  • the synthesis filterbank may further be configured such that the windower is configured to multiplying at least a plurality of output samples of the output frame with an output sample-specific windowing coefficient of a window function.
  • the synthesis filterbank may further be configured such that the windower is configured to multiplying each output sample of the output frame with an output sample-specific windowing coefficient of the window function.
  • the synthesis filterbank may further be configured such that the windower is configured such that N is equal to 960 and the window coefficients w(0) to w(2N-1) obey the relations given in table 1 in the annex.
  • the synthesis filterbank may further be configured such that the windower is configured such that the window coefficients w(0) to w(2N-1) comprise the values given in table 2 in the annex.
  • the synthesis filterbank may further be configured such that the windower is configured such that N is equal to 1024 and the window coefficients w(0) to w(2N-1) obey the relations given in table 3 in the annex.
  • the synthesis filterbank may further be configured such that the windower is configured such that the window coefficients w(0) to w(2N-1) comprise the values given in table 4 of the annex.
  • the synthesis filterbank may further be configured such that the windower is configured such that the window function attributes real-valued window coefficients to elements of a definition set.
  • the synthesis filterbank may further be configured such that the windower is configured such that the window function is asymmetric over the definition set with respect to a midpoint of a definition set.
  • the synthesis filterbank may further be configured such that the windower is configured such that the window function comprises more window coefficients with an absolute value of 10% of a maximum absolute value of the window coefficients of the window function in a first half of the definition set than in the second half of the definition set with respect to the midpoint of the definition set wherein the first half corresponds to the earlier half of the output values.
  • the synthesis filterbank may further be configured such that the windower is configured such that the window function is based on, mirrored variant of or identical with a window function based on which the input frames are generated for the synthesis filterbank.
  • the synthesis filterbank may further be configured such that the windower is configured such that the window function is a mirrored window function with respect to a midpoint of the definition set of the window function compared to a window function based on which the input frames are generated for the synthesis filterbank.
  • the synthesis filterbank may further be configured such that the windower is configured such that the predetermined value is 0.
  • the synthesis filterbank may further be configured such that the windower is configured to setting a windowed sample to a value in the predetermined range by at least one of setting the corresponding windowed sample to a value comprising an absolute value less than a minimum threshold and setting the corresponding windowed sample to a value comprising an absolute value more than a maximum threshold.
  • the synthesis filterbank may further be configured such that the minimum threshold or the maximum threshold is given by 10 s or 2 s , wherein s is an integer.
  • the synthesis filterbank may further be configured such that the minimum threshold is determined by a maximum absolute value representable by a least significant bit or a plurality of a least significant bits or the maximum threshold is determined by a minimum absolute value representable by a most significant bit or a plurality of most significant bits in the case of a binary representation of at least one of the input values, the output samples and the windowed samples.
  • the synthesis filterbank may further be configured such that the windower is configured such that the number of output samples disregarded or the number of windowed samples set to the predetermined value or to at least a value in the predetermined range is greater than or equal to the number of output samples of an output frame divided by 64.
  • the synthesis filterbank may further be configured such that the windower is configured such that the number of output values disregarded or the number of windowed samples set to the predetermined value or to at least a value in the predetermined range is greater than or equal to the number of added samples of an added frame divided by 16.
  • the synthesis filterbank may further be configured such that the windower is configured to disregarding 128 or 120 or setting to the predetermined value or to at least a value in the predetermined range 128 or 120 windowed samples.
  • the synthesis filterbank may further be configured such that the overlap/adder is configured to generating the added frame based on at least three consecutively generated windowed frames by the windower.
  • the synthesis filterbank may further be configured such that the overlap/adder is configured to generating the added frame based on at least three consecutively generated output frames by the frequency/time converter.
  • the synthesis filterbank may further be configured such that the overlap/adder is configured to generating the added frame comprising a number of added samples which is equal to the sample advance value.
  • the synthesis filterbank may further be configured such that the overlap/adder is configured to providing an added frame comprising a plurality of added samples based on at least 4 windowed samples from at least 4 different windowed frames for an added sample corresponding to a windowed sample, which is not based on a disregarded output sample, set to the predetermined value and to a value in the predetermined range by the windower, and based on at least 3 windowed samples from at least 3 different windowed frames for an added sample corresponding to an output sample which is disregarded or set to the predetermined value or to a value in the predetermined range by the windower.
  • the synthesis filterbank may further be configured such that the overlap/adder is configured to providing added frames comprising a number of added samples, which is less than the number of output values of an output frame divided by 2.
  • the synthesis filterbank may further be configured such that the overlap/adder is configured to providing added frames comprising a number of added samples, which is equal to the number of output samples of an output frame divided by an integer larger than 2.
  • the synthesis filterbank may further be configured such that the overlap/adder is configured to providing added frames comprising a number of added samples, which is equal to the number of output samples of an output frame divided by 4.
  • the synthesis filterbank may be comprised in a decoder.
  • the decoder may further comprise an entropy decoder configured to decoding a plurality of encoded frames, and wherein the entropy encoder is configured to providing a plurality of input frames based on the encoded frames to the synthesis filterbank.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Facsimile Transmission Control (AREA)
  • Telephonic Communication Services (AREA)
  • Complex Calculations (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Image Processing (AREA)
  • Monitoring And Testing Of Transmission In General (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Investigating Or Analysing Biological Materials (AREA)
  • Noise Elimination (AREA)
EP14199155.4A 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system Active EP2884490B1 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US86203206P 2006-10-18 2006-10-18
US11/744,641 US8036903B2 (en) 2006-10-18 2007-05-04 Analysis filterbank, synthesis filterbank, encoder, de-coder, mixer and conferencing system
EP07801974A EP2074615B1 (en) 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system
EP11173652.6A EP2378516B1 (en) 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system

Related Parent Applications (2)

Application Number Title Priority Date Filing Date
EP07801974A Division EP2074615B1 (en) 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system
EP11173652.6A Division EP2378516B1 (en) 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system

Publications (2)

Publication Number Publication Date
EP2884490A1 EP2884490A1 (en) 2015-06-17
EP2884490B1 true EP2884490B1 (en) 2016-06-29

Family

ID=38904615

Family Applications (5)

Application Number Title Priority Date Filing Date
EP09010179A Active EP2113911B1 (en) 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system
EP07801974A Active EP2074615B1 (en) 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system
EP14199155.4A Active EP2884490B1 (en) 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system
EP11173652.6A Active EP2378516B1 (en) 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system
EP09010178A Active EP2113910B1 (en) 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system

Family Applications Before (2)

Application Number Title Priority Date Filing Date
EP09010179A Active EP2113911B1 (en) 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system
EP07801974A Active EP2074615B1 (en) 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system

Family Applications After (2)

Application Number Title Priority Date Filing Date
EP11173652.6A Active EP2378516B1 (en) 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system
EP09010178A Active EP2113910B1 (en) 2006-10-18 2007-08-29 Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system

Country Status (22)

Country Link
US (6) US8036903B2 (he)
EP (5) EP2113911B1 (he)
JP (5) JP5546863B2 (he)
KR (3) KR101162455B1 (he)
CN (4) CN102243874B (he)
AT (3) ATE539432T1 (he)
AU (3) AU2007312696B2 (he)
BR (2) BRPI0716004B1 (he)
CA (3) CA2667059C (he)
ES (5) ES2386206T3 (he)
HK (4) HK1128058A1 (he)
IL (4) IL197757A (he)
MX (1) MX2009004046A (he)
MY (4) MY155486A (he)
NO (5) NO342445B1 (he)
PL (5) PL2074615T3 (he)
PT (1) PT2884490T (he)
RU (1) RU2426178C2 (he)
SG (2) SG174835A1 (he)
TW (1) TWI355647B (he)
WO (1) WO2008046468A2 (he)
ZA (1) ZA200901650B (he)

Families Citing this family (46)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7422840B2 (en) * 2004-11-12 2008-09-09 E.I. Du Pont De Nemours And Company Apparatus and process for forming a printing form having a cylindrical support
GB2439685B (en) 2005-03-24 2010-04-28 Siport Inc Low power digital media broadcast receiver with time division
US7916711B2 (en) * 2005-03-24 2011-03-29 Siport, Inc. Systems and methods for saving power in a digital broadcast receiver
WO2006138598A2 (en) * 2005-06-16 2006-12-28 Siport, Inc. Systems and methods for dynamically controlling a tuner
US8335484B1 (en) 2005-07-29 2012-12-18 Siport, Inc. Systems and methods for dynamically controlling an analog-to-digital converter
USRE50132E1 (en) 2006-10-25 2024-09-17 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating audio subband values and apparatus and method for generating time-domain audio samples
USRE50158E1 (en) 2006-10-25 2024-10-01 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating audio subband values and apparatus and method for generating time-domain audio samples
ATE547898T1 (de) * 2006-12-12 2012-03-15 Fraunhofer Ges Forschung Kodierer, dekodierer und verfahren zur kodierung und dekodierung von datensegmenten zur darstellung eines zeitdomänen-datenstroms
US8015368B2 (en) * 2007-04-20 2011-09-06 Siport, Inc. Processor extensions for accelerating spectral band replication
US8199769B2 (en) 2007-05-25 2012-06-12 Siport, Inc. Timeslot scheduling in digital audio and hybrid audio radio systems
US20090099844A1 (en) * 2007-10-16 2009-04-16 Qualcomm Incorporated Efficient implementation of analysis and synthesis filterbanks for mpeg aac and mpeg aac eld encoders/decoders
US9275648B2 (en) * 2007-12-18 2016-03-01 Lg Electronics Inc. Method and apparatus for processing audio signal using spectral data of audio signal
KR101253278B1 (ko) 2008-03-04 2013-04-11 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. 복수의 입력 데이터 스트림을 믹싱하는 장치 및 방법
CN103000178B (zh) 2008-07-11 2015-04-08 弗劳恩霍夫应用研究促进协会 提供时间扭曲激活信号以及使用该时间扭曲激活信号对音频信号编码
TWI496479B (zh) * 2008-09-03 2015-08-11 Dolby Lab Licensing Corp 增進多聲道之再生
KR101316979B1 (ko) 2009-01-28 2013-10-11 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. 오디오 코딩
TWI458258B (zh) 2009-02-18 2014-10-21 Dolby Int Ab 低延遲調變濾波器組及用以設計該低延遲調變濾波器組之方法
US8320823B2 (en) * 2009-05-04 2012-11-27 Siport, Inc. Digital radio broadcast transmission using a table of contents
US8971551B2 (en) 2009-09-18 2015-03-03 Dolby International Ab Virtual bass synthesis using harmonic transposition
US8831318B2 (en) * 2009-07-06 2014-09-09 The Board Of Trustees Of The University Of Illinois Auto-calibrating parallel MRI technique with distortion-optimal image reconstruction
CA2777182C (en) * 2009-10-09 2016-11-08 Dts, Inc. Adaptive dynamic range enhancement of audio recordings
PL2489041T3 (pl) * 2009-10-15 2020-11-02 Voiceage Corporation Jednoczesne kształtowanie szumu w dziedzinie czasu i w dziedzinie częstotliwości dla przekształcenia tdac
EP2372704A1 (en) * 2010-03-11 2011-10-05 Fraunhofer-Gesellschaft zur Förderung der Angewandten Forschung e.V. Signal processor and method for processing a signal
BR122021003688B1 (pt) 2010-08-12 2021-08-24 Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E. V. Reamostrar sinais de saída de codecs de áudio com base em qmf
US8489053B2 (en) 2011-01-16 2013-07-16 Siport, Inc. Compensation of local oscillator phase jitter
AU2012217216B2 (en) 2011-02-14 2015-09-17 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for coding a portion of an audio signal using a transient detection and a quality result
SG192746A1 (en) 2011-02-14 2013-09-30 Fraunhofer Ges Forschung Apparatus and method for processing a decoded audio signal in a spectral domain
CN102959620B (zh) * 2011-02-14 2015-05-13 弗兰霍菲尔运输应用研究公司 利用重迭变换的信息信号表示
PL3471092T3 (pl) 2011-02-14 2020-12-28 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Dekodowanie pozycji impulsów ścieżek sygnału audio
ES2534972T3 (es) 2011-02-14 2015-04-30 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Predicción lineal basada en esquema de codificación utilizando conformación de ruido de dominio espectral
RU2571561C2 (ru) * 2011-04-05 2015-12-20 Ниппон Телеграф Энд Телефон Корпорейшн Способ кодирования, способ декодирования, кодер, декодер, программа и носитель записи
JP5714180B2 (ja) 2011-05-19 2015-05-07 ドルビー ラボラトリーズ ライセンシング コーポレイション パラメトリックオーディオコーディング方式の鑑識検出
US9460729B2 (en) * 2012-09-21 2016-10-04 Dolby Laboratories Licensing Corporation Layered approach to spatial audio coding
EP2907324B1 (en) * 2012-10-15 2016-11-09 Dolby International AB System and method for reducing latency in transposer-based virtual bass systems
EP3291233B1 (en) * 2013-09-12 2019-10-16 Dolby International AB Time-alignment of qmf based processing data
DE102014214143B4 (de) * 2014-03-14 2015-12-31 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Verarbeiten eines Signals im Frequenzbereich
EP2980791A1 (en) * 2014-07-28 2016-02-03 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Processor, method and computer program for processing an audio signal using truncated analysis or synthesis window overlap portions
CN104732979A (zh) * 2015-03-24 2015-06-24 无锡天脉聚源传媒科技有限公司 一种音频数据的处理方法及装置
CN106297813A (zh) 2015-05-28 2017-01-04 杜比实验室特许公司 分离的音频分析和处理
EP3107096A1 (en) 2015-06-16 2016-12-21 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Downscaled decoding
WO2017050398A1 (en) * 2015-09-25 2017-03-30 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Encoder, decoder and methods for signal-adaptive switching of the overlap ratio in audio transform coding
US10762911B2 (en) * 2015-12-01 2020-09-01 Ati Technologies Ulc Audio encoding using video information
JP2018101826A (ja) * 2016-12-19 2018-06-28 株式会社Ｃｒｉ・ミドルウェア 音声通話システム、音声通話方法およびプログラム
US11282492B2 (en) 2019-02-18 2022-03-22 Bose Corporation Smart-safe masking and alerting system
US10991355B2 (en) * 2019-02-18 2021-04-27 Bose Corporation Dynamic sound masking based on monitoring biosignals and environmental noises
US11071843B2 (en) 2019-02-18 2021-07-27 Bose Corporation Dynamic masking depending on source of snoring

Family Cites Families (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5297236A (en) * 1989-01-27 1994-03-22 Dolby Laboratories Licensing Corporation Low computational-complexity digital filter bank for encoder, decoder, and encoder/decoder
CN1062963C (zh) * 1990-04-12 2001-03-07 多尔拜实验特许公司 用于产生高质量声音信号的解码器和编码器
US5869819A (en) 1994-08-17 1999-02-09 Metrologic Instuments Inc. Internet-based system and method for tracking objects bearing URL-encoded bar code symbols
US5408580A (en) * 1992-09-21 1995-04-18 Aware, Inc. Audio compression system employing multi-rate signal analysis
FI935609A (fi) 1992-12-18 1994-06-19 Lonza Ag Dihydrofuroimidatsolijohdannaisten asymmetrinen hydraus
JP3531177B2 (ja) * 1993-03-11 2004-05-24 ソニー株式会社 圧縮データ記録装置及び方法、圧縮データ再生方法
US5570363A (en) * 1994-09-30 1996-10-29 Intel Corporation Transform based scalable audio compression algorithms and low cost audio multi-point conferencing systems
US5867819A (en) 1995-09-29 1999-02-02 Nippon Steel Corporation Audio decoder
US5890106A (en) * 1996-03-19 1999-03-30 Dolby Laboratories Licensing Corporation Analysis-/synthesis-filtering system with efficient oddly-stacked singleband filter bank using time-domain aliasing cancellation
US5848391A (en) * 1996-07-11 1998-12-08 Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. Method subband of coding and decoding audio signals using variable length windows
SG54379A1 (en) * 1996-10-24 1998-11-16 Sgs Thomson Microelectronics A Audio decoder with an adaptive frequency domain downmixer
US5946352A (en) * 1997-05-02 1999-08-31 Texas Instruments Incorporated Method and apparatus for downmixing decoded data streams in the frequency domain prior to conversion to the time domain
JP4174859B2 (ja) * 1998-07-15 2008-11-05 ヤマハ株式会社 デジタルオーディオ信号のミキシング方法およびミキシング装置
US6226608B1 (en) * 1999-01-28 2001-05-01 Dolby Laboratories Licensing Corporation Data framing for adaptive-block-length coding system
JP2000267682A (ja) * 1999-03-19 2000-09-29 Victor Co Of Japan Ltd 畳み込み演算装置
US6687663B1 (en) * 1999-06-25 2004-02-03 Lake Technology Limited Audio processing method and apparatus
JP3518737B2 (ja) * 1999-10-25 2004-04-12 日本ビクター株式会社 オーディオ符号化装置、オーディオ符号化方法、及びオーディオ符号化信号記録媒体
JP2001134274A (ja) * 1999-11-04 2001-05-18 Sony Corp ディジタル信号処理装置および処理方法、ディジタル信号記録装置および記録方法、並びに記録媒体
FR2802329B1 (fr) * 1999-12-08 2003-03-28 France Telecom Procede de traitement d'au moins un flux binaire audio code organise sous la forme de trames
SE0001926D0 (sv) 2000-05-23 2000-05-23 Lars Liljeryd Improved spectral translation/folding in the subband domain
US6718300B1 (en) 2000-06-02 2004-04-06 Agere Systems Inc. Method and apparatus for reducing aliasing in cascaded filter banks
US6707869B1 (en) 2000-12-28 2004-03-16 Nortel Networks Limited Signal-processing apparatus with a filter of flexible window design
US6963842B2 (en) 2001-09-05 2005-11-08 Creative Technology Ltd. Efficient system and method for converting between different transform-domain signal representations
CN1682281B (zh) * 2002-09-17 2010-05-26 皇家飞利浦电子股份有限公司 在语音合成中用于控制持续时间的方法
JP2004184536A (ja) * 2002-11-29 2004-07-02 Mitsubishi Electric Corp 畳み込み演算装置及び畳み込み演算プログラム
US7318027B2 (en) 2003-02-06 2008-01-08 Dolby Laboratories Licensing Corporation Conversion of synthesized spectral components for encoding and low-complexity transcoding
US6982377B2 (en) * 2003-12-18 2006-01-03 Texas Instruments Incorporated Time-scale modification of music signals based on polyphase filterbanks and constrained time-domain processing
US7516064B2 (en) * 2004-02-19 2009-04-07 Dolby Laboratories Licensing Corporation Adaptive hybrid transform for signal analysis and synthesis
US7639823B2 (en) * 2004-03-03 2009-12-29 Agere Systems Inc. Audio mixing using magnitude equalization
KR20070001185A (ko) * 2004-03-17 2007-01-03 코닌클리케 필립스 일렉트로닉스 엔.브이. 오디오 코딩
US7630902B2 (en) * 2004-09-17 2009-12-08 Digital Rise Technology Co., Ltd. Apparatus and methods for digital audio coding using codebook application ranges
KR20070068424A (ko) * 2004-10-26 2007-06-29 마츠시타 덴끼 산교 가부시키가이샤 음성 부호화 장치 및 음성 부호화 방법
JP2006243664A (ja) * 2005-03-07 2006-09-14 Nippon Telegr & Teleph Corp <Ntt> 信号分離装置、信号分離方法、信号分離プログラム及び記録媒体
GB2426168B (en) * 2005-05-09 2008-08-27 Sony Comp Entertainment Europe Audio processing

Also Published As

Publication number Publication date
CN102243875B (zh) 2013-04-03
ES2374014T3 (es) 2012-02-13
CN101529502B (zh) 2012-07-25
USRE45294E1 (en) 2014-12-16
USRE45339E1 (en) 2015-01-13
EP2378516A1 (en) 2011-10-19
MY155486A (en) 2015-10-30
HK1163332A1 (en) 2012-09-07
RU2426178C2 (ru) 2011-08-10
CA2782476A1 (en) 2008-04-24
EP2113911B1 (en) 2011-12-28
SG174835A1 (en) 2011-10-28
TW200832357A (en) 2008-08-01
AU2007312696A8 (en) 2009-05-14
MY164995A (en) 2018-02-28
IL226225A0 (he) 2013-06-27
EP2884490A1 (en) 2015-06-17
USRE45276E1 (en) 2014-12-02
PL2884490T3 (pl) 2016-12-30
IL197757A0 (en) 2009-12-24
IL226225A (he) 2016-02-29
BRPI0716004A2 (pt) 2013-07-30
MY155487A (en) 2015-10-30
HK1128058A1 (en) 2009-10-16
NO342445B1 (no) 2018-05-22
HK1138674A1 (en) 2010-08-27
JP2014059570A (ja) 2014-04-03
AU2007312696B2 (en) 2011-04-21
NO20170982A1 (no) 2009-05-14
JP5859504B2 (ja) 2016-02-10
BRPI0716004B1 (pt) 2020-11-17
ATE554480T1 (de) 2012-05-15
AU2011201331A1 (en) 2011-04-14
EP2378516B1 (en) 2015-01-07
CN102243874A (zh) 2011-11-16
JP5700714B2 (ja) 2015-04-15
AU2007312696A1 (en) 2008-04-24
WO2008046468A3 (en) 2008-06-26
NO20091900L (no) 2009-05-14
CN102243873A (zh) 2011-11-16
ATE525720T1 (de) 2011-10-15
JP2012150507A (ja) 2012-08-09
CA2667059A1 (en) 2008-04-24
ES2386206T3 (es) 2012-08-13
PL2378516T3 (pl) 2015-06-30
KR20090076924A (ko) 2009-07-13
IL197757A (he) 2014-09-30
JP5520994B2 (ja) 2014-06-11
JP2013228740A (ja) 2013-11-07
NO342515B1 (no) 2018-06-04
CN102243873B (zh) 2013-04-24
CA2667059C (en) 2014-10-21
EP2074615A2 (en) 2009-07-01
US8036903B2 (en) 2011-10-11
ES2531568T3 (es) 2015-03-17
NO20170985A1 (no) 2009-05-14
USRE45526E1 (en) 2015-05-19
ES2592253T3 (es) 2016-11-29
KR20110049885A (ko) 2011-05-12
USRE45277E1 (en) 2014-12-02
US20080097764A1 (en) 2008-04-24
TWI355647B (en) 2012-01-01
CN101529502A (zh) 2009-09-09
ATE539432T1 (de) 2012-01-15
NO20170988A1 (no) 2009-05-14
CA2782609C (en) 2016-10-04
NO342516B1 (no) 2018-06-04
KR20110049886A (ko) 2011-05-12
EP2113910A1 (en) 2009-11-04
MX2009004046A (es) 2009-04-27
CN102243875A (zh) 2011-11-16
KR101162455B1 (ko) 2012-07-04
PL2074615T3 (pl) 2012-10-31
EP2113911A3 (en) 2009-11-18
IL226223A0 (he) 2013-06-27
HK1138423A1 (en) 2010-08-20
AU2011201330A1 (en) 2011-04-14
AU2011201331B2 (en) 2012-02-09
WO2008046468A2 (en) 2008-04-24
JP5546863B2 (ja) 2014-07-09
IL226224A0 (he) 2013-06-27
ZA200901650B (en) 2010-03-31
EP2074615B1 (en) 2012-04-18
JP2013210656A (ja) 2013-10-10
RU2009109129A (ru) 2010-11-27
KR101209410B1 (ko) 2012-12-10
PL2113910T3 (pl) 2012-02-29
JP2010507111A (ja) 2010-03-04
NO20170986A1 (no) 2009-05-14
EP2113911A2 (en) 2009-11-04
KR101162462B1 (ko) 2012-07-04
JP5700713B2 (ja) 2015-04-15
EP2113910B1 (en) 2011-09-21
NO342476B1 (no) 2018-05-28
IL226224A (he) 2016-02-29
ES2380177T3 (es) 2012-05-09
PT2884490T (pt) 2016-10-13
IL226223A (he) 2016-02-29
NO342514B1 (no) 2018-06-04
CA2782476C (en) 2016-02-23
BR122019020171B1 (pt) 2021-05-25
PL2113911T3 (pl) 2012-06-29
MY153289A (en) 2015-01-29
SG174836A1 (en) 2011-10-28
BRPI0716004A8 (pt) 2019-10-08
CN102243874B (zh) 2013-04-24
CA2782609A1 (en) 2008-04-24
AU2011201330B2 (en) 2011-08-25

Similar Documents

Publication Publication Date Title
EP2884490B1 (en) Analysis filterbank, synthesis filterbank, encoder, decoder, mixer and conferencing system
USRE50009E1 (en) Apparatus and method for generating audio subband values and apparatus and method for generating time-domain audio samples
KR101192241B1 (ko) 입력 데이터 스트림의 믹싱과 그로부터 출력 데이터 스트림의 생성
USRE50158E1 (en) Apparatus and method for generating audio subband values and apparatus and method for generating time-domain audio samples

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20141219

AC Divisional application: reference to earlier application

Ref document number: 2074615

Country of ref document: EP

Kind code of ref document: P

Ref document number: 2378516

Country of ref document: EP

Kind code of ref document: P

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

R17P Request for examination filed (corrected)

Effective date: 20150821

RBV Designated contracting states (corrected)

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20160121

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AC Divisional application: reference to earlier application

Ref document number: 2378516

Country of ref document: EP

Kind code of ref document: P

Ref document number: 2074615

Country of ref document: EP

Kind code of ref document: P

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 809654

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160715

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602007046861

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: PT

Ref legal event code: SC4A

Ref document number: 2884490

Country of ref document: PT

Date of ref document: 20161013

Kind code of ref document: T

Free format text: AVAILABILITY OF NATIONAL TRANSLATION

Effective date: 20160929

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160629

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2592253

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20161129

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160930

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160629

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 809654

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160629

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160629

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161029

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160629

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160629

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160629

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160629

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602007046861

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160629

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160831

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160629

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

26N No opposition filed

Effective date: 20170330

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160829

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160629

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160929

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160829

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20070829

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160629

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160831

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230516

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: TR

Payment date: 20230821

Year of fee payment: 17

Ref country code: IT

Payment date: 20230831

Year of fee payment: 17

Ref country code: ES

Payment date: 20230918

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: SE

Payment date: 20230823

Year of fee payment: 17

Ref country code: PL

Payment date: 20230821

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20240821

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240819

Year of fee payment: 18

Ref country code: FI

Payment date: 20240820

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20240822

Year of fee payment: 18

Ref country code: PT

Payment date: 20240828

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: BE

Payment date: 20240820

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240821

Year of fee payment: 18