BRPI0914056B1 - MULTI-RESOLUTION SWITCHED AUDIO CODING / DECODING SCHEME - Google Patents

Abstract

multiresolution switchable encoding and decoding scheme an audio encoder for encoding an audio signal comprises a first coding branch (400), the first coding branch comprises a first converter (410) for converting a time domain signal into a frequency domain. In addition, the audio encoder comprises a second coding branch (500) comprising a second frequency / time converter (53). In addition, a signal analyzer (300/525) is provided for analyzing the audio signal. The signal analyzer in turn determines whether an audio portion is effective at the encoder output signal as a first coded signal from the first coding branch or a second coded end of a second coding branch. on the other hand, the signal analyzer determines a time / frequency resolution to be applied by converters - (410, 523) when generating the encoded signals. An output interface includes, in addition to the first coded signal and the second coded signal, a resolution information identifying the resolution used by the first time / frequency converter and used by the second time / frequency converter.

Description

MULTI-RESOLUTION SWITCHED AUDIO ENCODING / DECODING SCHEME

Frequency domain encoding schemes are advantageous in that they show high quality at low bit rates for music signals. A problem, however, is the quality of voice signals at a low bit rate.

Voice coding schemes show high quality for speech signals, even at low bit rates, but have poor quality for other signals at low bit rates.

It is an objective of the present invention to provide a better concept of encoding / decoding.

This object is achieved by an audio encoder according to claim _1, _ an encoding method. audio - according to claim 9, a decoder according to claim 10, a decoding method according to claim - “signal encoded according to claim 20 or a computer program according to claim 21.

The current invention is based on the discovery that a hybrid or dualmode switching encoding / decoding scheme is advantageous in that the best encoding algorithm can always be selected for a certain signal characteristic. In other words, the present invention does not seek a signal encoding algorithm that is perfectly adapted to all the characteristics of the signal. Such a scheme would always be a compromise, as can be seen from the great differences between, on the one hand, the audio encoders of the prior art and, on the other, the voice encoders. Instead

In addition, the present invention combines different coding algorithms, such as a speech coding algorithm, on the one hand, and, on the other, an audio coding algorithm within a switching scheme so that, for each portion of the audio signal, the optimally corresponding encoding algorithm is selected. In addition, it is also a feature of the present invention that both branches of coding comprise a time / frequency converter, but in a branch of coding, another domain converter such as an LPC processor is provided.

This domain converter ensures that the second encoding branch is more suitable for a certain characteristic signal than the first encoding branch. However, it is also a feature of the present invention that the output of the .pel-by-the-domain processor signal is also transformed into a spectral representation.

- - - - Both 'õs' converters, that is, the first converter in the first coding branch and the second converter in the second coding branch are configured to apply a multi-resolution transformation encoding, where the resolution of the corresponding converter is dependent on the configuration of the audio signal and, in particular, dependent on the audio signal truly encoded in the corresponding encoding branch so that a good fit between quality, on the one hand, and bit rate, on the other hand, or in In view of a certain fixed quality, the lowest bit rate, or in view of a fixed bit rate, the highest quality is obtained.

According to the present invention, the time / frequency resolution of the two converters can preferably be

3/114 configured independently of each other so that each time / frequency transformer can be perfectly matched to the corresponding signal's time / frequency resolution requirements. The bit efficiency, that is, the ratio between useful bits on the one hand, and the bits of collateral information on the other hand, is highest for the longest window lengths / block sizes. Therefore, it is preferable that both converters are more inclined to a longer window length, since, basically, the same amount of collateral information refers to a longer time portion of the audio signal in relation to the application of smaller sizes transformation lengths / window lengths / shorter block sizes. Preferably, the time / frequency resolution in the coding branches can also be influenced by other coding / decoding tools located in these branches. Preferably, '' 0 'second' coding branch comprising the domain converter, such as an LPC processor comprises another hybrid scheme such as an ACELP branch on the one hand and a TCX scheme on the other hand, where the second converter is included in the TCX scheme. Preferably, the resolution of the time / frequency converter located in the TCX branch is also influenced by the coding decision, so that a portion of the signal in the second coding branch is processed in the TCX branch, the second converter having or not having a time / frequency converter in the

ACELP.

Basically, neither the domain converter, nor the second branch of coding, and particularly, the first branch of processing in the second branch of coding and the second branch of

4/114 processing in the second branch of coding, must be elements related to speech such as an LPC analyzer for the domain converter, a TCX encoder for the second processing branch and an ACELP encoder for the first processing branch. Other applications are also useful when evaluating other signal characteristics of an audio signal other than speech, on the one hand, and music, on the other. Any domain converters and coding branch implementations can be used and the best matching algorithm can be found through an analysis-by-synthesis scheme so that, on the encoder side, for each portion of the audio signal, they are all coding alternatives are conducted and the best result is selected ,, .where- the best fesult. can be found when applying a target function to the coding results. Then, the identification of collateral information to a decoder, the underlying 'dè' encoding algorithm for a certain portion of the encoded audio signal is attached to the encoded audio signal by an encoder output interface, so that the decoder does not have to take care of any decisions on the encoder side or any signal characteristics, but simply select your encoding branch depending on the collateral information transmitted. In addition, the decoder will not only select the correct decoding branch, but will also select, based on collateral information encoded in the encoded signal, which time / frequency resolution should be applied to a corresponding first decoding branch and a second decoding branch corresponding.

Thus, the present invention provides a scheme of

5/114 encoding / decoding that combines the advantages of all different encoding algorithms and avoids the disadvantages of these encoding algorithms that arise, when the signal portion would have to be encoded, by an algorithm that does not fit in with a given coding. In addition, the present invention avoids any disadvantages that would arise if the different time / frequency resolution requirements raised by different audio signal portions in different coding branches were not accounted for. Instead, due to the variable time / frequency resolution of time / frequency converters in both branches, no artifact is at least reduced or even completely avoided, as they would arise in the scenario where the same time / frequency resolution would be applied to both coding branches, or where only a fixed time / frequency resolution would be possible for any coding branches— - - - - “

The second switching again decides between two processing branches, but in a different domain than the first external branch domain. Again, an internal branch is motivated mainly by a source model or by SNR calculations, and another internal branch can be motivated by a sink model and / or a psychoacoustic model, that is, masking or at least including aspects of frequency domain coding. /spectrum. Exemplarily, an internal branch has a frequency domain spectral converter / encoder and another branch has an encoder encoding in another domain such as the LPC domain, where this encoder is, for example, a CELP or ACELP quantizer / scalimeter. processing a signal input without a conversion

Spectral 6/114.

Another more preferred configuration is an audio encoder comprising a first encoding branch of encoding information such as a spectral domain encoding branch, a second information source or SNR orientation encoding branch such as an encoding branch of information. LPC domain, and a key for switching between the first coding branch and the second coding branch, where the second coding branch comprises a converter for a specific domain other than the time domain as an LPC analysis stage generating a excitation signal, and wherein the second decoding branch, furthermore, has a specific domain such as the LPC domain processing branch and a specific spectral domain such as LPC spectral domain processing branch, and an additional key for switching between the specific domain-coding branch and the data coding branch specific spectral domain.

A more advanced configuration of the invention is an audio decoder comprising a first domain, such as a spectral domain decoding branch, a second domain such as an LPC domain decoding branch to decode a signal such as an excitation signal in the second domain and a third domain such as a LPC spectral decoder branch for decoding a signal as an excitation signal in a third domain such as a LPC spectral domain, where the third domain is obtained by performing a frequency conversion from the second domain in which the first switch is provided for the second domain signal and the third

7/114 domain signal, and in which a second key is provided for switching between the first domain decoder and the decoder for the second domain or third domain.

Preferred embodiments of the present invention are described subsequently with respect to the accompanying figures, in which:

Fig encoding the invention;

is a block diagram of a scheme according to a first aspect of this

Fig. Lb is a block diagram of a decoding scheme according to the first aspect of the present invention;

Fig. _Lc is a block diagram of a coding scheme according to another aspect of the present invention;

Fig. 2a is a block diagram of a coding scheme according to a second aspect of the present invention;

Fig. 2b is a schematic diagram of a decoding scheme according to the second aspect of the present invention.

Fig. 2c is a block diagram of a coding scheme according to another aspect of the present invention;

Fig. 3a illustrates a block diagram of an encoding scheme according to another aspect of the present invention;

Fig. 3b illustrates a block diagram of a decoding scheme according to the most comprehensive aspect of the present invention;

Fig. 3c illustrates a schematic representation of the coding apparatus / method with cascade switching;

8/114

Fig. 3d illustrates a schematic diagram of an apparatus or method for decoding, in which cascaded combiners are used;

Fig. 3e shows an illustration of a time domain signal and an encoded signal representation illustrating the short cross-fade regions that are included in both encoded signals;

Fig. 4a illustrates a block diagram with a key positioned before the coding branches;

Fig. 4b illustrates a block diagram of an encryption scheme with the key positioned subsequently to the encryption branches;

Fig. 5a illustrates a waveform of a time domain speech segment as a quasi-periodic signal segment or as a pulse;

__ _ Fig. 5b illustrates — a segment spectrum — of figure 5a; Fig. 5c illustrates a speech segment of the deaf speech domain as an example of a segment as noise;

Fig. 5d illustrates a time domain waveform spectrum of Fig. 5c;

Fig. 6 illustrates a block diagram of a CELP synthesis analysis encoder;

Fig. 7a to illustrate audible / muted excitation signals from 7d as an example for signals such as pulse;

Fig. 7e illustrates an encoder side LPC stage providing short-term forecast information and the forecast error (excitation) signal;

Fig. 7f illustrates another configuration of a

9/114 LPC device for generating a weighted signal;

Fig. 7g illustrates an implementation for transforming a weighted signal into an excitation signal by applying an inverse weighting operation and a subsequent excitation analysis as required in converter 537. of Fig. 2b;

Fig. 8 illustrates a block diagram of a joint multi-channel algorithm according to a configuration of the present invention;

Fig. 9 illustrates a preferred configuration of a bandwidth extension algorithm;

Fig. 10a illustrates a detailed description of the switching when making an open cycle decision; and

Fig. 10b shows an illustration of the switching when operating in a closed loop decision mode;

Fig. 11A illustrates a block diagram of an audio encoder in accordance with another aspect of the present invention;

Fig. 11B illustrates a block diagram of another configuration of an inventive audio decoder;

Fig. 12A illustrates another configuration of an inventive encoder;

Fig. 12B illustrates another configuration of an inventive decoder;

Fig. 13A illustrates the interrelationship between resolution and window / transformation lengths;

Fig. 13B illustrates an overview of a set of transformation windows for the first coding branch and a transition from the first to the second coding branch;

11/104

Fig. 13C illustrates a plurality of different window sequences, including window sequences for the first coding branch and sequences for a transition to the second branch;

Fig. 14A illustrates the structure of a preferred configuration of the second coding branch;

Fig. 14B illustrates short windows applied to the second coding branch;

Fig. 14C illustrates medium-sized windows applied to the second branch of coding;

Fig. 14D illustrates long windows applied by the second coding branch;

Fig. 14E illustrates an exemplary sequence of ACELP and TCX structures within a super division of structures;

Fig. 14F illustrates different transformation lengths, corresponding to different "time / frequency" resolutions for the second coding branch; and

Fig. 14G illustrates the construction of a window using the definitions in Fig. 14F

Fig. 11A illustrates an audio encoder configuration for encoding an audio signal. The encoder comprises a first encoding branch 400 for encoding an audio signal using a first encoding algorithm to obtain a first encoded signal.

The audio encoder, furthermore, comprises a second encoding branch 500 for encoding an audio signal using a second encoding algorithm for obtaining a second encoded signal. The first coding algorithm is

11/114 different from the second coding algorithm. In addition, a first key 200 is provided for switching between the first encoding branch and the second encoding branch so that, by a portion of the audio signal, either the first encoded signal or the second encoded signal is on an output signal from encoder 801.

The audio encoder shown in Fig. 11A further comprises a signal analyzer 300/525, which is configured to analyze a portion of the audio signal to determine whether the portion of the audio signal is represented as the first encoded signal or the second signal encoded at the signal output of encoder 801.

The signal analyzer 300/525, moreover, is configured to variably determine a respective resolution time / frequency of a first converter 410 in the first coding branch 400 or a second converter 523 in the second ^- give encoding branch 500. This resolution time / frequency is applied when the first encoded signal or the second encoded signal representing the portion of the audio signal is generated.

The audio encoder additionally comprises an output interface 800 for generating the signal output from encoder 801 comprising an encoded representation of the audio signal portion and information indicating whether the audio signal representation is the first encoded signal or the second signal encoded, and indicating the time / frequency resolution used to decode the first encoded signal and the second encoded signal.

The second branch of coding is preferably

12/114 different from the first decoding branch in which the second encoding branch additionally comprises a domain converter for converting the audio signal from the domain, in which the audio signal is processed in the first encoding branch to a different domain . Preferably, the domain converter is an LPC processor 510, but the domain converter can be implemented in any other way, as long as the domain converter is different from the first converter 410 and the second converter 523.

The first converter 410 is a time / frequency converter, preferably comprising a windscreen

410a and a transformer 410b. The winder 410a applies an analysis window to the input audio signal, and transformer 410b performs a conversion of the window signal into a spectral representation.

Similarly, - the -second-converter '523 preferably comprises a winder 523a and a transformer

523b connected subsequently. The winder 523a receives the signal output by the domain converter 510 and outputs the window representation present there. The result of an analysis window applied by the winder 523a is available as input to the transformer 523b to form a spectral representation. The transformer can be an FFT or, preferably, MDCT processor implementing a corresponding algorithm in software or hardware or in a mixed hardware / software implementation.

Alternatively, the transformer can be a filter bank implementation such as a QMF filter bank that can be based on a real or complex value modulation of a filter

13/114 prototype. For specific filter bank implementations, a window is applied. However, for other filter bank implementations, no window action is required as required for a transformation algorithm based on an MDCT FFT. When a filter bank implementation is used, then the filter bank is a variable resolution filter bank and the resolution controls the frequency resolution of the filter bank and, in addition, the time resolution or just the frequency resolution , not time resolution. When, however, the converter is implemented as an FFT or MDCT or any other corresponding transformer, then the frequency resolution is connected to the time resolution in which an increase in the frequency resolution obtained by a higher synchronized block length automatically corresponds to a smaller and more vicious time resolution.

- - In addition, the — first 'coding' branch may comprise a quantizer / encoder stage 421, and the second coding branch may also comprise one or more 524 encoding tools.

It is important to mention that the signal analyzer is configured to generate a resolution control signal for the first converter 510 and for the second converter 523. Thus, an independent resolution control is implemented in both branches of coding in order to have a scheme encoding which, on the one hand, provides a low bit rate, and on the other hand, guarantees maximum quality in view of the low bit rate. To achieve the low bit rate target, preference is given to longer window lengths or

14/114 longer transformation times, but in situations where these long lengths will result in an artifact, due to low time resolution, shorter window lengths and shorter transformation lengths are applied, which results in a lower frequency resolution . Preferably, the signal analyzer applies a statistical analysis or any other analysis that is suitable for the corresponding algorithms in the coding branches. In an implementation mode, in which the first coding branch is a frequency domain coding branch such as an AAC based encoder, and in which the second coding branch comprises, as a domain converter, a data processor. LPC 510, the signal analyzer performs speech / music discrimination so that the speech portion of the audio signal is fed into the second corresponding control coding branch of switching 200. A portion of the audio signal is fed into the first— branch-coding 400 by the corresponding switching control 200 as indicated by the switching control lines.

Alternatively, as will be discussed later with reference to Fig. 1C or Fig. 4B, the switch can also be positioned before the output interface 800.

In addition, the signal analyzer can receive mrc the audio signal input for switch 200 or the audio signal output for switch 200. In addition, the signal analyzer performs an analysis in order not to feed only the signal audio for the corresponding coding branch, but also to determine the time / frequency resolution of the respective converter in the corresponding coding branch, such as the first

15/114 converter 410 and the second converter 523 as indicated by the controlled resolution lines connecting the signal analyzer and the converter.

Fig. 11B comprises a preferred configuration of an audio decoder compatible with the audio encoder in Fig. 11A.

The audio decoder in Fig. 11B is configured to decode an encoded audio signal, such as the signal output from encoder 801 that exits through the 800 output interface in Fig. 11A. The encoded signal comprises a first encoded audio signal encoded according to a first encoding algorithm, a second encoded signal, encoding in accordance with a second encoding algorithm, the second encoding algorithm being different from the first encoding and information algorithm indicating whether the first coding algorithm or the second coding algorithm is used to decode the first encoded signal and the second encoded signal, and a time / frequency resolution information for the first encoded audio signal and the second encoded audio signal.

The audio decoder comprises a first decoding branch 431, 440 for decoding the first encoded signal based on the first encoding algorithm. In addition, the audio decoder comprises a second decoding branch for decoding the second encoded signal using the second encoding algorithm.

The first decoding branch comprises a first controllable converter 440 for converting a domain

16/114 spectral in the time domain. The controllable converter is configured to be controlled using the time / frequency resolution information from the first encoded signal to obtain the first decoded signal.

The second decoding branch comprises a second controllable converter for converting a spectral representation to a time representation, the second controllable converter 534 being configured to be controlled using the time / frequency resolution information 991 for the second encoded signal.

The decoder further comprises a controller 990 for controlling the first converter 540 and the second converter 534, according to the time / frequency resolution information 991.

In addition, the decoder comprises a domain converter for the generation of a synthesis signal using the second decoded signal in order to cancel the domain conversion applied by the domain converter 510 in the network encoder.

Fig. 11A.

Preferably, the domain converter 540 is an LPC synthesis processor, which is controlled using LPC filter information included in the encoded signal, where that LPC filter information was generated by the LPC processor 510 in Fig. 11A and has input for the encoder signal output as collateral information. The audio decoder finally comprises a combiner 600 for combining the first output of the signal decoded by the first domain converter 440 and the synthesis signal to obtain a decoded audio signal 609.

11/174

In the preferred implementation, the first decoding branch additionally comprises a quantizer / decoder stage 431 to reverse or at least partially reverse the operations performed by the corresponding encoder stage 421. However, it is clear that the quantization cannot be reversed, since this is a lossy operation. However, a quantizer will reverse a certain non-uniformity in a quantization such as a logarithmic or compander (compression) quantization.

In the second branch of decoding, the corresponding stage 533 is applied to undo certain coding operations applied by stage 524. Preferably, stage 524 comprises a uniform quantization. Therefore, the corresponding stage 533 will not have a specific quantization stage to undo a certain uniform quantization.

. _ The first —404 converter, —as well as the second 534 converter, may include a corresponding reverse transformer stage 440a, 534a, a synthesis window stage 440b,

534b, and the subsequently connected add / overlap stage 440c, 534c. The overlap / addition stages are necessary when converters, and, more specifically, transformer stages 440A, 534a apply slope introducing transformations such as a modified discrete cosine transformation. Then, the overlap / add operation will perform a time domain aliasing (TDAC). When, however, transformers apply a non-skewed input transformation such as a reverse FFT, then an overlap / add stage 440c is not

11/184 requested. In such an implementation, a cross fading operation to avoid blocking artifacts can be applied.

Similarly, combiner 600 can be a switching combiner or cross fading combiner, or when slope is used to avoid blocking artifacts, a transition window operation is implemented by the combiner similar to the overlap / addition stage within the combiner itself. branch.

Fig. 1 illustrates a configuration of the invention having two cascade switching. A mono signal, a stereo signal or a multi-channel signal are triggered as an input for switch 200. Switch 200 is controlled by decision stage 300. The decision stage receives a signal input in block 200 as an input. Alternatively, the decision table 300 can also receive collateral information that is included in the mono signal, the stereo signal or the multichannel signal- or - at least associated with - a- signal - like this? where “the information exists, which was, for example, generated when it originally produced the mono signal, the stereo signal or the multi-channel signal.

Decision stage 300 triggers switching 200 in order to feed a signal to both the frequency coding portion 400 shown on an upper branch of Fig. 1a and the domain coding portion of LPC 500 shown on a lower branch in Fig. . there. A key element of the frequency domain coding branch is the spectral conversion block 410, which is operational for converting a common preprocessing stage output signal (as discussed later) into a spectral domain. The spectral conversion block can include a

19/114 MDCT algorithm, a QMF, an FFT algorithm, an analysis

Wavelet or a filter bank such as a crucial sampling filter bank having a number of filter bank channels, where the subband signals in this filter bank can be real valuation signals or complex valuation signals. The output of the spectral conversion block 410 is encoded using a spectral audio encoder 421, which can include processing blocks as known from the AAC encoding scheme.

Generally, branch processing is processing in a perception-based model or information sink model. Thus, this branch shapes the human auditory system receiving the sound. Contrary to it, the processing in the branch 500 is to generate an excitation signal, residual or domain of LPC._ Generally, the processing in the branch 500 is a processing in a speech model or in an information generation model. For speech signals, - this model is a mccelc of human speech / sound generation systems. If, however, a sound from a different source that requires a different sound generation model has to be encoded, then processing on branch 500 may be different

In the lower coding branch 500, a key element is an LPC device 510, which outputs LPC information which is used to control the characteristics of an LPC filter. This LPC information is transmitted to a decoder. The LPC stage output signal 510 is an LPC domain signal, which consists of an excitation signal and / or a weighted signal.

The LPC device generally outputs an LPC domain signal, which can be any signal in the domain

20/114 of LPC such as the excitation signal in Fig. 7e or a weighted signal in Fig. 7f or any other signal, which was generated by applying LPC filter coefficients to an audio signal.

In addition, an LPC device can also determine these coefficients and can also quantize / encode those coefficients.

The decision at the decision stage can be an adaptive signal, so that the decision stage performs a music / speech discrimination and controls switching 200 in such a way that the music signals are inserted in the upper branch

400, and the speech signals are inserted into the lower branch 500. In an embodiment, the decision stage feeds its decision information into an output bit stream, such that the decoder can use this decision information in order to to perform correct decoding operations.

Such a decoder is illustrated in figure 1b. The signal output by the encoder — audio — spectral- 421 — is, after transmission, inserted into a 431 spectral audio decoder.

The output of the spectral audio decoder 431 is inserted into a 440 time domain converter. Similarly, the output from the LPC 500 domain coding branch of Fig. 1a is received at the decoder side and processed by elements 531, 533 , 534, and 532 for obtaining an LPC excitation signal. The LPC excitation signal is inserted into a synthesis stage of LPC 540, which receives, as an additional insert, the LPC information generated by the corresponding analysis stage of LPC 510. The output of the 440 and / / time domain converter or output from the LPC 540 synthesis stage are inserted in a 600 switching.

600 is controlled via a switching control signal that was,

21/114 for example, generated by decision stage 300, or that was provided externally, such as by a creator of the original mono signal, stereo signal or multi-channel signal. The switching output

600 is a full mono, stereo or multichannel signal.

The input signal for switching 200 and decision stage 300 can be a mono signal, a stereo signal, a multi-channel signal or generally an audio signal. Depending on the decision that can be derived from the switching input signal 200 or from any external source such as an original audio signal producer underlying the signal input to stage 200, the switching switches between the frequency 400 and the LPC 500 coding branch. The 400 frequency coding branch comprises a spectral conversion stage 410 and a subsequent connected--421 quantization / coding stage. · —The - quantization / coding stage can include any of the features as known from modern frequency domain encoders, such as the AAC encoder. In addition, the quantization operation in the quantization / coding stage 421 can be controlled via a psychoacoustic module that generates psychoacoustic information such as a psychoacoustic masking limit on the frequency, when that information is entered for the 421 stage.

In the LPC coding industry, the switching output signal is processed via an LPC analysis stage 510 generating LPC collateral information and an LPC domain signal.

The excitation encoder inventively comprises a switching

Additional 22/114 for switching further processing of the LPC domain signal between a quantization / encoding operation 522 in the LPC domain or a quantization / encoding stage 524, which are the processing values in the spectral domain of LPC. For this purpose, a spectral converter 523 is provided at the input of the quantization / coding stage 524. Switching 521 is controlled in the form of an open cycle or a closed cycle form, depending on the specific configurations as, for example, described in the technical specification AMR-WB +.

For closed loop control mode, the encoder additionally includes a reverse quantizer / encoder 531 for the LPC domain signal, a reverse quantizer / encoder 533 for the LPC spectral domain signal and a reverse spectral converter 534 for the output of item 533. Both signals encoded and decoded again in the processing branches- of the second decoding branch -are-inserted- for switching control device 525. In switching control device 525, these two output signals are compared to each other and / or to a target function or a target function is calculated, which can be based on a comparison of the distortion on both signals so that the signal with the least distortion is used to decide which position to switch over 521 must take. Alternatively, if both branches provide non-constant bit rates, the branch providing the lowest bit rate can be selected, even when the signal / noise ratio of this branch is less than the signal / noise ratio of another branch. Alternatively, the target function could use, as an input, the signal-to-noise ratio of each signal and a bit rate of each

23/114 additional signal and / or criteria in order to find the best decision for a specific objective. If, for example, the goal is that the bit rate should be as low as possible, then the target function will rely heavily on the bit rate of the two signal outputs by elements 531 and 534. However, when the main objective is have the best quality for a certain bit rate, then switching control 525 can, for example, discard each signal that is above the allowable bit rate and when both signals are below the allowable bit rate, switching control would select the signal having the best signal / noise ratio, that is, with the least quantization / encoding distortions.

The decoding scheme according to the present invention is, as said before, illustrated in Fig. Lb. For each of the three possible output signal types, there is a specific decode / re-quantize stage 431, 531 or 533. While stage 431 outputs a 'time' spectrum which is converted to the time domain using the frequency / time converter 440, stage 531 outputs an LPC domain signal, and item 533 performs an output from an LPC spectrum.

In order to make sure that the input signals for switching

532 are both in the LPC domain, the LPC-spectrum / LPCconverter 534 is provided. The switching output data 532 is transformed back into the time domain using an LPC 540 synthesis stage, which is controlled via information generated and transmitted from the side of the LPC encoder. Then, subsequent to block 540, both branches have time domain information that is switched according to a switching control signal, in order to finally obtain an audio signal, such as a mono signal,

24/114 a stereo signal or a multi-channel signal, which depends on the signal input for the coding scheme of Fig. 1a.

Fig. 1c illustrates another configuration with a different arrangement of switch 521 similar to the principle of Fig.

4b.

Fig. 2a illustrates a preferred coding scheme according to a second aspect of the invention. A common pre-processing scheme connected to switching input 200 may comprise a stereo surround / joint block

101 that generates, as an output, the joint stereo parameters and a mono signal output, which is generated by the downmix of the input signal, which is a signal that has two or more channels. Generally, the signal at the output of block 101 can also be a signal having more channels, but due to the downmix functionality of block 101, the number of channels at the output of block 101 will be less than the number of input channels at block 101 - - - - - - - ~ ~ ~ - The common pre-processing scheme may alternatively comprise block 101 or in addition to block 101 a bandwidth extension stage 102. In the configuration of Fig. 2a, the output of the block 101 is inserted into the bandwidth extension block 102, which, in the encoder of figure 2a, outputs a limited band signal, such as the low band signal or the low pass signal at its output. Preferably, this sign is decimated (for example, by a factor of two) as well. In addition, for the high band of the signal input in block 102, bandwidth extension parameters such as spectral envelope parameters, reverse filtering parameters, noise floor parameters, etc., known from the MPEG- HE-AAC profile 4 are

25/114 generated and forwarded to a 800 bit stream multiplexer.

Preferably, decision stage 300 receives signal input for block 101 or input for block 102 in order to decide between, for example, a music mode or a speech mode. In music mode, the upper coding branch 400 is selected, while in speech mode, the lower coding branch 500 is selected. Preferably, the decision stage additionally controls the joint stereo block 101 and / or the bandwidth extension block 102 to adapt the functionality of these blocks to the specific signal. Thus, when the decision stage determines that a certain portion of the input signal is the first mode such as the music mode, then the specific characteristics of block 101 and / or block _102_ can be controlled by decision stage 300. Alternatively , when the decision stage 300 determines that the signal is in a mode of speech_cu, in general, - in a second mode- of LPC domain; then the specific characteristics of blocks 101 and 102 can be controlled according to the output of the decision stage.

Preferably, the spectral conversion of the coding branch 400 is done using an MDCT operation, which, even more preferably, is the disfigured time MDCT operation, where the force or, generally, the deformation force can be controlled between zero and a high deformation force. At a zero strain force, the MDCT operation in block 411 is a direct MDCT operation known in the art. The disfigured time force with the disfigured time collateral information can be transmitted / inserted into the bitstream multiplexer 800 as collateral information.

11/26

In the LPC coding industry, the LPC domain encoder can include a core of ACELP 52 6 calculating from a height gain, with height lag and / or codebook information such as an index and codebook gain. The TCX mode as it is known from 3GPP TS 26.290 incurs a perceptually weighted signal processing in the transformation domain. The weighted Fourier transformation signal is quantized using a divided multi-rate network quantization (algebraic VQ) with noise factor quantization. A transformation is calculated in the sample windows 1024, 512, or 256. The excitation signal is recovered by reverse filtering the quantized weighted signal through an inverse weighting filter.

In the first branch of coding 400, a spectral converter preferably comprises a specific adaptation operation MDCT having certain window functions followed by the quantization / entropy coding stage, which can ^- consist of a single vector quantization stage, but preferably it is a combined scalar quantizer / entropy encoder similar to the quantizer / encoder in the frequency domain coding branch, that is, in item 421 of Fig. 2a.

In the second branch of coding, there is the LPC block 510 followed by a switch 521, again followed by an ACELP 526 block or a TCX 527 block. ACELP is described in 3GPP TS

26.190 and TCX is described in 3GPP TS 26.290. Generally, the block of

ACELP 526 receives an LPC excitation signal, calculated by a procedure described in Fig. 7e. The TCX block 527 receives a weighted signal as generated by Fig. 7f.

In TCX the transformation is applied to the signal

27/114 weighted computed by filtering the input signal through a weighted filter based on LPC. The preferred configurations used for the weighted filter of the invention are given by (1 - A (z //)) / (1 - μζ ~ ^χ ). Thus, the weighted signal is an LPC domain signal, and its transformation is an LPC spectral domain. The signal processed by the ACELP block 526 is the excitation signal and is different from the signal processed by the block 527, but both signals are in the LPC domain.

Next to the decoder in Fig. 2b, after the inverse spectral transformation in block 537, the inverse of the weighted filter is applied, that is, (1 - / zz ^_1 ) / (l - A (zIγ ^). the signal is filtered through (lA (z)) to go to the LPC excitation domain.

Thus, the conversion to LPC _domain block. 534 · and -bl-hollow of LPC ^-1 537 include an inverse transformation and then filtering (1- / ZZ ^-1 ) (1 - A (z)). to convert from the weighted domain using (1- A (z / y)) to the excitation domain.

Despite item 510 in Figs. 1a, lc, 2a, 2c illustrate a single block, block 510 can output different signals, as long as those signals are in the LPC domain. The actual mode of block 510, such as the excitation signal mode or the weighted signal mode, may depend on the actual switching state. Alternatively, block 510 may have two parallel processing devices, where a device similar to Fig. 7e is implemented and the other device is implemented as in Fig. 7f. Thus, the LPC domain at output 510 can represent either an LPC excitation signal or the LPC weighted signal or any other LPC domain signal.

11/28

In the second branch of encoding (ACELP / TCX) of figures 2a or 2c, the signal is preferably pre-emphasized through a filter 1 - 0.68z ^_1 before encoding. In the ACELP / TCX decoder in Fig. 2b, the synthesized signal is de-emphasized with the filter 1 / (1 —0.68z ^_1 ) The pre-emphasis can be part of the LPC 510 block where the signal is pre-emphasized before analysis in

LPC and quantification. Likewise, the de-emphasis may be part of the LPC ^-1 540 synthesis LPC block.

Fig. 2c illustrates another configuration for the implementation of Fig. 2a, but with a different arrangement of switch 521 similar to the principle of Fig. 4b.

In a preferred configuration, the first switching 200 (v-ide- Fig. Ia -or 2a) is controlled via an open loop decision (as in Fig. 4a) and the second switching is controlled via a closed loop decision (as in figure 4b f.

For example, in Fig. 2c, there is the second switch placed after the ACELP and TCX branches as in Fig. 4b. Then, in the first processing branch, the first LPC domain represents the LPC excitation and in the second processing branch, the second LPC domain represents the weighted signal LPC. That is, the first LPC domain signal is obtained by filtering through (lA (z)) to convert the residual LPC domain, while the second LPC domain signal is obtained by filtering through the filter (1 - ^ 4 (^ //)) / (1 - / zz ~ ') to convert the weighted domain to LPC.

Fig. 2b illustrates a decoding scheme

29/114 corresponding to the coding scheme of Fig. 2a. The bit stream generated by the bit stream multiplexer 800 of Fig. 2a is inserted into a bit stream demultiplexer 900. Depending on the information obtained, for example, from the bit stream via the 601 mode detection block, a decoder side switch 600 is controlled to transmit signals from the upper branch or signals from the lower branch to the bandwidth extension block 701. The bandwidth extension block 701 receives, from bit stream demultiplexer 900, , collateral information and, based on this collateral information and 601 mode decision output, reconstructs the high band based on the low band output by switching 600.

The full band signal generated by block 701 is the input to the joint stereo / surround processing stage

702, which reconstructs two stereo channels or multiple multi-channels.

Generally, the -block- 702 will carry out more channels _do_ which have been inserted in this block. Depending on the application, the input to block 702 may even include two channels, such as in a stereo mode, and may even include more channels, as long as the output on this block has more channels than the input for this block.

Switch 200 has been shown to switch between both branches so that only one branch receives a signal to process and the other branch does not receive a signal to process. In an alternative configuration, however, switching can also be arranged subsequently, for example, to audio encoder 421 and excitation encoder 522, 523, 524, which means that both branches 400, 500 process the same signal in parallel. In order not to double the bit rate, however, it is

30/114 only the signal output by one of these 400 or 500 coding branches is selected to be written in the output bit stream.

The decision stage will then operate in such a way that the signal written in the bit stream minimizes a certain cost function, where the cost function can be the bit rate generated or the perceptual distortion or a combined rate / distortion cost function. . Thus, whether in this mode or in the mode illustrated in the figures, the decision stage can also operate in a closed loop mode, in order to ensure that, finally, only the output of the coding branch is written in the bit stream it has for a certain perceptual distortion at the lowest bit rate or, for a given bit rate, has the least perceptual distortion. In closed loop mode, the feedback input can be obtained from the outputs of the three quantizer / scalimeter blocks 421, 522 and 424 in Fig.

Ia.

-Having two * switchings' in ¹ implementation, ^_ that is, the first switching 200 and the second switching 521, it is preferable that the time resolution for the first switching is lower than the time resolution for the second switching. In other words, the blocks of the input signal for the first switch, which can be switched via a switching operation, are larger than the blocks switched by the second switch operating in the LPC domain. Exemplarily, the frequency / switching domain of LPC 200 can switch blocks of a length of 1,024 samples, and the second switch 521 can switch blocks having 256 samples each.

Although some of the figures 1 through 10b are illustrated as block diagrams for an apparatus, these figures

31/114 are simultaneously an illustration of a method, where the block functionalities correspond to the steps of the method.

Fig. 3 illustrates an audio encoder for generating an encoded audio signal as an output from the first decoding branch 400 and a second encoding branch 500. In addition, the encoded audio signal preferably includes collateral information such as parameters of pre-processing, from the common pre-processing stage or, as discussed in connection with previous figures, switching control information.

Preferably, the first decoding branch is operative in order to encode an intermediate audio signal

195 according to a first coding algorithm, wherein the first coding algorithm has an _.sink _ model of information. The first encoding branch 400 generates the first encoder output signal, which is a representation of the encoded -spectral information of the ^- “audio ^- intermediate” signal

195.

In addition, the second encoding branch 500 is adapted for encoding intermediate audio signal 195 according to a second encoding algorithm, the second encoding algorithm having an information source model and generating, in a second output signal of the encoder, encoded parameters for the information source model representing the intermediate audio signal.

The audio encoder furthermore comprises the common pre-processing stage for pre-processing an audio input signal 99 to obtain the intermediate audio signal 195. Specifically, the common pre-processing stage is operative to process the audio input signal 99 so that the intermediate audio signal 195, i.e., the output of the common pre-processing algorithm is a compressed version of the audio input signal.

A preferred audio encoding method for generating an encoded audio signal comprises an encoding step 400 of an intermediate audio signal 195 according to a first encoding algorithm, the first encoding algorithm having an information sink model and generating , in a first output signal, spectral encoded information representing the audio signal, an encoding step 500 of an intermediate audio signal 195 according to a second encoding algorithm, the second _de__co_dificação algorithm having an information source model and generating, in a second output signal, encoded parameters for the information source model. representing the signal - intermediate * 195, ”and“ a common preprocessing step 100 of an audio input signal 99 to obtain the audio signal intermediate 195, in which it is processed, in the common pre-processing step of audio input signal 9 9, so that the intermediate audio signal 195 is a compressed version of the audio input signal 99, wherein the encoded audio signal includes, for a certain portion of the audio signal, both the first output signal and the second audio signal. output. The method preferably includes stepping forward by encoding a certain portion of the intermediate audio signal either using the first encoding algorithm or using the second encoding algorithm or encoding the signal using both algorithms and outputting an encoded signal, either the

33/114 result of the first coding algorithm or the result of the second coding algorithm.

Generally, the audio coding algorithm used in the first 400 coding branch reflects and models the situation in an audio sink. The audio information sink is normal for the human ear. The human ear can be modeled as a frequency analyzer. Therefore, the first decoding branch outputs coded spectral information. Preferably, the first decoding branch, furthermore, includes a psychoacoustic model to additionally apply a psychoacoustic masking limit. This limit of psychoacoustic masking is used when the spectral values of audio quantization, where, preferably, quantization is performed in such a way that a quantization noise is introduced by the quantization of spectral audio values, which are hidden below the limit of masking-psychoacoustic. “_ - - The next coding branch represents a model of information source, which reflects the generation of audio sound. Therefore, information source models can include a speech model that is reflected by an LPC analysis stage, that is, transforming a time domain signal into an LPC domain and subsequently processing the residual LPC signal, that is, the sign of excitement. Alternative sound source models, however, are sound source models to represent a certain instrument or any other sound generator, such as a specific sound source in the real world. A selection between different sound source models can be made when multiple sound source models are

34/114 available, for example, based on an SNR calculation, that is, based on a calculation, which of the source models is the most suitable for encoding a certain time portion and / or frequency portion of a signal of audio. Preferably, however, the switching between the coding branches is carried out in the time domain, that is, that a certain portion of time is encoded using a model and a certain portion of a certain time different from the intermediate signal is encoded using another coding branch .

Information source models are represented by certain parameters. In view of the speech model, the parameters are the LPC parameters and encoded excitation parameters, when considered a modern speech encoder such as AMR-WB +. The AMR-WB + comprises an ACELP encoder and a TCX encoder. In this case, the coded excitation parameters can be an overall gain, floor noise and variable length codes.- - - - - - - - -

Fig. 3b shows a decoder corresponding to the encoder shown in Fig. 3a. Generally, Fig. 3b illustrates an audio decoder for decoding an encoded audio signal to obtain a decoded audio signal 799. The decoder includes the first decoding branch 450 for decoding an encoded signal according to a first coding algorithm having an information sink model. The audio decoder, furthermore, includes a second decoding branch 550 for decoding an encoded information signal, being encoded according to a second encoding algorithm having an information source model. The audio decoder, moreover, includes a combiner for

35/114 combine output signals from the first decoding branch 450 and the second decoding branch 550 to obtain a combined signal. The combined signal, which is illustrated in Figure 3b as the decoded intermediate audio signal 699 is inserted into a common post-processing stage for post-processing of the intermediate decoded audio signal 699, which is the combined signal output by the combiner 600 such that an output signal from the common preprocessing stage is an expanded version of the combined signal. Thus, the decoded audio signal 799 has an enriched information content compared to the decoded intermediate audio signal 699. This information expansion is provided by the common post-processing stage with the help of pre / post-processing parameters that can be transmitted from an encoder to a decoder, or which may be derived from the decoded intermediate audio signal itself. Preferably, no. However, _ pre / post-processing parameters are transmitted from an encoder to a decoder, since this procedure improves the quality of the decoded audio signal.

Fig. 3c illustrates an audio encoder for encoding an audio input signal 195, which can be the same as the intermediate audio signal 195 of Fig. 3a, according to the preferred configuration of the present invention. The audio input signal 195 is present in a first domain which can, for example, be the time domain, but which can also be any other domain, such as a frequency domain, an LPC domain, a spectral domain LPC or any other domain. Generally, converting from one domain to another domain is

36/114 performed by a conversion algorithm, such as any well-known time / frequency conversion algorithms or time / frequency conversion algorithms.

An alternative transformation from the time domain, for example, in the LPC domain, is the result of the LPC filtering of a time domain signal which results in a residual LPC signal or excitation signal. Any other filtering operations producing a filtered signal, which has an impact on a substantial number of signal samples before the transformation can be used as a transformation algorithm as the case may be. Therefore, weighting an audio signal using an LPC-based weighting filter is a more comprehensive transformation, which generates a signal in the LPC domain. In a time / frequency transformation, modifying a single spectral value will have an impact on all domain values from._time before, from. transformation Similarly, - a- modification of a time domain sample will have an impact on each frequency domain sample. Similarly, a modification of a sample of the excitation signal in an LPC domain situation will, due to the length of the LPC filter, have an impact on a substantial number of samples before LPC filtering.

Similarly, a modification of a sample before an LPC transformation will have an impact on many samples obtained by that LPC transformation due to the memory effect inherent in the LPC filter.

The audio encoder of Fig. 3c includes a first encoding branch 400, which generates a first encoded signal. This first coded signal can be in a fourth

37/114 domain that is, in the preferred configuration, the domain of the time spectrum, that is, the domain that is obtained when a time domain signal is processed via time / frequency conversion.

Therefore, the first encoding branch 400 for encoding an audio signal uses a first encoding algorithm to obtain a first encoded signal, where this first encoding algorithm may or may not include a time / frequency conversion algorithm.

The audio encoder, moreover, includes a second encoding branch 500 for encoding an audio signal. The second encoding branch 500 uses a second encoding algorithm to obtain a second encoded signal, which is different from the first encoding algorithm.

The audio encoder, furthermore, includes a first switch 200 to switch between the first encoding branch 400 and the second encoding branch * 500 so that for a portion of the audio signal input, it is the first signal encoded at the output of block 400, or the second encoded signal at the output of the second encoding branch is included in an encoder output signal. Thus, when for a certain portion of the audio input signal 195, the first signal encoded in the fourth domain is included in the output signal of the encoder. The second encoded signal which is both the first signal processed in the second domain and the second signal processed in the third domain is not included in the encoding output signal. This ensures that this encoder is efficient in the bit rate. In configurations, any time portions of the audio signal that are included in two different encoded signals are small

38/114 compared to a frame length of a frame as will be discussed in connection with Fig. 3e. These small portions are useful for cross fading from one encoded signal to the other encoded signal, in the case of a switching event, in order to reduce artifacts that can occur without any cross fade. Therefore, with the exception of the cross fade region, each time domain block is represented by a coded signal from only a single domain.

As illustrated in Fig. 3-C, the second encoding branch 500 comprises a converter 510 for converting the audio signal in the first domain, i.e., signal 195 into a second domain. In addition, the second encoding branch 500 comprises a first transformation branch 522 for processing an audio signal in the second domain to obtain a first processed signal which is preferably also in the second domain so that the first Processing 522 does not perform a domain change. - - - - - - - - - - - - - - - The second encoding branch 500, furthermore, comprises a second processing branch 523, 524 that converts the audio signal in the second domain into a third domain, which is different from the first domain and which is also different from the second domain and which processes the audio signal in the third domain to obtain a second signal processed at the output of the second processing branch of 523, 524.

In addition, the second coding branch comprises a second switch 521 for switching between the first processing branch 522 and the second processing branch

523, 524 so that, for an audio signal input portion for the second encoding branch, it is the first signal

39/114 processed in the second domain and the second signal processed in the third domain, are in the second encoded signal.

Fig. 3d illustrates a corresponding decoder for decoding an encoded audio signal generated by the encoder of Fig. 3c. Generally, each block of the first domain audio signal is represented either by a second domain signal, a third domain signal or a fourth encoded domain signal, with the exception of the optional cross fade region which is preferably short in comparison to the length of a frame in order to obtain a system that is, as far as possible, at the critical sampling limit. The encoded audio signal includes the first encoded signal, a second encoded signal in a second domain, and a third encoded signal. —In a third domain, where the first encoded signal, the second encoded signal and the third encoded signal all relate to different time portions of the “give signal ^- decoded audio and in which the second domain, the third domain and the first decoded audio signal domain are different from each other.

The decoder comprises a first decoding branch for decoding based on the first encoding algorithm. The first decoding branch is illustrated at 431, 440 in Fig. 3d, and preferably comprises a frequency / time converter. The first signal is preferably encoded in a fourth domain and is converted to the first domain, which is the domain for the decoded output signal.

The decoder of Fig. 3d, furthermore, comprises a decoding branch comprising several elements. These elements are a first branch of processing

40/114 reverse 531 for reverse processing the second encoded signal to obtain a first reverse processing signal in the second domain in output block 531. The second decoding branch further comprises a second reverse processing branch 533, 534 for the reverse transformation of a third encoded signal to obtain a second reverse processing signal in the second domain, where the second reverse transformation branch comprises a converter for converting the third domain to the second domain.

The second decoding branch further comprises a first combiner 532 for combining the first processed reverse signal and the second processed reverse signal to obtain a signal in the second domain, where the combined signal is, in the first instance, only influenced by first processed reverse signal and, in a first instant, it is influenced only by the second processor reverse signal “~ ^{- -}

The second decoding branch further comprises a converter 540 for converting the combined signal to the first domain.

Finally, the decoder shown in Fig. 3d comprises a second combiner 600 for combining the first block decoded signal 431, 440 and the output signal of the converter 540 to obtain a decoded output signal in the first domain. Again, the output signal decoded in the first domain is, in the first instance, only influenced by the signal output by the converter 540 and, in another instant, it is influenced only by the first signal output decoded by block 431, 440.

41/114

This situation is illustrated, from an encoder perspective, in Fig. 3e. The upper portion of Fig. 3e illustrates the schematic representation, a first domain audio signal, such as a time domain audio signal, where the time index increases from left to right and item 3 can be considered as a stream of audio samples representing signal 195 in Fig. 3c. Fig. 3e illustrates frames 3a, 3b, 3c, 3d, which can be generated by switching between the first coded signal and the first processed signal and the second processed signal as illustrated in item 4 in Fig. 3e. The first encoded signal, the first processed signal and the second processed signals are all in different domains and in order to ensure that switching between different domains does not result in an artifact on the decoder side. Tables 3a, 3b of the time domain signal have an overlapping range which is indicated as a cross fade region, and such region * 'cross fade ^_' * exists in Table 3b and 3c. However, no cross fade region exists between the 3d and 3c frames, which means that the 3d frame is also represented by a second processed signal, that is, a signal in the third domain and there is no change of domain between the frames 3c and 3d. Therefore, in general, it is preferable not to provide a cross fade region where there is no change of domain and to provide a cross fade region, that is, a portion of the audio signal that is encoded by two subsequent encoded / processed signals when there is a change. domain, that is, a switching action on both switches. Preferably, cross fades are performed for other domain changes.

In the configuration in which the first signal

42/114 encoded or the second processed signal was generated by MDCT processing having, for example, a 50 percent overlap, each time domain sample is included in two subsequent frames. Due to the characteristics of MDCT, however, this does not result in an overhead, since the MDCT is a crucial sampling system. In this context, crucial sampling means that the number of spectral values is the same as the number of time domain values. MDCT is advantageous in that the crossover effect is provided without a specific crossover region, so that a crossover from an MDCT block to the next MDCT block is provided without any overhead that would violate the crucial sampling requirement.

Preferably, the first coding algorithm in the first coding branch is based on an information sink model and the second coding algorithm in the second coding branch is based -de ¹ in a source ^- de ^ SNR information or model. An SNR model is a model that is not specifically related to a specific sound generation mechanism, but that is a coding mode that can be selected from a plurality of coding modes based, for example, on a closed loop decision. Thus, an SNR model is any coding model available, but it does not necessarily have to be related to the physical constitution of the sound generator, but it is every parameterized coding model different from the information sink model, which can be selected by a closed-loop decision and specifically when comparing different SNR results from different models.

As illustrated in Fig. 3c, a

43/114 controller 300, 525. This controller can include the functionality of decision stage 300 of Fig. La and, additionally, can include the functionality of switching control device 525 in Fig. La. Generally, the controller is to control the first switch and the second switch in the form of an adaptive signal. The controller is operative to analyze a signal input for the first switching or output by the first or second coding branch or signals obtained by coding and decoding the first and second coding branch in relation to a target function. Alternatively, or in addition, the controller is operative to analyze the signal input for the second switching or output by the first processing branch or the second — processing branch or — having obtained the reverse processing and processing of the first processing branch and of the second processing branch again with respect to a target function.

In one configuration, the first coding branch or the second coding branch comprises a serrated introductory time / frequency conversion algorithm such as an MDCT or an MDST algorithm, which is different from a direct FFT transformation, which has no effect of serrated. In addition, one or both branches comprise a quantizer / entropy encoder block. Specifically, only the second processing branch of the second coding branch includes the time / frequency converter introducing a serrated operation and the first processing branch of the second coding branch comprises a quantizer and / or entropy encoder and does not introduce any serrated effect. The knurl

44/114 introducing the time / frequency converter, preferably, comprises a window for the application of an analysis window and an MDCT transformation algorithm. Specifically, the window is operative to apply the window function to subsequent frames in the form of overlap, so that a sample of a window signal occurs in at least two subsequent window frames.

In one configuration, the first processing branch comprises an ACELP encoder and a second processing branch comprises an MDCT spectral converter and the quantizer for quantizing spectral components to obtain quantized spectral components, where each quantized spectral component is zero _or is - defined by a quantizer index of the plurality of different possible quantizer indices.

In addition, it is preferable that the first switching

-200 -operate and I'm ‘an open loop form and the second switching operation operates in a closed loop form.

As stated earlier, both branches of encoding are operative to encode the audio signal in a block mode, where the first switching or the second switching switches in a blocking mode, so that a switching action occurs, at least, after a block of a predefined number of samples of a signal, the predefined number forming a frame length for the corresponding switching. Thus, the switching granule by the first switching can be, for example, a block of samples 2048 or 1028, and the frame length on the basis of which the first switching 200 is switching can be variable, but is preferably fixed to such a period well

45/114 long.

In contrast to the present, the block length for the second switch 521, that is, when the second switch 521 switches from one mode to another, is substantially less than the block length for the first switch. Preferably, both block lengths for switching are selected in such a way that the longest block length is an integer multiple of the shortest block length. In the preferred configuration, the block length of the first switch is 2048 or 1024 and the block length of the second switch is 1024 or more, preferably 512 and even more, preferably 256 and even more, preferably 12 8 samples, so that, at most, the second switch can switch 16 times when the first switch only switches once. The preferred maximum block length ratio, however, is 4: 1.

“In‘ another * configuration, the 300, 525 controller is operative to perform music-speech discrimination for the first switching in such a way that the decision for speech is favored with respect to a decision for music. In this configuration, the decision for speech is made even when a portion of less than 50% of a frame for switching is speech and the portion of more than 50% of the frame is music.

In addition, the controller is operative to already switch to speech mode, when a very small portion of the first frame is speech, and specifically, when a portion of the first frame is speech, that is, 50% of the length of the second smallest frame . Thus, a preferential / favored speech switching decision already switches to speech, even when,

46/114 for example, only 6% or 12% of a block corresponding to the frame length of the first switch is speech.

This procedure is preferably intended to fully exploit the bit rate while saving the capacity of the first transformation branch, which has a sonic speech core in one configuration and not to lose any quality, even for the rest of the first large frame, which is a non-speech due to the fact that the second branch of processing includes a converter and is therefore useful for audio signals that have non-speaking signals as well. Preferably, this second processing branch includes an overlap of MDCT, which is crucially sampled, and which even in small window sizes provides highly efficient and serrated-free operation due to the anti-aliasing processing time domain such as overlap and addition on the decoder side. In addition, a codefimehtõ 3e ^_ wide block for the first decoding branch which is preferably a MDCT encoding branch such as useful AAC, since the non-speech signals are usually very still and a long window of time transformation provides a high frequency resolution and therefore of high quality and additionally provides a bit rate efficiency due to a psychoacoustically controlled quantization module, which can also be applied to a coding mode based on the transformation in the second branch of processing the second branch of coding.

In view of the illustration of the decoder of Fig. 3d, it is preferable that the transmitted signal includes an explicit indicator as collateral information 4a as illustrated in Fig. 3e. This collateral information 4a is extracted by a flow analyzer

47/114 bits not shown in Fig. 3d, in order to transmit the corresponding first encoded signal, the first processed signal or the second processed signal to the correct processor, such as the first decoding branch, the first reverse processing branch or of the second branch of inverse processing in Fig. 3d.

Therefore, an encoded signal not only has the encoded / processed signals, but also includes collateral information related to those signals. In other configurations, however, there may be an implicit signaling that allows a bit stream analyzer on the decoder side to distinguish between the right signals. In view of figure 3e, it is pointed out that the first processed signal or the second processed signal is the output of the second coding branch, and therefore. the second _coded signal.

Preferably, the first decoding branch and / or the second reverse processing branch includes an MDCT transformation to convert from the spectral domain

to the domain of time. For this end , an adder in overlap is provided to perform a feature in cancellation of serrated domain in time that at the same time, provides an effect cross fade, in order to avoid O artifacts from block. Usually, O first branch in

decoding converts an encoded signal in the fourth domain to the first domain, while the second reverse processing branch performs a conversion from the third domain to the second domain and the converter subsequently connected to the first combiner provides a conversion from the second domain to the first domain so that, at the entrance of combiner 600, only

48/114 the first domain signals are present, which represents, in the configuration of Fig. 3d, the decoded output signal.

Fig. 4a and 4b illustrate two different modalities, which differ in the position of switch 200. In Fig. 4, switch 200 is positioned between an output of the common preprocessing stage 100 and the input of the two coded branches

400, 500. A configuration of Fig. 4a ensures that the audio signal is inserted in a single branch of coding, and the other branch of coding, that is not connected to the pre- common processing, not operates and therefore is off or on

a sleep mode. This configuration is preferable when the non-active coding branch does not consume energy and computational resources that are useful for mobile applications in particular, which are powered by batteries and, therefore, have the general limitation of energy consumption.

__ _, _By._ on the other hand, - however, - the configuration in Fig. -4b may be preferable when power consumption is not an issue.

In this configuration, both coding branches 400, 500 are active all the time, and only the output of the selected coding branch for a certain period of time and / or a certain frequency portion is forwarded to the bit stream formatter which can be implemented as a bitstream multiplexer 800. Thus, in the configuration of Fig. 4b, both coding branches are active all the time, and the output of a coding branch that is selected by decision stage 300 is inserted in the flow of output bits, while the output of another unselected coding branch 400 is discarded, that is, it has not entered the output bit stream, that is, the audio signal

49/114 coded.

Preferably, the second encoding rule / decoding rule is an LPC-based encoding algorithm. In LPC-based speech coding, a differentiation is made between excitation signal segments such as quasi-periodic pulses or signal portions, and excitation signal segments such as noise or signal portions. This is accomplished by very low bit rate (2.4 kbps) LPC vocoders as in

Fig. 7b. However, in medium rate CELP encoders, excitation is achieved by adding scale vectors from an adaptive codebook and a fixed codebook.

Excitation signal segments as a quasi-periodic pulse, that is, signal segments having a specific tone are encoded with different mechanisms than excitation signals such as noise. While excitation signals such as quasi-periodic pulses are connected to the speech-speaker? the ^- signs like ”noises are related to deaf speech.

For example, reference is made to figures 5a to

5d. Here, signal segments such as quasi-periodic pulses or signal portions and signal segments such as noise or signal portions are exemplarily discussed. Specifically, the speech speech as illustrated in figure 5a in the time domain and in Fig. 5b in the frequency domain is discussed as an example of a signal portion as a quasi-periodic pulse, and a speech segment as an example of a portion of signal as noise is discussed in connection with figures 5c and 5d. Speech can generally be classified as voiced, deaf, or mixed. The time and frequency domain graphs for sound and deaf segments of

50/114 sampling are shown in figures 5a to 5d. Sound speech is almost periodic in the time domain and harmoniously structured in the frequency domain, while the deaf speed is similar to random and broadband. The short-term spectrum of sonorous speech is characterized by its fine harmonic formant structure. The fine harmonic structure is a consequence of the almost periodicity of speech and can be attributed to the vibration of the vocal cords. The structure of formants (spectral envelope) is due to the interaction of the source and the vocal tracts. The vocal tract consists of the pharynx and the oral cavity. The shape of the spectral envelope that fits the short-term spectrum of sound speech is associated with the characteristics of transferring the vocal tract and the spectral inclination (6 dB / octave), due to the glottal pulse. The spectral envelope is characterized by a set of peaks that are called formants. Formants are the resonance modes of. vocal tract .-. For the average vocal tract, —there are “three to five 'formants below 5 kHz. The amplitudes and positions of the first three formants, occurring generally below 3 kHz, are very important, both in speech synthesis and perception.

Higher formants are also important for broadband and deaf speech representations. Speech properties are related to the physical speech production system as follows. Sound speech is produced by vocal tract excitation with quasi-periodic glottal air pulses generated by the vibration of the vocal cords. The frequency of the periodic pulses is referred to as the fundamental frequency or tone. Deaf speech is produced by forcing air through a constriction in the vocal tract. Nasal sounds are due to the acoustic coupling of the nasal tract to the vocal tract, θ

51/114 plosive sounds are produced by the abrupt release of air pressure that was built behind the closure in the tract.

Thus, a portion as noise of the audio signal does not show either a time domain structure as a pulse, nor a harmonic frequency domain structure, as illustrated in Fig. 5c and in Fig. 5d, which is different from the portion as pulses almost -periodic as, for example, illustrated in Fig. 5a and Fig. 5e. As will be described later, however, the differentiation between portions as noise and portions as quasi-periodic pulse can also be observed after an LPC for the excitation signal. LPC is a method that models the vocal tract and extracts it from the excitation signal of the vocal tract.

In addition, portions like quasi-periodic pulse and portions like noise can occur in a timely manner, that is, which means that one portion of the audio signal in time is noisy and another portion of the audio signal in time is almost -periódicaj that is, tonal. Alternatively, or in addition, the characteristic of a signal may be different in different frequency bands.

Thus, the determination, whether the audio signal is noisy or tonal, can also be performed at selective frequency so that a certain frequency band or several frequency bands are considered noisy and other frequency bands are considered tonal. In this case, a certain portion of the audio signal may include tonal components and noisy components.

Fig. 7 illustrates a linear model of a speech production system. This system presupposes two-stage excitation, that is, a set of impulses for sound speech, as shown in figure 7c, and a random noise for deaf speech, as

52/114 as shown in figure 7d. The vocal tract is modeled as a total pole filter 70 that processes the pulses of Fig. 7c or Fig.

7d, generated by the glottal model 72. Thus, the system of figure 7a can be reduced to a total pole filter model of Fig. 7b, having a gain stage 77, a progressive path 78, a return path of 79 years , and an addition stage of 80. On return path 79, there is a forecast filter 81, and the entire source model synthesis system illustrated in Fig. 7b can be represented using z domain functions, as follows:

S (z) = g / (lA (z)) -X (z), where g represents the gain, A (z) is the prediction filter as determined by an LP analysis, X (z) is the excitation signal , and S (z) is the synthesis speech output.

Figures 7c and 7d give a description of the graphical time domain of speech synthesis using the linear source system model. This - system and -the “excitation parameters in the above equation are unknown and must be determined from a finite set of speech samples.

The coefficients of A (z) are obtained using a linear prediction of the input signal and a quantization of the filter coefficients. In a linear p-th forwarding predictor, the present speech sequence sample is predicted from a linear combination of past samples of p. The predictor coefficients can be determined by well-known algorithms, such as the Levinson-Durbin algorithm, or in general an autocorrelation method or a reflection method.

Fig. 7e illustrates a more detailed implementation of the LPC 510 analysis block. The audio signal is

53/114 inserted in a filter determination block that determines the filter information A (z). This information outputs as the short-term forecast information needed for a decoder. The short-term forecast information is required by the actual forecast filter 85. In a subtractor 86, a current sample of the audio signal is inserted and a forecast value for the current sample is subtracted so that, for this sample, the forecast error signal is generated on line 84. A sequence of such forecast error signal samples is schematically illustrated in figures 7c and 7d. Therefore, Fig. 7a, 7b can be considered as a kind of signal as a rectified pulse.

While Fig. 7e illustrates a preferred way to calculate the excitation signal, Fig. 7f illustrates a preferred way to calculate the weighted signal. In contrast to figure 7e, filter 85 is different, when γ is different from 1. A

A value less than 1 is preferred for γ. In addition, block 87 is present, εμέ preferably a number less than 1.

Generally, the elements in Fig. 7e and 7f can be implemented as in 3GPP TS 26.190 or 3GPP TS 26.290.

Fig. 7g illustrates reverse processing, which can be applied to the decoder side, as in element 537 of figure 2b. In particular, block 88 generates an unweighted signal from the weighted signal and block 89 calculates an excitation of the unweighted signal. Generally, all signals except the unweighted signal in Fig. 7g are in the LPC domain, but the excitation signal and the weighted signal are different signals in the same domain. Block 89 outputs an excitation signal that can then be used with the output of block 536. Then, the

54/114 common reverse LPC transformation can be performed in block 540 of

Fig. 2b.

Subsequently, an analysis-by-synthesis CELP encoder will be discussed in connection with Fig. 6 in order to illustrate the modifications applied to this algorithm. This CELP encoder is discussed in detail in thepeech Coding: A Tutorial Review,

Andreas Spanias, Proceedings of the IEEE, Vol. 82, No. 10, October 1994, pages 1541-1582. The CELP encoder, as shown in Fig. 6 includes a long-term forecast component 60 and a short-term forecast component 62. In addition, a codebook that is indicated at 64 is used. A perceptual weighting filter

W (z) is implemented in 66, and an error minimization controller is provided in __68. s_ (n) is the time domain input signal.

After being weighted perceptually, the weighted signal is inserted in a subtractor 69, which calculates the error between the weighted synthesis signal ^- in the output “dò block * 66 and“ the original weighted signal s _w (n). Generally, the coefficients of the short-term forecast filter A (z) are calculated by an analysis stage LP and their coefficients are quantized in  (z), as shown in figure 7e. The long-term forecast information A _L (z), including long-term forecast gain and vector quantization index, that is, the codebook references are calculated on the forecast error signal for at the output of the analysis stage LPC referred to as 10a in Fig. 7e. The LTP parameters are the tone and gain delay. In CELP this is usually implemented as an adaptive codebook containing the past excitation signal (not the residual). The delay and gain of adaptive CB are found by minimizing the weighted square mean error (cycle tone search

55/114 closed).

The CELP algorithm then encodes the residual signal obtained after short-term and long-term predictions using a codebook of, for example, Gaussian sequences. The LPC algorithm, where A stands for Algebraic has a specific codebook designed algebraically.

A codebook can contain more or less vectors where each vector is a few long samples. A gain factor g scales the code vector and the acquired code is filtered by the long-term forecast synthesis filter and short-term forecast synthesis filter. The optimal code vector is selected so that the error of the perceptually weighted square mean at the output of subtractor 69 is minimized. The research process at CELP is done by an analysis-by-synthesis optimization as illustrated in Fig. 6.

·· —- For specific cases, when a —frame is a mixture of soft and loud speech or when speech about music occurs, a TCX encoding may be more appropriate for the excitation code in the LPC domain. TCX encoding processes the weighted signal in the frequency domain without making any assumptions about excitation production. TCX is therefore more generic than CELP encoding and is not restricted to a model of deafness or excitation sound source. TCX is also a source-oriented model encoding using a linear predictive filter for modeling the signal formants like speech.

In coding as AMR-WB +, a selection between the different models of TCX and ACELP occurs as it is known from

56/114 of the AMR-WB + description. The TCX modes are different in that the length of the discrete block Fourier Transform is different for the different modes and the best mode can be selected by a synthesis analysis approach or by a direct feedforward mode.

As discussed in connection with Fig. 2a and 2b, the common pre-processing stage 100 preferably includes a multi-channel set (surround / joint stereo device) 101 and, in addition, a bandwidth extension stage 102.

Correspondingly, the decoder includes a bandwidth extension stage 701 and a subsequently connected multi-channel set stage 702. Preferably, the multi-channel set stage 101 is, with respect to the encoder, connected before the bandwidth extension stage 102, and, on the decoder side, the bandwidth extension stage-701 is connected before the multi-channel array stage ^- 702 with respect to the signal processing direction .

Alternatively, however, the common preprocessing stage can include a multi-channel set stage without the subsequently connected bandwidth extension stage or a bandwidth extension stage without a multi-channel set stage connected.

A preferred example for a multi-channel set stage on the encoder side 101a, 101B and on the decoder side 702a and 702b is illustrated in the context of Fig. 8. A number of original input channels E is inserted into the downmixer 101a for that the downmixer manages a number of transmitted channels K, where the number K is greater than or equal to one and is less than or equal to E.

57/114

Preferably, input channels E perform input on a multi-channel set parameter analyzer 101b that generates parametric information. This parametric information is preferably encoded entropically such as by a difference encoding and subsequent Huffman encoding or, alternatively, subsequent arithmetic encoding. The block-encoded parametric information output 101b is transmitted to a parameter decoder 702b which can be part of item 702 in Fig. 2b. The parameter decoder 702b decodes the transmitted parametric information and forwards the decoded parametric information to the upmixer 702a. The upmixer 702a receives the transmitted K channels and generates a number of output channels L, where the number of L is greater than or equal to K and less than or equal to E.

Parametric information can have level differences, inter-channel, inter-channel time differences *, inter-channel phase differences and / or inter-channel coherence measurements as is known from the BCC technique or as it is known and described in detail in the MPEG surround standard. The number of channels transmitted can be a single mono channel for applications with ultra-low bit rate or it can include a compatible stereo application or it can include a compatible stereo signal, that is, two channels. Typically, the number of input channels E can be five or maybe even more. Alternatively, the number of input channels E can also be audio objects E as it is known in the context of spatial audio object coding (SAOC).

In an implementation, the downmixer performs a weighted or unweighted sum of the initial input channels

58/114 original E or an addition of the input audio objects E. In the case of audio objects such as input channels, the multi-channel set parameter analyzer 101b will calculate the audio object parameters such as a correlation matrix between audio objects, preferably for each portion of time and even more preferably for each frequency band. For this purpose, the frequency range can be divided into at least 10 and preferably 32 or 64 frequency bands.

Fig. 9 illustrates a preferred configuration for implementing the bandwidth extension stage 102 in the

Fig. 2a and corresponding bandwidth extension stage

701 in Fig. 2b. On the encoder side, the bandwidth extension block 102 preferably includes a low pass filter block 102b, a downsarapler block, which follows the lowpass, or which is part of the inverse QMF, which acts only on half of the QMF bands, .— and a high band 102a analyzer. The original audio signal input for the bandwidth extension block 102 is low-pass filtered to generate the low-band signal, which is then inserted into the coding and / or switching branches. The low pass filter has a cutoff frequency that can be in the range of 3kHz to 10kHz. In addition, the bandwidth extension block 102 includes a high-band analyzer for the calculation of the bandwidth extension parameters such as a spectral envelope parameter information, a floor noise parameter information, an information of the reverse filtering parameter, greater parametric information related to certain harmonic lines in the high band and additional parameters, as discussed in detail in the MPEG-4 standard, in

59/114 chapter related to spectral band replication.

On the decoder side, the bandwidth extension block 701 includes a patcher 701a, a regulator 701b and a combiner 701c. The combiner 701C combines the decoded low band signal and the reconstructed high band signal output adjusted by regulator 701b. The input for regulator 701b is provided by a patcher which is operated to obtain the high band signal from the low band signal such as by spectral band replication or, in general, by bandwidth extension. The splice performed by the patcher 701a can be a splice performed in a harmonic or non-harmonic way. The signal generated by the patcher

701a is subsequently adjusted by regulator 701b using the transmitted parametric bandwidth extension information.

As shown in Fig. 8 and Fig. 9, the described blocks - can have a mode control input in a preferred configuration. This mode control input is derived from decision stage 300. In such a preferred configuration, a characteristic of a corresponding block can be adapted for the decision stage output, that is, if, in a preferred configuration, the decision for the speech or a decision for the song is made for a certain amount of time of the audio signal. Preferably, the mode control refers only to one or more of the functionality of these blocks, but not to all the functionality of the blocks. For example, the decision may influence only the patcher 701a, but it may not influence the other blocks in Fig. 9, or it may, for example, influence only the parameter analyzer of the multi-channel set 101b in Fig. 8

60/114 but not the other blocks in figure 8. This implementation is preferably such that greater flexibility and higher quality and lower bit rate signal output is achieved by providing flexibility in the common pre-processing stage. On the other hand, however, the use of algorithms in the common preprocessing stage for both types of signals allows an efficient coding / decoding scheme to be implemented.

Fig. 10 and Fig. 10b illustrate two different implementations of decision stage 300. In Fig. 10, an open cycle decision is indicated. Here, the signal analyzer 300a at the decision stage has certain rules in order to decide whether a certain portion of time or a certain portion of frequency of the input signal has a characteristic that requires that this portion of the signal to be encoded by the first encoding branch 400 or by the second coding branch 500. For this purpose, the signal analyzer 300a can analyze the audio input signal for the ^- common preprocessing stage or can analyze the audio signal output by the common preprocessing stage, i.e. that is, the intermediate audio signal or can analyze an intermediate signal within the common pre-processing stage such as the output of the downmix signal which can be a mono signal or which can be a signal having k channels indicated in Fig. 8. On the output side, signal analyzer 300a generates the switching decision to control switch 200 on the encoder side and the corresponding switch 600 or combiner 600 on the decoder side.

Although not discussed in detail for the second switch 521, it should be noted that the second switch 521 can be positioned similarly as the first switch 200,

61/114 as discussed in connection with Fig. 4a and Fig. 4b. Thus, an alternative switching position 521 in Fig. 3c is the output of both processing branches 522, 523, 524, so that both processing branches operate in parallel and only the output of one processing branch is written in a bit stream via a previous bit stream that is not illustrated in Fig. 3d.

In addition, according to the combiner 600, it may have specific cross fading functionality, as discussed in Fig. 4c.

Alternatively or in addition, the first 532 combiner can have the same cross fading functionality. In addition, both combiners may have the same cross fading functionality or may have different cross fading functionality or may not have cross fading functionality at all so that both combiners are switching without any additional cross fading functionality.

-— ------- As-discussed-earlier, both ^ switching can be controlled via an open cycle decision or a closed cycle decision, as discussed in connection with Fig. 10a and Fig. 10b, where the controller 300, 525 of Fig. 3c can have different or the same functionalities of both switching.

In addition, a time warp functionality that is signal-adaptive can exist not only in the first coding branch or first decoding branch, but it can also exist in the second processing branch of the second coding branch on the encoder side, as well as on the decoder side. Depending on the processed signal, both time deformation features may have the same time deformation information so that the same deformation

62/114 time applies to signals in the first domain and the second domain. This saves processing load and can be useful in some examples, in cases where subsequent blocks have a similar time warping characteristic. In alternative configurations, however, it is preferable to have independent time strain estimators for the first branch of coding and the second processing branch in the second branch of coding.

The inventive encoded audio signal can to be stored in a digital storage medium or can to be transmitted in a means of transmission such as a means in streaming without wire or a wired transmission medium, such as the

Internet.

In a different configuration, switching 200 from

Fig. 1a and 2a switches between the two coding branches 400, 500.

_In a. another configuration, __ can .... _ there are additional coding branches, such as a third coding branch or even a fourth coding branch or even more coding branches. On the decoder side, switch 600 of Fig. Lb or 2b switches between the two decoder branches 431, 440 and 531, 532, 533,

534, 540. In another configuration, there may be additional decoding branches, such as a third decoding branch or even a fourth decoding branch or even more decoding branches. Similarly, the other 521 or 532 switches can switch between more than two different encoding algorithms, when such additional encoding / decoding branches are provided.

Fig. 12A illustrates a preferred configuration of

63/114 is an encoder implementation, and Fig. 12B illustrates a preferred configuration of corresponding decoder implementation. In addition to the elements discussed above with respect to the corresponding reference numbers, the configuration of Fig. 12A illustrates a separate psychoacoustic module 1200 and, furthermore, illustrates a preferred implementation of the encoder tools illustrated in block 421 of Fig. 11A. These additional tools are a temporal noise modeling tool (TNS) 1201 and a mid / side (M / S) coding tool 1202. In addition, additional element functionality

421 and 524 are illustrated in block 421/542 as a combined implementation of scaling, noise filling analysis, quantization, arithmetic coding of spectral values.

In the implementation of the corresponding decoder Fig ..,. 12B, additional elements are illustrated, which are a decoding tool M / S 1203 and a '' TNS decoder tool 1204. In addition, a bottom postfilter not shown in the previous figures is indicated in 1205. The transition window block 532 corresponds to element 532 in Fig. 2B, which is illustrated as a switch, but which performs a kind of cross fading that can be a crucially sampled cross fading or cross fading excess. The latter is implemented as an MDCT operation, in which two serrated time portions are superimposed and added. This crucially sampled transition processing is preferably used where appropriate, since the total bit rate can be reduced without loss of quality. 0 additional transition window block

600 corresponds to combiner 600 in Fig. 2B, which is again

64/114 illustrated as a switch, but it is clear that this element performs a type of cross fading either crucially sampled or not crucially sampled, in order to avoid block artifacts and, specifically switching artifacts, when a block was processed in the first branch and another block was processed in the second branch. When, however, the processing in both branches is perfectly matched with its other, then the cross fading operation can degrade to a difficult switching, while a cross fading operation is understood as

soft switching between two branches. allows the O concept at Fig. 12A and 12B signal coding having an arbitrary mixing of speech and

* audio content, and this concept is comparable to or better than the best coding technology that can be tailored, specifically for the coding of both speech and general audio content. The general structure of the encoder * and - the decoder can be described in that there is a common pre / post processing that consists of a functional MPEG surround unit (MPEGs) to handle stereo or multi-channel processing and an enhanced SBR unit (eSBR), which deals with the parametric representation of higher audio frequencies in the input signal. So, there are two branches, one consisting of an advanced modified audio encoding tool (AAC) path and the other consisting of a base path (LP or LPC domain) of linear prediction encoding, which in turn presents both a frequency domain representation as well as a residual LPC time domain representation. All spectra transmitted for both AAC and LPC are represented in the

65/114 MDCT domain following quantization and arithmetic coding. The time domain representation uses an LPC excitation coding scheme. The basic structure is shown in

Fig. 12A for the encoder and Fig. 12B for the decoder. The data flow in this diagram is from left to right, top to bottom. The functions of the decoder are to find the quantized audio spectrum description or time domain representation in the bitstream payload and to decode the quantized values and other reconstruction information.

In the case of transmitted spectral information, the decoder must reconstruct the quantized spectra, process the reconstructed spectrum through any tools that are atoms in the load of the bit stream, a; order to arrive at the actual signal spectrum, as described by the input bitstream payload, and, finally, convert the frequency domain spectrum into the time domain. Following the initial reconstruction and ~ enlargement of the * - - do reconstruction ^- '“spectroj— -exist —- optional tools that modify one or more of the spectra, in order to provide more efficient coding.

In the case of representation of the transmitted time domain signal, the decoder must reconstruct the quantized time signal, process the reconstructed time signal through any tools that are active in the bitstream load, in order to arrive at the domain signal real-time as described by the input bitstream payload.

For each of the optional tools that operate on the signal data, the option for pass through is maintained, and in all cases where processing is omitted,

66/114 spectrum or time at its entry is passed directly through the tool without modification.

In places where the bit stream changes its signal representation from the time domain to the representation of the frequency domain or from the LP domain to the non-LP domain or vice versa, the decoder should facilitate the transition from one domain to another through the appropriate transition overlay addition window.

ESBR and MPEGS processing is applied in the same way for both coding paths after transition handling.

The input to the bitstream load demultiplexer tool is a bitstream load. The demultiplexer separates the bitstream payload for the parts of each tool and provides for each of the tools the bitstream payload information related to that * tool-_________ _: _______________

The outputs of the bitstream load demultiplexer tool are:

- Depending on the type of core coding in the current framework, either:

the quantized and noise-free coded spectra represented by (s)

- scale factor information

- arithmetically coded spectral lines

- or: linear prediction parameters (LP) with an excitation signal represented by:

- quantized and coded spectral lines

67/114 arithmetically (encoded transformation excitation, TCX) or

- encoded ACELP time domain excitation

Spectral noise fill information (optional)

- M / S decision information (optional)

- Temporal noise modeling (TNS) information (optional)

- The filter bank control information

- Time deformation control information (TW) (optional)

- Enriched spectral bandwidth replication control information (eSBR)

The control information of MPEG Surround (MPEGS)

The scale decoder tool has a scale factor — le-va_information, _a_par_tir_ of the bitstream load demultiplexer, analyzes this information and decodes it using Huffman and DPCM encoded scale factors.

The entry for the scale factor silent decoding tool is:

The scaling factor information for the noise-encoded spectra

The output of the scale factor silent decoding tool is:

The entire decoded representation of the scale factors:

The silent decoding tool

68/114 spectral takes information from the bitstream charge demultiplexer, analyzes that information, decodes the arithmetically encoded data, and reconstructs the quantized spectra. The entry for this silent decoding tool is:

- Silently encoded spectra

The output of this silent decoding tool is.

- The quantized values of the spectra

The inverse quantization tool takes the quantized values for the spectra and converts the whole values for the reconstructed non-scale spectra. This quantizer is a companding quantizer (compression), whose compression factor depends on the coding mode of the chosen core.

~ - A ^- entry for “the tool ^ Inverse ^ Quantizer is:

- Quantized values for the spectra

The output of the inverse quantizer tool is:

The inverse quantized spectra are not dimensioned,

The noise filling tool is used to fill the spectral gaps in the decoded spectra, which occur when the spectral values are quantized to zero, for example, due to a strong restriction in the demand for bits in the encoder. The use of the noise fill tool is optional.

The inputs for the noise fill tool are:

69/114

The inverse quantized spectra are not dimensioned,

- Noise filling parameters

- The entire decoded representation of the scale factors

The outputs for the noise fill tool are:

- Inversely quantized spectral values not dimensioned for spectral lines that were previously quantized to zero.

- Modified entire representation of scale factors

The resizing tool converts the entire representation of the scale factors to the actual values, and multiplies the inverse quantized spectra not dimensioned by the ‘relevant scale factors. - - -

The inputs for the scale factor tool are:

- The entire decoded representation of the scale factors

- The inverse quantized spectra are not dimensioned

The output of the scale factor tool is:

- Spectra inversely quantized in scale For an overview of the M / S tool, refer to the ISO / IEC 14496-3 standard, subitem 4.1.1.2.

For an overview of the temporal noise modeling (TNS) tool, see ISO / IEC 1449670/114

3, sub-item 4.1.1.2.

The filter bank / block switching tool applies the inverse of the frequency mapping that was performed on the encoder. Ά Modified inverse discrete cosine transformation (IMDCT) is used for the filter bank tool. The IMDCT can be configured to support the spectral coefficients 120, 128, 240, 256, 320, 480, 512, 576, 960, 1024 or

1152.

The entries for the filter bank filter tool is (are):

time domain

- Spectra (inversely quantized)

- The filter bank control information

The output (s) from the database tool

The reconstructed audio signal (s) from

The defaced time filter bank / block switching bank filter tool replaces the normal / block switching tool when the time warp mode is enabled. The filter bank is the same (IMDCT) as the normal filter bank. In addition to windowed time domain samples, they are mapped from the disfigured time domain to the linear time domain by varying time resampling.

The entries for the defaced time filter bank tools are:

- Inverse quantized spectra

- The filter bank control information

71/114

- Disfigured time tracking information

The output (s) from the filter bank tool is (are):

- 0 (s) audio signal (s) reconstructed from linear time domain.

The highlighted SBR (eSBR) tool regenerates the high bandwidth of the audio signal. It is based on the replication of the truncated harmonic sequences during encoding. It adjusts the spectral envelope of the generated high band, applies reverse filtering, and adds noise and sinusoidal components in order to recreate the spectral characteristics of the original signal.

The entry for the eSBR tool is:

- Quantized envelope data

- Misc control data.

A time domain signal from the core decoder — from AAC - _ “~” _ _ - ·

The eSBR tool output is:

- a time domain signal or a QMF domain representation of a signal, for example, if the MPEG Surround tool is used.

The MPEG Surround tool (MPEGS) produces multiple signals from one or more input signals by applying a sophisticated upmix procedure to the input signal (s) controlled by appropriate spatial parameters. In the context of USAC, MPEGs are used to encode a multi-channel signal, by transmitting parametric collateral information alongside a transmitted downmix signal.

72/114

The entry for the MPEGS tool is:

- a downmix time domain signal; or

- a QMF representation and domain of a downmix signal from the eSBR tool

The output of the MPEGS tool is:

- a multi-channel time domain signal

The Signal Classifier tool analyzes the original input signal and generates from it the control information that triggers the selection of the different coding modes. The analysis of the input signal is implementation dependent and will attempt to choose the ideal coding core mode for a given input signal frame. The output of the signal classifier can (optionally) also be used to influence the behavior of other tools, for example, MPEG -Surround, enhanced SBR ', defaced time filter and others.

The entry for the Signal Classifier tool is:

- the original input signal unchanged

- - parameters dependent on additional implementation

The output of the Signal Classifier tool is:

- a control signal to control the selection of the core codec (non-LP filtered frequency domain coding, LP filtered frequency domain coding or LP filtered time domain coding)

According to the present invention, the time / frequency resolution in block 410 of Fig. 12A and in converter 523 of

73/114

Fig. 12A is dependent on the control of the audio signal. The interrelation between the window length, transformation length, time resolution and frequency resolution is illustrated in figure 13A, where it becomes clear that, for a long window length, the time resolution is low, but the frequency resolution it gets high, and for a small window length, the time resolution is high, but the frequency resolution is low.

In the first coding branch, which is preferably the AAC coding branch indicated by elements 410,

1201, 1202, 4021 of Fig. 12A, different windows can be used, where the window shape is determined by a signal analyzer which is preferably encoded in the signal classifier block 300, but which can also be a module separate. The encoder selects one of the windows illustrated in Fig. 13B, which have — different 'time' / frequency resolutions. The "time / frequency resolution of the first long window, the second window, the fourth window, the fifth window and the sixth window is equal to 2,048 sample values for a transformation length of

1.02-4. The small window illustrated in the third line of Fig. 13B has a time resolution of 256 sample values corresponding to the size of the window. This corresponds to a transformation length of 128.

Similarly, the last two windows have a window length of 2,304, which is a better frequency resolution than the window in the first line, but a smaller time resolution. The transformation length of the windows in the last two lines is equal to 1,152.

74/114

In the first branch of coding, the sequences of different windows, which are erected from the transformation windows in Fig. 13B, can be constructed. Although in Fig.

13C only a short sequence is illustrated, while the other sequences consist of only a single window, larger sequences consisting of more windows can also be constructed.

Note that according to Fig. 13B, for the smallest number of coefficients, that is, 960 instead of 1,024, the time resolution is also less than the corresponding largest number of coefficients, such as 1024.

Fig. 14A - 14G illustrates different resolutions / window sizes in the second coding branch. In a preferred embodiment of the present invention, the second branch of coding has a first branch of processing, which is an ACELP 526 time domain encoder, and the second processing branch comprises a “filter” bank 523 7 In this branch, a super frame of, for example, 2,048 samples, is sub-divided into frames of 256 samples. Individual 256 sample frames can be used separately, so that a sequence of four windows, each window spanning two frames, can be applied when an MDCT with a 50 percent overlap is applied. Then, a high time resolution is used as illustrated in Fig.

14D. Alternatively, when the signal allows for longer windows, the sequence as in Fig. 14C can be applied, where a double window size is applied having 1,024 samples for each window (medium windows), so that a window covers four frames and there are a 50 percent overlap.

Finally, when the signal is such that a window

75/114 long can be used, this long window spans 4,096 samples again with a 50 percent overlap.

In the preferred configuration, in which there are two branches, where a branch has an ACELP encoder, the position of the ACELP frame indicated by A in the super frame can also determine the size of the window applied to two adjacent TCX frames indicated by T in Fig. 14E. Basically, there is an interest in using long windows whenever possible. However, short windows must be applied when a single T frame is between two A frames. Medium windows can be applied when there are two adjacent T frames. However, when there are three adjacent T frames, a corresponding larger window cannot be efficient. due to the added complexity. Therefore, the third T frame, although not preceded by an A frame, can be processed by a short window. When the entire super frame only has T-frames, then a long-window can be applied.

Fig. 14F illustrates several alternatives for windows, where the window size is always 2x the lg number of spectral coefficients due to a preferred 50 percent overlap. However, other percentages of overlap for all branches of coding can be applied so that the relationship between window size and transformation length can also be different from two and even approach one when no serrated is applied. time domain.

Fig. 14G illustrates the rules for building a window based on the rules given in figure 14F. The ZL value illustrates zeros at the beginning of the window. The value L illustrates a number of window coefficients in a serrated zone. The values in the

76/114 M portion are 1 values without introducing any serrations due to an overlap with an adjacent window, which has zero values in the portion corresponding to Μ. The M portion is followed by the right overlapping zone R, which is followed by a ZR zone of zeros, which would correspond to a M portion of a subsequent window.

Reference is made to the annex below, which describes a preferred and detailed implementation of an inventive audio encoding / decoding scheme, particularly with respect to the decoder side.

Attachment

1. Windows and window sequence

Quantization and coding are done in the frequency domain. For this purpose, the time signal is mapped to the frequency domain in the encoder. The decoder performs Ό reverse mapping; as ^- described in subitem 2.

Depending on the signal, the encoder can change the time / frequency resolution using three different window sizes:

2304, 2048 and 256. To switch between windows, the LONG_START_WINDOW, LONG_STOP_WINDOW, START_WINDOW_LPD, STOP_WINDOW_1152, STOP_START_WINDOW and transition windows are used

STOP_START_WINDOW_1152. Table 5.11 lists the windows, specifies the corresponding transformation length and shows the windows format schematically. Three transformation lengths are used: coefficients 1152, 1024 (or 960) (referred to as long transformation) and 128 (or 120) (referred to as short transformation).

Window sequences are composed of windows in

77/114 a way that a raw_data_block always contains data representing 1024 (or 960) output samples. The windowjsequence data element indicates the window sequence that is actually used. Fig. 13C lists how the window sequences are composed of individual windows. See sub-item 2 for more detailed information about the transformation and the windows.

1.2 Scale factor and grouping bands

See ISO / IEC 14496-3, subpart 4, subitem

4.5.2.3.4

As explained in ISO / IEC 14496-3, subpart 4, sub-item 4.5.2.3.4, the width of the scale factor bands is built to imitate the crucial bands of the human auditory system. For this reason, the number of scale factor bands in a spectrum and their width depend on the transformation length and the sampling frequency. From table 4.110 to Table 4.128, in- .ISO / IEC- 14496-3, - subpart 4, ~ subitem 4.5.4, list the offset to the beginning of each scalar factor band.

transformation lengths 1024 (960) and 128 (120) and on the sampling frequencies. Tables originally designed for LONG_WINDOW, LONG_START_WINDOW and LONG_STOP_WINDOW are also used for START_WINDOW_LPD and STOP_START_WINDOW. The shift tables for STOP WINDOW_1152 and STOP_START_WINDOW_1152 are from Table 4 to Table 10.

1.3 Decoding of lpd_channel_stream ()

The bit stream element lpd_channel_stream () contains all the information needed to decode a linear predicted domain encoded signal frame. It contains the payload for an encoded signal frame that has been

78/114 encoded in the LPC domain, that is, including an LPC filtering step. The residue of this filter (called excitation) is then represented either with the help of an ACELP module or in the transformation domain MDCT (encoded transformation excitation, TCX). To allow a closer adaptation to the signal characteristics, a frame is divided into four smaller units of equal size, each of which is encoded, either with an ACELP or TCX coding scheme.

This process is similar to the coding scheme described in 3GPP TS 26.290. Inherited from this document, it is a slightly different terminology, where a super frame means a segment of the 1024 sample signal, where a frame is exactly one quarter of this, that is, 256 samples. Each of these frames is subdivided into four sub frames of equal length. Note that this sub-chapter adopts this terminology _ «, - - - - - * 1.4 Definitionsj Elements Ide Data acelp_core_mode This bit field indicates the exact bit allocation scheme in case ACELP is used as a lpd encoding mode .

lpd mode The bit field mode defines the encoding modes for each of the four frames within a super frame of lpd_channel_stream () (corresponds to an AAC frame). The encoding modes are stored in an array mod [] and can take values from 0 to 3. The mapping of lpd_mode to mod [] can be determined in Table 1 below.

Table 1 - Mapping of encoding modes for lpd_channel_s tream ()

meaning of bits in field mode Appetizer

79/114

bit mod [] remains s cen t es lpd_mode bit 4 bit 3 bit 2 bit 1 bit 0 0.. 15 0 mmod [ mmod [ mmod [ mmod [ 3] 2] 1] 0] 16..19 1 0 0 mmod [ mmod [ mod [1] = 2 3] 2] mod [0] = 2 20..23 1 0 1 mmod [ mmod [ mod [3] = 2 1] 0] mod [2] = 2 24 1 1 0 0 0 mod [3] = 2 mod [2] = 2 - - - - mod [l] = 2 „.. mod [0] = 2 25 1 1 0 0 1 mod [3] = 3 - - - - mod [2] = 3 -mod [-1-] = 3 · - mod [0] = 3 26..31 reserved

mod [0 .. 3] The values in array mod [] indicate the respective encoding modes in each frame:

Table 2 - Coding modes indicated by mod []

mod value [x] encoding mode in picture Flow element bits 0 ACELP Acelp coding () 1 a TCX frame tcx coding ()

80/114

2 TCX covering half a super painting tcx_coding () 3 TCX covering all super frame tcx_coding ()

acelp_coding () Syntax element that contains all the data to decode an excitation frame from

LPC.

tcx_coding () Syntax element that contains all the data to decode an MDCT-based encoded transformation excitation frame (TCX).

first_tcx_flag Flag indicating whether the current processed TCX frame is the first in the super frame.

lpc_data () Syntax element that contains all data to decode all parameter settings of - £ Liter “of ~ LPC 'required to decode the current superframe. ______ _ ____._ · ____— first_lpd_flag Flag indicating whether the current super frame is the first in a sequence of super frames that are encoded in the LPC domain. This flag can also be determined from the history of the bit stream element core_mode (core_mode0 and core_model in the case of a channel_pair_element) according to Table 3.

Table 3 - Definition of f irst_lpd._f lag

core_mode previous frame (super core mode of the current frame (super f irs t_lpd_flag painting) painting)

81/114

0 1 1 1 1 0

last_lpd_mode Indicates the lpd_mode of the previously decoded frame.

1.5 Decoding Process

In lpd_channel_stream, the decoding order is:

Get acelp_core_mode

Get lpd_mode and determine from it the content of the auxiliary variable []

Get the data acelp_coding or tcx_coding, depending on the contents of the auxiliary variable [] mod

Get lpc_data _ _ __ .......

- - - · * 1.6 mode coding combinations

ACELP / TCX. In - analogy · a * ’['8 J' section 5.2.2, there are 2 6 allowed combinations of ACELP or TCX within a..super — frame - of a lpd_channel_stream load. One of these 26 mode combinations is signaled in the bit stream element lpd_mode. The mapping of lpd_mode to the actual encoding modes of each frame in a sub-frame is shown in Table 1 and Table 2.

Table 4 - scale factor bands for a window length of 2304 for

STOP_START_1152_WINDOW and STOP_1152_WINDOW at 44.1 and 48 kHz fs [kHz]

44.1.48

82/114

in a lon swb g window 49 swb swb offset long window n n 1 4 2 8 3 12 4 16 5 20 6 24 7 28 8 32 9 36 10 40 11 48 12 56 13 64 14 72 15 - - 80 - - · 16 88 17 96 18 108 19 120 20 - - - 132 - ~ 21 144 22 160 23 -17 6 ---- 24 196

Table

swb swb_offse t long wi ndow 9 R 91 Λ 26 240 27 264 28 292 29 320 30 352 31 384 32 416 33 448 34 480 35 512 36 544 37 576 38 608 39 640 40 - · 672 - · 41 704 42 736 43 768 44 800 45 "---- 832 ~ ~ 46 864 47 896 -48 -92 8 1152

scale factor bands for 2304 window length for

STOP START 1152 WINDOW and STOP 1152 WINDOW at 32 kHz

fs [kHz] 32 in a lon swb g window 51 swb swb offset long window n n

Swb swb offse t long wi ndow 9fi 9zin

83/114

1 4 2 8 3 12 4 16 5 20 6 24 7 28 8 32 9 36 10 40 11 48 12 56 13 64 14 72 15 80 16 88 17 96 18 108 19 120 20 132 21 144 22 160 23 17 6 24 196 25 216

27 264 28 292 29 320 30 352 31 384 32 416 33 448 34 480 35 512 36. 544 37 576 38 608 39 640 40 672 41 704 42 736 43 768 44 800 45 832 46 8 64 47 896 48 928 '49 > 960 50 992 1152

__ ___Table 6 - scale factor bands for 2304 window length for

STOP START 1152 WINDOW and STOP 1152 WINDOW at 8 kHz

fs [kHz] 8 in a lon swb 40 g window swb swb offs et_long_ window n n 1 12 2 24 3 36 4 48 5 60 6 72 7 84 8 96 9 108 10 120 11 132 12 144 13 156

swb Swb offse t long wi ndow 91 9fifi 22 308 23 328 24 348 25 372 26 396 27 420 28 448 29 476 30 508 31 544 32 580 33 620 34 664

84/114

14 172 15 188 16 204 17 220 18 236 19 252 20 268

35 712 36 764 37 820 38 880 39 944 1152

Table 7 - bands of scale facts for 2304 window length for

STOP_START_1152_WINDOW and STOP_1152_WXNDOW

11,025, 12 and 16 kHz

f s [kHz] 11,025, 12, 16 on one swb long 43 window swb swb offse t long wi ndow n n i - . _ - 8 2 16 3 24 • 4 —--_— -32 ------- 5 40 6 48 7 56 8 64 9 72 10 80 11 88 12 100 13 112 14 124 15 136 16 148 17 160 18 172 19 184 20 196 21 212

Swb swb offse t long wi ndow 79 99fi 23 244 24 260 25 280 .26. .30.0 27 320 28 344 29 368 30 396 31 424 32 456 33 492 34 532 35 572 36 616 37 664 38 716 39 772 40 832 41 896 42 960 1152

Table 8 - scale factor bands for a window length of 2304 for

STOP_START_1152_WINDOW and STOP_1152_WINDOW at 22.05

85/114 and 24 kHz

f s [kHz] 22.05 and 24 num_swb_lon g window 47 swb swb offs et long window n n 1 4 2 8 3 12 4 16 5 20 6 24 7 28 8 32 9 ' -1 1 1 ---- 36 10 40 11 44 12 52 13 60 14 68 15 7 6 16 84 17 92 too- - 18 19 108 20 116 21 124 22 136 23 148

Table 9

swb swb offse t long wi ndow i sn 25 172 26 188 27 204 28 220 29 240 30 260 31 284 32 308 33 336 34 364 35 396 36 432 37 468 38 508 39 552 40 600 41 652 -42 '704 .... ..... 43 768 44 832 45 896 46 960 1152

scale factor bands of for window length of

2304 for

STOP START 1152 WINDOW and STOP 1152 WINDOW at kHz

fs [kHz] 64 in an Ion swb 47 (46) g_ window

86/114

swb swb_offs et long window n n 1 4 2 8 3 12 4 16 5 20 6 24 7 28 8 32 9 36 10 40 11 44 12 48 13 52 14 56 15 64 16 72 17 80 18 .88 19 100 20 112 21 124 22 140 23 156

swb swb offse t_long window 9 Δ 199 25 192 26 216 27 240 28 268 29 304 30 344 31 384 32 424 33 464 34 504 35 544 36 584 37 624 38 664 39 704 40 744 41 784 42 824 43 864 44 904 45 944 46 984 1152

Table 10 - scale factor bands for a window length of 2304 for STOP_START_1152_WINDOW and STOP_1152_WINDOW_ a_88., 2_e_9 6_kHz___________ „

fs [kHz] 8 8.2 and 96 in a lon swb 41 g window swb swb offs et long window n n 1 4 2 8 3 12 4 16 5 20

swb swb offse t long wi ndow 91 1 9fl 22 132 23 144 24 156 25 172 26 188

87/114

6 24 7 28 8 32 9 36 10 40 11 44 12 48 13 52 14 56 15 64 16 72 17 80 18 88 19 96 20 108

1.7 References

27 212 28 240 29 276 30 320 31 384 32 448 33 512 34 576 35 640 36 704 37 768 38 832 39 896 40 960 1152

of scale factor band tables

For all other scale factor band tables, refer to ISO / IEC 14496-3 subpart 4, section 4.5.4,

Table 4,129 to Table 4,147.

1.8 Quantization For the quantization of the spectral coefficients of

AAC in the encoder, a non-uniform quantizer is used.

Therefore, the decoder must perform inverse non-uniform quantization after Huffman decoding the scale factors (see subitem 6 ”. 3) and the silent decoding of spectral data (see subitem 6.1).

For the quantization of the spectral coefficients of

TCX, a uniform quantizer is used. Non-inverse quantization is required in the decoder, after the silent decoding of spectral data.

2. Filter bank and block switching

2.1 Tool description

The time / frequency representation of the signal is mapped into the time domain when fed to the filter bank module. This module consists of a discrete transformation of

88/114 modified inverse cosine (IMDCT), a window and an additional overlay function. In order to adapt the filter bank's time / frequency resolution to the characteristics of the input signal, a block switching tool is also adopted. N represents the length of the window, where N is a function of the window_sequence (see subitem 1.1). For each channel, the N / 2 time-frequency values Xi, k are transformed into time domain values N Xi, _n via the IMDCT. After applying the window function, for each channel, the first half of the sequence z _{i (n} is added to the second half of the previous block window sequence z ₍ ii), _n to reconstruct the output samples for each outer channel _{i / n} .

'““ 2.2 Definitions - _ _ _.

window_sequence 2-bit indicating that the window sequence (block size, for example) is used.

window_shape 1 bit that indicates that the window function is selected.

13C shows the eight window sequences (ONLY_LONG_SEQUENCE, LONG_START_SEQUENCE, EIGHT_SHORT_SEQUENCE, LONG_STOP_SEQUENCE, STOP_START_SEQUENCE, STOP_1152_SEQUENCE,

LPD_START_SEQUENCE, STOP_START_1152_SEQUENCE).

The following LPD_SEQUENCE refers to all window mode / coding combinations allowed within the so-called linear forecast domain codec (see section 1.3). In the context of decoding a frequency domain encoded frame it is important to know if only one next frame is encoded with LP domain encoding modes, which is represented by an LPD_SEQUENCE. However, the exact structure

89/114 within LPD_SEQUENCE is taken care of when decoding an encoded LP domain frame.

2.3 The decoding process

2.3.1 IMDCT

The analytical expression of IMDCT is:

Q <n <N ^N , for ⁼ - X ^ ^ec ft] R] cos ^ - ^ («+ where:

n = simple index i = window index k = spectral coefficient index

N = window length based on the value windowj ^ sequence ^ - - - - nO = (N / 2 + 10) / 2

The synthesis window length N for the inverse transformation is a function of the window__sequence syntax element and the algorithm_text. It_is_defined- as follows:

Window length 2304:

2304, if STOP_1152_SEQUENCE 2304, if STOP_START_1152_SEQUENCE

Window length 2048:

90/114

2048, if ONLYLONGSEQUENCE 2048, if LONG_START_SEQUENCE 256, if EIGHTSHORTSEQUENCE 2048, if LONGSTOPSEQUENCE 2048, if STOP_START_SEQUENCE 2048, if LPD_START_SEQUENCE

The following transitions:

From ONLY_LONG_SEQUENCE to from LONG_START_SEQUENCE to from LONG_STOP_SEQUENCE to ...

from EIGHT_SHORT_SEQUENCE to from LPD_SEQUENCE to from STOP_START_SEQUENCE to from LPD_START_SEQUENCE to from STOP_1152_SEQUENCE to from STOP_START_1152_SEQUENCE significant blocks are

ONLY_LONG_SEQUENCE

LONGSTARTSEQUENCE

LPD_START_SEQUENCE

I EIGHT_SHORT_SEQUENCE [LONG STOP SEOUENCE

ONLY_LONG_SEQUENCE <LONG_START_SEQUENC1 LPD_START_SEQUENCE 'EIGHT_SHORT_SEQUENCE

LONG_STOP_SEQUENCE STOP START SEQUENCE___ '“lpõ / sêquênce

STOPl 152SEQUENCE

STOP_START_1152SEQUENCE

EIGHTSHORTSEQUENCE

LONGSTOPSEQUENCE {LPDSEQUENCE

ONLYLONGSEQUENCE LONGSTARTSEQUENCE

EIGHTSHORTSEQUENCE P ^ara 1 LONGSTOPSEQUENCE

91/114

2.3.2 Window and block switching

Depending on the element window sequence and window_shape, different transformation windows are used. A combination of the window halves described as follows offers all possible window_sequences.

For window_shape == 1, the window coefficients are given by the Kaiser - Bessel derived window (KBD) as follows:

KBD LEFT, N

() =

W (n) =

KBD RIGHT, '

Where:

p = 0

)]

ΣΜρ, α)] p = 0 to 0 <n <

N for - <η <N

W, Kaiser-Bessel kernel window function, see also [5], is defined below

I

W (n, a) = _ιη (n-NI4 \ πα, 1.0— V l Λ7 / 4) _paxa 0 <«<γ / ₀ [ττα]

I.W = Σ k = 0

k \ _a - kernel window alpha factor

Otherwise, to use a sine window from the following í 4 is N = 2048 (1920) ^to ~ [óforN = 256 (240) window_shape == 0 form:

92/114, χ π 1 Ν

W "" _{T τ Λ Λη)} = sin (- ( '+ -)) T ^o Ο <η <-

SIN LEFT, N ^{y 7} 2 2 / X. π 1 Ν

IF ΓΛ'Ζ * Γ ^ ^ rl ^W / ^{= S} Í ⁿ (- («+ ~)) Ρ ^ ΓΗ - <Π <Ν

SIN_RIGHT, N ^{K 7} Ν 2 2

0 length from window N can be 2048 (1920) or 256 (240) for the KBD and the window sinusoidal. In case in STOP_ _1152_SEQUENCE and STOP_START_1152_SEQUENCE, N still can to be

2048 or 256, the window slopes are similar, but the upper plane regions are longer.

Only in the case of LPD_START_SEQUENCE the right part of the window is a 64-sample sine window.

How to obtain the possible window sequences is explained in parts a) to h) of this sub-item.

For all types of window_sequences, the window_shape of the left half of the first transformation window is determined by the shape of the previous block window. The following — formula — ex-hurry — that — f a t o ':

^ LEFT, n (F) ~

WkbD_ LEFT, if

W _S I _N ΕΕΕΤ, Ν ^ ⁷⁷ ) ' ^If

V. - window shape_previousblock == 1 window shape_previousblock = - 0 where:

window_shape_previous_block ·. window_shape of the previous block (i-1).

For the first raw_data_block () to be decoded, the window_shape of the left and right half of the window are identical.

a) ONLY_LONG_SEQUENCE:

93/114

The window_sequence == ONLY_LONG_SEQUENCE is equal to a LONG_WINDOW with a total window length of 2048 (1920).

For window_shape == 1 the window to

ONLY_LONG_SEQUENCE is given as follows:

\ _ í ^ íEFr, N_t ( ⁿ ) ' ^for θ - η <N_ ι / 2 ^U p- N_ 1/2 <η <N_ 1

If window_shape == 0 the window stops

ONLY_LONG_SEQUENCE can be described as follows:

w (n] =

W,

LEFT, N_

SIN _RIGHT, N _ l ζλ?), For 0 <Π <N_ / / 2 _z (w), For N __ l / 2 <Π <N_ l

After the window, the _de domain values. time (Zi, _n ) can be expressed as:. „

Α, η = w (rí) -x _in ;

b) LONG_START_SEQUENCE:

LONG_START_SEQUENCE is required to obtain a correct overlay and add a block transition from, from ONLY_LONG_SEQUENCE to EIGHT_SHORT_SEQUENCE.

The window length N_1 and N_s is set to 2048 (1920) and 256 (240), respectively.

If window_shape == 1, the

LONG START SEQUENCE is given as follows:

window for w (n) =

LEFT, N_

1.0, for <η <N 1/2

KBD _RIGHT, N _s

0.0,

- »τ ί z /« x 3./V 1 ~ Ή_s for l / 2 <n <j for ³ ^ ^Z T ^ < _n <

for

3N l + N s <n <N l

S and window_shape

0, window to

94/114

LONG_START_SEQUENCE will look like this:

W, w (n) =

LEFT, N

1.0,

M, for <η <N 1/2

SIN _RIGHT, N _s

0.0, (n +

N s 3N l-N s for j for for

7V _ // 2 <n <-

3N _l-N _s < _{n <} 3N _l + N _s

3N l + N s <n <N l

The time domain window values can be calculated with the formula explained in a).

c) EXGHT SHORT

The window_sequence == EIGHT_SHORT comprises eight overlapping and added SHORT_WINDOWS with an N_s length of 256 (240) each. The total length of the window_sequence with leading and trailing zeros is 2048 (1920). Each of the eight short blocks is first separated from the windows. The short block number is indexed to the variable j = 0,. .., ^{ M = N_l / N_s' ⁾

The window_shape of the previous block influences only the first of the eight short blocks (Wo_ (n) _) _._ Se_window — shape == 1, the window functions can be given as follows:

ζ X _ [W _LEF T, N_s ( ⁿ )> for 0 <Π <N_S / 2 θ y ^ siN_R / GHT, N_s ( ⁿ ) f for N_s / 2 Π <N_S

W _{s / n} _lef T, N_s ( ⁿ ), y ^ SIN_RIGHT, for 0 <n <N_s / 2 for N_s / 2 <n <N_s

Otherwise, if window_shape of window can be described as follows:

0, the functions

95/114 [W _LEFTN _ _s (n), for 0 <n <N_s! 2 u ™ ~ \ W _SIN _ _RIGHTN _ _s (laughs), for N_s / 2 <n <N_s

W _S in_LEF T, N_s ( ⁿ )>

z x_ I - o /; v_r _C r ,,, _v _ ₅ v -,> to 0 <n <N_ s / 2

1 - (/ 14-1) \ W _S iN_RiGHT, N_s ( ⁿ )> for N_s / 2 <n <N_S

The overlap and addition between the window_sequence EIGHT_SHORT resulting in the window time domain values zi, _n are described as follows:

N l-N s

0, n- ^ o ('for

N l-N_s \ for

N l-N s N I + N s λ ~ <n <- ~. -

ÍV / + (2 / -3> íV,

N _l + (2j — 3) N _s 4

N_l + (2j - \) N ~ S ') + - / / y _ / + (2 / -i) / y_, · ₄ /,

J> ⁿ 4 ^J

M - \, nNJ + (2M-3) N s

W _M _Snpara

N 1+ (2M-3 ') N s),

i) 7v; _í 0,

N 1+ (2Μ-Γ) Ν s - N l + (2M + V) N s for - ₄ - <Π <~ ₄ - for < _{n <N} _J this wmaow sequence is necessary to switch to and one. EXGHI — S-BOT ^ SEQUENCE-de-vo ± ta — pãfãHüm — ONLY_LONG_SEQUENCE.

If window_shape == 1, the window

LONG_STOP SEQUENCE is given as follows:

for w (n) =

0.0, w

LEFT, N _s

1.0, w, (nN l-N s),

KBD_R1GHT, N _l («)»

If

LONG_START_SEQUENCE is determined by:

N l-N s for 0 <n <

NJ-N_s N_l + N_s for 4 - n < ₄ for ^ / ^ <n <JV_Z / 2 for window_shape

N_l / 2 <n <N_l == 0, the window for

96/114 w (n) =

0.0, w

^rr LEFT, N_s

1.0, z N l — N s \ (n ~ ~ 4 ~), \ N stops for N_l-N_s

N l + N s for for ^^ <n <N_l / 2 N l / 2 <n <N l

SIN RIGHT.

The time domain values with window can be calculated with the formula explained in a).

e) STOP_START_SEQÜENCE:

STOP_START_SEQUENCE is required to achieve correct overlap and addition for block transition

EIGHT_SHORT_SEQUENCE for an EIGHT_SHORT_SEQUENCE when only one ONLY_LONG_SEQUENCE is needed.

The window length 2 \ 7_1 and N_s is set to 2048 (1920) and 256 __ (240), respectively .__

If window_shãpe == 1, the window to

STOP START SEQÜENCE is given as follows:

0.0,

N l - N s for 0 <n <- = - = W (laughs) =

W,

WlEFT, N s ( ⁿ ) 7

-forJV

KBD RIGHT, N _s (n +

1.0, for

N S 3N l - N Λ ', for

3N l - N s

0.0,

3N l + N s for

STOP_START_SEQUENCE will look like this:

N l + N s - = - = - <n <

N l + N

37V 4 l-N s 4 - ^3N . l + N s

97/114

W,

0.0,, N lN s ç _n - = _— = _), _c N lN s for 0 <n <- = - = W (n) =

LEFT, N_s - 4

1.0, for

N l-N s

SIN RIGHT.N s (n +

N _s 3N_l - N s 2 4

0.0, for

3N l-N s),

N l + N s <n <

3JV 1 1 o> 4 . ^3ΛΓ . l + N s 4

_nara 3N l + N s for - = - < _n <N /

The time domain values with window can be calculated with the formula explained in a)

f) LPD_START_SEQUENCE:

LPD_START_SEQÜENCE is required to obtain correct overlap and addition for a block transition. ONLY_LONG_SEQUENCE to an LPD_SEQUENCE.

The window length 2 \ 7_1 and N_s is set to 2048 (1920) and 256 (240), respectively.

If window_shape == 1, the window to

LPD START SEQUENCE is dada_da_segu-i-nte — fvrntar

W,

LEFT, N

1.0,

W (laughs) = for

N l for Q < _{n <} -—

N l 3N lN s - = - <n <= KBD RIGHT, ^N - ^s /.

, N s 3N lN S '3N lN s 3N l, (n + - = ---- = -), ^for = - = - <n <- = ^ ~

0.0,

3N l to -1 < _{n <} N l

If windowjshape

0, the window for

LPD_START_SEQUENCE will look like this:

98/114

W _T

LEFT, N, («)>

W (n) =

SIN RIGHT

N s.

(n +

1.0,; V_s 3N_l-N_s 4 4

0.0, to), to

N lp ³ ™ 0 <n <- ^ = -

N l 3N l-N s - = - <n <= -

4

3N lN s 3N l -—--- - <n <- = - for ^ L = L < _{n <N} /

The time domain values with window can be calculated with the formula explained in a).

g) STOP_1152_SEQUENCE:

STOP_1152_SEQUENCE is required to obtain correct overlap and addition for a block transition

LPD_SEQUENCE ONLY_LONG_SEQUENCE.

_ .C length ¹ gives '' window and 2V_s is set to 2048 (1920) and 256 (240), respectively.

If window_shape == 1, the window to

STOP_1152_SEQUENCE is given as follows:

0.0 /

W _L eft, n ~ A ⁿ ) »

W (n) = w _t

1.0,

N l 2N 1 + 3N s' N Ϊ for 0 <n <—— _Dara N / N 1 + 2N s for —_ _ < _M <- - = -

4

N 1 + 2N s 2N 1 + 3N s for - = - -— <n <- = KBD RIGHT,

0.0,), for 2 JV _ / + 3 TV _ s < _{n <N l +} 3N_s

3N s for _{N 1+} ³¹ - - <, _n <N l + tf _s 4 “

If window_shape

0, the window for

STOP_1152_SEQUENCE will look like this:

99/114

0.0, θ <„ _< X_ = L

W,

LEFT, N_s

C ^N - \ ( ⁿ -; -)>

For N l N 1 + 2N s - = - <n <- = - = -

W (n) =

1.0, _Dara N 1 + 2N s 2N 1 + 3N s P ^d - ₌ - < _{n <} --- = -

SIN RIGHT.N, (n +, _r 7 nxr, O at 2JV I + 3N S, 3N S

N_l 2N_l + 3N_s _Á ^for - = - _: -— <, n <N 1 + - =

0.0,

4 for N _l + ^3N ~ ^S - <η <N _l + N _s

The time domain values with window can be calculated with the formula explained in a).

h) STOP_START_1152_SEQUENCE:

STOP_START_1152_SEQUENCE is necessary to obtain correct overlap and _addition- for a block transition from an LPD_SEQUENCE to an EIGHT_SHORT_SEQUENCE when only one ONLY_LONG_SEQUENCE is needed.

The window length 7 \ 7_1 and N_s is set to 2048 (1920) and 256 (240), respectively .___ __—------ ^- - See window_shape == 1, the window for

STOP START SEQUENCE is given as follows:

0.0,

W (laughs) =

W,

LEFT, N_s ( ⁿ ) '

1.0,

KBD R1GHT.N s (n +

N _s 3N_l N s 2 4 ⁺

0.0,

If „N l for 0 <« <—— for N l N l + 2N S - <n <- = - = - for NJ + 2N_s ^ 3N_l _| N_s

3N l N s for -— + - <n <

), 4 2 for 3N l

2

3N ¹ RT - = - \ - N s + N s <n <N l + N s window shape

0, window to

STOP_START_SEQUENCE will look like this:

100/114

W,

0.0,

LEr-Τ, Ν_s ( ⁿ 4) 'for n ^{N 1} for 0 <n <- = -

N l N 1 + 2N s - = - <n <- = - = -

W (n) =

1.0,

S1N RIGHT.N s («+

N 1 + 2N s 3N l N s - = - = - <n <- H = -

4 2

3N l N s 3N l _xr 4 2 4

3N l for = - + N S <η <N l + N S for

N S 3N l N for

0.0,),

The time domain values with window can be calculated with the formula explained in a).

2.3.3 Overlay and addition with previous window sequence

In addition to the overlap and addition within the window_sequence EIGHT_SHORT _the first, part - (left) of each window_sequence is superimposed and added with a second (right) part of the previous window sequence, resulting in the final time domain values out ,,

Matemática mathematical expression for this operation can be described

In case of

LONG_START_SEQUENCE, EIGHT_SHORT_ _for NCE, STOP START SEQUENCE, LPD START SEQUENCE:

ONLY_LONG_SEQUENCE,

LONG STOP SEQUENCE, = ^z ,; „+ ^z . , n t - \, n + - 2; forOS «<-, N = 2048 (1920) * 2.

in case of

STOP 1152 SEQUENCE,

STOP START 1152 SEQUENCE ^OU ^ i, n “ ^Z i, n ^{+ Z} , N l 3N s'; - !,« + —— + -— to) <«<N l = 2048, N_s = 256

101/114

In the case of LPD_START_SEQUENCE, the next sequence is an LPD_SEQUENCE. A SIN or KBD window is applied to the left of LPD_SEQUENCE to have good overlap and addition.

W

SIN

LEFT, N

for _Λ ^N o <«<-

With N = 128

In case of STOP_1152_SEQUENCE, STOP_START_1152_SEQUENCE, the previous sequence is an LPD_SEQUENCE. A TDAC window is applied to the right of the LPD_SEQUENCE to have a good overlap and addition.

Previous LPD frame

Lk + 1— Rk

TDA iTDA

Figure caption: Current frame = current frame, Previous__LPD_.frame = qu-adro · ^- from EPD'anterior, Windowing = windowing, Folc / ing = fold (folding), Unjfo2ding = des folding,

Adding = addition.

3. IMDCT

See subsection 2.3.1

3.1 Window and block switching

Depending on the use of oversampled transformation window prototypes of a different element window_shape, the length of oversampled windows is

N _os = 2 · n_ long · os _ factor _ win

For window_shape == 1, the window coefficients

102/114 are given by the Kaiser-Bessel-derived window (KBD) as follows:

Nns-n-i

KBD f N n - ^ k 2)

Zk (p. “)]

P = ° / Vos / 2 for

N, the <n <N, the

Σ [^ (λ «)] p = 0 where:

W \ kemel Kaisser-Bessel window function, see also [5], is defined below:

! οΜ = Σ fc = 0 λ X

2a k \ ⁿ * os!

N _O J ^

I _O |> 6Z] for Q < _n <

N, os a - kemel window alpha factor, a = 4

Otherwise, for wmdow_shape - = 0, a sine window is used as follows:

SIN

N, os = sin

I saw for

N, the <n <N, the

For all types of window_sequences, the protoype used for the left window part is determined by the window shape of the previous block. The following formula expresses this fact:

103/114 left_window_shape \ p \ =

Wkbd [n], if window_shape_previous_block - 1 W sin [n], if window_shape_previous_block = - 0

Likewise, the prototype for the right window shape is determined by the following formula:

rzg / zZ_wz «6 / ow_5 / zape [n] =

W kbd [n], if window _shape = 1 - W sin [n], if window _shape = 0

Since the transition lengths are already determined, they should only be differentiated between EIGHT_SHORT_SEQUENCE and all others:

a) EIGHT SHORT SEQUENCE:

The following c-code as a portion describes the window and internal overlap-addition of an EIGHT_SHORT_SEQUENCE:

tw_windowing_short (X [] [], z [], first_pos, last_pos, warpe_trans_len_left, warped_trans_len_right, left_window_s hape [] shaqeJJJJ right window ______--- 'offset = n_long - 4 * n_short - n_short / 2; tr_scale_l ⁼

0.5 * n_long / warped_trans_len_left * os_factor_win;

tr_pos_l = warped_trans_len_left + (first_posn_long / 2) 0.5) * tr_scale_l;

tr_scale_r = 8 * os_factor_win;

tr_pos_r = tr_scale_r / 2;

for (i = 0; i <n_short; i ++) {z [i] = X [0] [i];

for (i = 0; i <first pos; i ++)

104/114 z [i] = Ο. ;

for (i = n_long-l-first_pos; i> = first_pos; 1-) {z [i] * = left_window_shape [floor (tr_pos_l)]; tr_pos_l + = tr_scale_l;

} for (i = 0; i <n_short; i ++) {z [offset + i + n short] =

X [0] [i + n_short] * right_window_shape [floor (tr_pos_r)];

tr_pos_r + = tr_scale_r;

} offset + = n_short;

for (k = 1; k <7; k + +) {tr_scale_l = n_short * os_factor_win; tr_pos_l = tr_scale_l / 2;

tr_pos_r = os_factor_win * n_long-tr_pos_l; for (i = 0; i <n_short; i ++) {________ ______z [-i—— ~ ’+ offset] + =

X [k] [i] * right_window_shape [floor (tr_pos_r)];

z [offset + n_short + i] =

X [k] [n_short +

i] * right_window_shape [floor (tr_pos_l)];

tr_pos_l + = tr_scale_l; tr_pos_r - = tr_scale_l;

} offset + = n_short;

} tr_scale_l = n_short * os_factor_win;

105/114 tr_pos_l = tr_scale_l / 2;

for (i = n_short - 1; i> = 0; i--) {z [i + offset] +

X [7] [i] * right_window_shape [(int) floor (tr_pos_l)];

tr_pos_l + = tr_scale_l;

} for (i = 0; i <n_short; i ++) {z [offset + n_short + i] = X [7] [n_short + i];

} tr_scale_r

0.5 * n_long / warpedTransLenRight * os_factor_win;

tr_pos_r = 0.5 * tr_scale_r +. 5; '' tr_pos_r = (1.5 * n_long- (float) wEnd0.5 + warpedTransLenRight) * tr_scale_r;

for (i = 3 * n_long-l-last_pos; i <= wEnd; i ++) {z [i] * = right window shape [floor (tr pos r)]; _ _________- 1r3dõsHr + = - t'r_s cal e_r;

} for (i = lsat_pos + l; i <2 * n_long; i ++) z [i] = 0. ;

b) all others:

tw_windowing_long (X [] [], z [], first_pos, last_pos, wa rpe_trans_len_left, warped_trans_len_right, left_window_shape [], righ t_window_shape []) {for (i = 0; i <first_pos; i ++) z [i] = 0. ;

for (i = last pos + 1; i <N; i ++)

106/114 ζ [i] = 0 .;

tr_scale =

0.5 * n_long / warped_trans_len_left * os_factor_win;

tr_pos = (warped_trans_len_left + first_posN / 4) +0.5) * tr_scale;

for (i = N / 2-l-first_pos; i> = first_pos; i—) {z [i]

X [0] [i] * left_window_shape (floor (tr_pos)]);

tr_pos + = tr_scale;

} tr_scale =

0.5 * n_long / warped_trans_len_right * os_factor_win;

_ tr__pos - - - = “- - - (3 * N / 4-lastJpos0.5 + warped_trans_len_right) * tr_scale;

for (i = 3 * N / 2-l-last_pos; i <= last_pos; i ++) {z [i]

X [0] [i] * right_window_shape [floor (tr_pos)]);

tr_pos + = tr_scale;

}}

4. MDCT-based TCX

4.1 Description of the tool

When core_mode is equal to ale when one or more of the three TCX modes are selected as linear domain-prediction encoding, that is, one of the 4 matrix entries of mod [] is greater than 0, the TCX tool based on in

MDCT. The MDCT-based TCX receives the quantized spectral coefficients from the arithmetic decoder. The coefficients

Quantities 107/114 are first completed by a comfort noise before applying an inverse MDCT transformation to obtain a weighted time domain synthesis, which is then fed into the weighted synthesis LPC filter.

4.2 Definitions lg Number of output of spectral coefficients quantized by the arithmetic decoder noise_factor noise level quantization index noise led Noise level injected into the reconstructed spectrum noise [] Generated noise vector global_gain quantization index_of.

resized g Resized gain rms root mean quadratic root of the synthesized time domain signal, x [],

-------- xy-j Synthesized time domain signal

4.3 The decoding process

The MDCT-based TCX requires from the arithmetic decoder a number of quantized spectral coefficients, lg, which is determined by mod [] and last_lpd_mode values. These two values also define the window length and the shape that will be applied in the reverse MDCT. The window is composed of three parts, a left side overlay of the L samples, a part in the middle of the M samples and a right overlap part of the R samples.

To obtain a 2 * lg length MDCT window,

108/114 ZL zeros on the left and ZR zeros on the right, as indicated in fi.

14G for Table 3 / Fig. 14F.

Table 3 - Number of Spectral Coefficients as a function of last_lpd_mode and mod []

Value of last lpd mo in Value of mod [x] Lg number of coefficients spectral ZL L M R ZR 0 1 320 160 0 256 128 96 0 2 576 288 0 512 128 224 0 3 1152 512 128 1024 128 512 1. .3 1 256 64 128 128 128 64 1. .3 2 512 192 128 384 128 192 1. . 3 3 1024. , 44 8. 128 - 896 - - 12 8 - ‘448

The MDCT window is given ί ^ (ή)

I ⁿ 0 <n <ZL

W _SÍN _ left, l ( ⁿ ~ ___ for _______ ZL_ <m ^ ~ ZL ^. L ---- 1 for ZL + L <n <ZL + L + M right r (P ~ ZL - L - Μ} P ZL + L + M <n <ZL + L + M + R 0 ZL + L + Af + R <n <2 lg

The spectral quantized coefficients, quant [], emitted by the arithmetic decoder are completed by a comfort noise. The level of injected noise is determined by the noise__factor decoded as follows:

noise_level = 0.0625 * (8-noise_factor)

A noise vector, noise [], is then computed using a random function, random_sign (), emitting -1 or +1 at random.

109/114 noise [i] = random_sign () * noise_level;

The vectors quant [] and noise [] are combined to form the vector of reconstructed spectral coefficients, r [], so that the execution of 8 consecutive zeroes in quant [] are replaced by the noise [] components. An execution of 8 nonzero is detected according to the formula:

r / [z] = 1 for ie [0, lg / 6 [r / [lg / 6 + z] = yj | ^ zzízzzí [lg / 6 + 8. [[z / 8j + Ãr] J for ze [0, 7. lg / 6 [fc = 0

The reconstructed spectrum is obtained as follows:

r [z] quant \ í \ se r / [z] = 1 noise [i] otherwise

Prior to the application of the inverse MDCT, a deformation spectrum is applied according to the following steps:

1. Calculate the eneraia .. £ \ .. of the 8-dimensional block in the index m for each 8-dimensional block of the first quarter of the spectrum

2. Calculate the ratio of R _m -sqrt (E _m / Ei), where I is the block index with the maximum value of all E _m

3. If R _m <0, l, then set R _m = 0, l

4. If R _m <R _m -l, then set R _m = R _m -l

Each 8-dimensional block belonging to the first quarter of the spectrum is then multiplied by the factor of R _m .

The reconstructed spectrum is fed into an inverse MDCT. The non-windowed output signal, x [], is again re110 / 114 scaled by the gain, g, obtained by an inverse quantization of the decoded global_gain index:

g _ j Qglobal-gainf28l (2.rms)

Where rms is calculated as:

3 * lg / 2-l <= lg / 2 rms =

L + M + R

The resized dome signal is then equal to:

= * [/] · £

After resizing, windowing and adding overlay are applied. - · ''

The reconstructed TCX target x (n) is then filtered through the weighted synthesis inverse filter state zero ^•: - to find the excitation signal which is

A (z) (l - az ~ ^l ) / (A (z / λ) applied to the synthesis filter. Obs.er.ve_que — the synthesized time EP-fi-iter interpolated by subframe is used in Once the excitation is determined, the signal is reconstructed by filtering the excitation through the synthesis filter ae, 1 / A (z) then the de-emphasis by filtering through the filter,

1 / (1 — 0.68z) as described above.

Note that the excitation is also required to update the adaptive ACELP codebook and allow switching from TCX to ACELP in a subsequent frame. Also note that the length of the TCX synthesis is given by the length of the TCX frame (without overlapping): samples of 256, 512 or 1024 for the

111/114 mod [] of 1.2 or 3, respectively.

Normative References [1] ISO / IEC 11172-3: 1993, Information technology

- Coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbit / s, Part 3: Audio.

[2] ITÜ-T Rec.H.222.0 (1995) | ISO / IEC 138181: 2000, Information technology - Generic coding of moving pictures and associated audio Information: - Part 1: Systems.

[3] ISO / IEC 13818-3: 1998, Information technology

Generic coding of moving pictures and associated audio

Information: - Part 3: Audio.

[4] ISO / IEC 13818-7: 2004, Information technology

Generic coding of moving pictures and associated - audio

Information: - Part 7: Advanced Audio Coding (AAC).

[5] ISO / IEC 14496-3: 2005, Information technology

- Coding of audio-visual objects - Part 1: Systems [6] ISO / IEC 14496-3: 2005, Information technology ç. .Coding — of-audi-o — vi-sual-Ob j'éct “s ^- - ^__ Part 3: Audio [7] ISO / IEC 23003-1: 2007, Information technology - MPEG audio technologies - Part 1: MPEG Surround [8] 3GPP TS 26.290 V6.3.0, Extended Adaptive

Multi-Rate - Wideband (AMR-WB +) codec; Transcoding functions [9] 3GPP TS 26.190, Adaptive Multi-Rate Wideband (AMR-WB) speech codec; Transcoding functions [10] 3GPP TS 26,090, Adaptive Multi-Rate (AMR) speech codec; Transcoding functions

Definitions

Definitions can be found in ISO / IEC

112/114

14496-3, subpart 1, subitem 1.3 (Terms and definitions) and in 3GPP

TS 26.290, Section 3 (Definitions and abbreviations).

Although some aspects have been described in the context of an apparatus, it is evident that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method phase or a characteristic of a method phase. Similarly, the aspects described in the context of a method phase also represent a description of a corresponding block, or item or characteristic of a corresponding device.

The inventive encoded audio signal can be stored on a digital storage medium or can be transmitted on a transmission medium, such as a wireless transmission medium or a wired transmission medium, such as

Internet.

Depending on certain implementation requirements, configurations of the invention can be implemented in hardware or software-A — implementation can be performed using a digital storage medium, for example, a floppy disk, a DVD, CD, ROM, PROM, EPRÕM, EEPROM or FLASH memory, having electronically readable control signals stored there, which cooperate (or are capable of cooperating) with a programmable computer system in such a way that the respective method is carried out.

Some configurations according to the invention comprise a data carrier having electronically readable control signals, which are able to cooperate with a programmable computer system, so that one of the methods described here is performed.

113/114

Generally, the configurations of the present invention can be implemented as a computer program product with a program code, the program code being operative to perform one of the methods, when the computer program product is run on a computer. The program code can, for example, be stored on an optical reader.

Other configurations include a computer program for carrying out one of the methods described here, stored in an optical reading conveyor.

In other words, a configuration of the inventive method is, therefore, a computer program having a program code to execute one of the described methods. Here, when the computer program is executed on a computer.

Another configuration of the inventive methods is, therefore, a data carrier '(or a digital storage medium, or a computer-readable medium), comprising, grayado_nele, —- the-program ^- give ^- ' computer for the realization of a of the methods described here.

Another configuration of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for carrying out one of the methods described here. The data flow or signal sequence can, for example, be configured to be transferred via a data communication connection, for example, via the Internet.

Another configuration comprises processing means, for example, a computer or a programmable logic device, configured or adapted to execute one of the

114/114 methods described here.

Another configuration comprises a computer that has the computer program installed on it to perform one of the methods described here.

In some configurations, a programmable logic device (for example, a programmable gate array field) can be used to perform some or all of the functionality of the methods described here. In some configurations, a programmable field gate array can cooperate with a microprocessor to perform one of the methods described here. Generally, the methods are preferably performed by any hardware device.

—The configurations described above · are merely illustrative for the principles of the present invention. It is understood that the modifications and variations of the arrangements and the details described in this document will be apparent to others qualified in the art. It is the intention, therefore, to be limited by the scope of the impending patent claims and not by the specific details presented through the description and explanation of the configurations presented here.

Claims (17)

1. Audio encoder to encode an audio signal characterized by comprising: a first code conversion branch to encode an audio signal using a first code conversion algorithm to capture a first encoded signal, the first code conversion branch comprising the first converter to convert an input signal in a spectral domain; a second code conversion branch to encode an audio signal using a second code conversion algorithm to capture a second encoded signal, where the first code conversion algorithm is different from the second code conversion algorithm, the second branch of code conversion comprising a domain converter for converting an input signal from an input domain to an output domain, and a second converter for converting an input signal to a spectral domain; a switch to switch between the first code conversion branch and the second code conversion branch so that, for a portion of the audio input signal, both the first encoded signal and the second encoded signal are on an output signal of encoder; a signal analyzer for analyzing the audio signal portion to determine whether the audio signal portion is represented as the first encoded signal or as the second encoded signal in the encoder output signal, Petition 870190024003, of 03/13/2019, p. 14/176 2/17 wherein the signal analyzer is additionally configured to variablely determine a respective time / frequency resolution of the first converter and the second converter, when the first encoded signal or the second encoded signal representing the audio signal portion is generated; and an output interface for generating an encoder output signal comprising the first encoded signal and the second encoded signal and information indicating the first encoded signal and the second encoded signal, and information indicating the time / frequency resolution applied to encode the first encoded signal and to encode the second encoded signal, in which the signal analyzer is configured to determine the time / frequency resolution to be selected from a plurality of different window lengths, the different window lengths being are at least two out of 2304, 2048, 256, 1920, 2160, 240 samples, or use a plurality of different transformed lengths, where the different transformed lengths comprise at least two of the group comprising 1152, 1024, 1080, 960, 128, 120 coefficients per transformed block, or where the signal analyzer is configured to determine the resolution time / frequency of the second converter as one of a plurality of different window lengths, with the plurality of different window lengths being at least two out of 640, 1152, Petition 870190024003, of 03/13/2019, p. 15/176 3/17 2304, 512, 1024 or 2048 samples, or use a plurality of different transformed lengths, wherein the different transformed lengths comprise at least two of the group comprising 320, 576, 1152, 256, 512, 1024 spectral coefficients per transformed block. Audio encoder according to claim 1, characterized in that the signal analyzer is configured to classify the portion of the audio signal as a speech audio signal or a music audio signal and to perform a transient detection in the case of a music signal to determine the time / frequency resolution of the first converter or to perform synthesis analysis processing to determine the time / frequency resolution of the second converter. Audio encoder according to claim 1, characterized in that the first converter and the second converter comprise a variable window transformation processor comprising a window function with a variable window size and a transformation function with a length of variable transformation, and in which the signal analyzer is configured to control, based on the signal analysis, the window size and / or the transformation length. claim 1, characterized audio, according to the fur second branch in branch of processing for domain determined fur that a second branch in Petition 870190024003, of 03/13/2019, p. 16/176 4/17 processing comprises the second converter, in which the signal analyzer is configured to subdivide the portion of the audio signal into a sequence of sub-portions, and in which the signal analyzer is configured to determine the time / frequency resolution of the second converter that depends on the position of the sub-portion processed by the first processing branch in relation to a sub-portion of the portion processed by the second processing branch. Audio encoder according to claim 4, characterized in that the first processing branch comprises an ACELP encoder, in which the second processing branch comprises an MDCT-TCX processing device, in which the signal analyzer is configured to configure the time resolution of the second converter to a first value determined by a length of a sub-portion or a second value determined by a length of the sub-portion multiplied by an integer value greater than one, where the second value is less than the first value . 6. Audio encoder according to claim 1, characterized in that the signal analyzer is configured to determine a signal rating in a constant scan that covers a plurality of blocks of equal size of audio samples, and to subdivide a block into a variable number of blocks that depend on the audio signal, where a length of the sub-block determines the first time / frequency resolution or the second time / frequency resolution. Petition 870190024003, of 03/13/2019, p. 17/176 5/17 7. Audio encoder according to claim 1, characterized by the second branch of conversion into code comprising: a first processing branch for processing an audio signal; a second processing branch, the second processing branch comprising the second converter; and an additional switch to switch between the first processing branch and the second processing branch so that, for a portion of the audio signal input in the second code conversion branch, both a first processed signal and a second processed signal are in the second coded signal. 8. Audio coding method an audio signal characterized by comprising: encode, in a first branch of conversion into code, an audio signal using the first algorithm of conversion into code to capture a first encoded signal, the first branch of conversion in code comprising the first converter to convert a signal entry into a spectral domain; encode, in a second branch of code conversion, an audio signal using a second code conversion algorithm to capture a second encoded signal, in which the first code conversion algorithm is different from the second conversion algorithm code, and the second branch of conversion into code comprises a data converter Petition 870190024003, of 03/13/2019, p. 18/176 6/17 domain to convert an input signal from an input domain to an output domain, and a second converter to convert an input signal to a spectral domain; switching between the first code conversion branch and the second code conversion branch so that, for a portion of the audio input signal, both the first encoded signal and the second encoded signal are on an encoder output signal; analyze the audio signal portion to determine whether the audio signal portion is represented as the first encoded signal or as the second encoded signal in the encoder output signal, variablely determine the respective time / frequency resolution of the first converter and the second converter, when the first encoded signal or the second encoded signal representing the portion of the audio signal is generated; and generate one exit sign in encoder that comprises the first coded signal and the second sign encoded and information indicating O first sign encoded And the second encoded signal, and information that indicate the resolution of time / frequency applied to encode the first encoded signal and to encode the second encoded signal, in which the analysis determines the time / frequency resolution to be selected from a plurality of different window lengths, the different Petition 870190024003, of 03/13/2019, p. 19/176 7/17 window lengths are at least two out of 2304, 2048, 256, 1920, 2160, 240 samples, or use the plurality of different transformed lengths, where the different transformed lengths comprise at least two of the group comprising 1152, 1024, 1080, 960, 128, 120 coefficients per transformed block, or where the analysis determines the time / frequency resolution of the second converter as one of a plurality of different window lengths, the plurality of different window lengths being at least two out of 640, 1152, 2304, 512, 1024 or 2048 samples, or uses a plurality of different transformed lengths, where the different transformed lengths comprise at least two of the group comprising 320, 576, 1152, 256, 512, 1024 spectral coefficients per transformed block. 9. Audio decoder to decode an encoded signal, the encoded signal comprising a first encoded signal, a second encoded signal, an indication indicating the first encoded signal and the second encoded signal and time / frequency resolution information to be used to decode the first encoded signal and the second encoded audio signal, characterized by comprising: a first decoding branch to decode the first encoded signal using a first controllable frequency / time converter, the first controllable frequency / time converter being configured to be controlled using the information from Petition 870190024003, of 03/13/2019, p. 20/176 8/17 time / frequency resolution for the first encoded signal to capture a first decoded signal; a second decoding branch to decode the second encoded signal using a second controllable frequency / time converter, the second controllable frequency / time converter being configured to be controlled using the time / frequency resolution information for the second encoded signal; a controller for controlling the first frequency / time converter and the second frequency / time converter using the time / frequency resolution information; a domain converter for generating a synthesis signal using the second decoded signal; and a combiner to combine the first decoded signal and the synthesis signal to capture a decoded audio signal, in which the controller is configured to control the first frequency / time converter and the second frequency / time converter so that, for the first frequency / time converter, the time / frequency resolution is selected from a plurality of different window lengths, with the different window lengths being at least two 2304, 2048, 256, 1920, 2160, 240 samples, ie selected from a plurality of different transformed lengths, with the different transformed lengths comprising at least two of Petition 870190024003, of 03/13/2019, p. 21/176 9/17 the group comprising 1152, 1024, 1080, 960, 128, 120 coefficients per transformed block, or for the second frequency / time converter, the time / frequency resolution is selected as one of a plurality of different lengths of window, the plurality of different window lengths being at least two out of 640, 1152, 2304, 512, 1024 or 2048 samples, or selected from a plurality of different transformed lengths, with the different transformed lengths comprising at least two among the group comprising 320, 576, 1152, 256, 512, 1024 spectral coefficients per transformed block. Audio decoder according to claim 9, characterized in that the second decoding branch comprises a first reverse processing branch for reverse processing of a first processed signal which is further comprised in the encoded signal to capture a first reverse processed signal; wherein the second controllable frequency / time converter is located in a second reverse processing branch configured for reverse processing of the second encoded signal in a domain identical to the domain of the first reverse processed signal to capture a second reverse processed signal; an additional combiner to combine the first reverse processed signal and the second reverse processed signal to capture a combined signal; and Petition 870190024003, of 03/13/2019, p. 22/176 10/17 in that the sign combined is inserted at the combiner. 11. Decoder audio, a deal with The claim 9, featured for the first converter in frequency / time and the second frequency / time converter are time domain override converters that comprise an overlap / addition unit to cancel a time domain spectrum overlap comprised in the first encoded signal and on the second encoded signal. Audio decoder according to claim 9, characterized in that the encoded signal comprises code conversion mode information that identifies whether an encoded signal is the first encoded signal and the second encoded signal, and wherein the decoder further comprises a input interface for interpreting the code conversion mode information to determine whether the encoded signal is to be provided in both the first decoding branch and the second decoding branch. Audio decoder according to claim 9, characterized in that the first encoded signal is arithmetically encoded, and wherein the first branch of code conversion comprises an arithmetic decoder. Audio decoder according to claim 9, characterized in that the first code conversion branch comprises a quantizer that comprises a non-uniform quantizer characteristic for canceling a Petition 870190024003, of 03/13/2019, p. 23/176 11/17 result of a non-uniform quantization applied when generating the first coded signal, in which the second branch of conversion into code comprises a dequantizer using the different dequantization characteristic. 15. Audio decoder according to claim 9, characterized in that the controller is configured to control the first frequency / time converter and the second frequency / time converter, applying a discrete time / frequency resolution to each converter of several different possible discrete time / frequency resolutions, with the various possible different discrete time / frequency resolutions being superior for the second converter compared to the various possible different time / frequency resolutions for the first converter. 16. Audio decoder, according to claim 9, characterized in that the domain converter is an LPC synthesis processor that generates the synthesis signal using PC filter information, the LPC filter information being comprised in the signal encoded. 17. Method of audio decoding an encoded signal, the encoded signal comprising a first encoded signal, a second encoded signal, an indication indicating the first encoded signal and the second encoded signal and time / frequency resolution information a be used to decode the first encoded signal and the second encoded audio signal, Petition 870190024003, of 03/13/2019, p. 24/176 12/17 characterized by understanding: decode, by a first decoding branch, the first signal encoded using a first controllable frequency / time converter, the first controllable frequency / time converter being configured to be controlled using the time resolution information / frequency for the first encoded signal to capture a first decoded signal; decode, by a second decoding branch, the second encoded signal using a second controllable frequency / time converter, and the second controllable frequency / time converter is configured to be controlled using the time resolution information / frequency for the second coded signal; control the first frequency / time converter and the second frequency / time converter using the time / frequency resolution information; generate, by a domain converter, a synthesis signal using the second decoded signal; and combining the first decoded signal and the synthesis signal to capture a decoded audio signal, where the control of the first frequency / time converter and the second frequency / time converter is so that, for the first frequency / time converter , the time / frequency resolution is selected from a plurality of different window lengths, with the different window lengths being at least two Petition 870190024003, of 03/13/2019, p. 25/176 13/17 2304, 2048, 256, 1920, 2160, 240 samples, ie selected from a plurality of different transformed lengths, with the different transformed lengths comprising at least two of the group comprising 1152, 1024,1080, 960, 128, 120 coefficients per transformed block, or for the second frequency / time converter, the time / frequency resolution is selected as one of a plurality of different window lengths, with the plurality of different window lengths being at least two out of 640, 1152, 2304, 512, 1024 or 2048 samples, or is selected from a plurality of different transformed lengths, with the different transformed lengths comprising at least two of the group comprising 320, 576, 1152, 256, 512, 1024 spectral coefficients per transformed block. 18. Non-transitory storage media characterized by having stored in it a computer program to execute, when executed in a processor, an audio coding method of an audio signal, the method comprising: encode, in a first branch of conversion into code, an audio signal using the first algorithm of conversion into code to capture a first encoded signal, the first branch of conversion in code comprising the first converter to convert a signal entry into a spectral domain; Petition 870190024003, of 03/13/2019, p. 26/176 14/17 encode, in a second branch of code conversion, an audio signal using a second code conversion algorithm to capture a second encoded signal, in which the first code conversion algorithm is different from the second algorithm code conversion, the second branch of code conversion comprises a domain converter to convert an input signal from an input domain to an output domain, and a second converter to convert an input signal to an spectral domain; switching between the first code conversion branch and the second code conversion branch so that, for a portion of the audio input signal, both the first encoded signal and the second encoded signal are on an encoder output signal; analyze the audio signal portion to determine whether the audio signal portion is represented as the first encoded signal or as the second encoded signal in the encoder output signal, variablely determine the respective time / frequency resolution of the first converter and the second converter, when the first encoded signal or the second encoded signal representing the portion of the audio signal is generated; and generating an encoder output signal comprising the first encoded signal and the second encoded signal and information indicating the first encoded signal and the second encoded signal and information that Petition 870190024003, of 03/13/2019, p. 27/176 15/17 indicate the time / frequency resolution applied to encode the first encoded signal and to encode the second encoded signal, in which the analysis determines the time / frequency resolution to be selected from a plurality of different window lengths, being that the different window lengths are at least two out of 2304, 2048, 256, 1920, 2160, 240 samples, or use of the plurality of different transformed lengths, where the different transformed lengths comprise at least two of the group comprising 1152, 1024, 1080, 960, 128, 120 coefficients per transformed block, or where the analysis determines the time / frequency resolution of the second converter as one of a plurality of different window lengths, the plurality of different window lengths being at least two out of 640, 1152, 2304, 512, 1024 or 2048 samples, or uses a plurality of different transformed lengths, where the different transformed lengths comprise at least two of the group comprising 320, 576, 1152, 256, 512, 1024 spectral coefficients per transformed block. 19. Non-transitory storage media characterized by having stored in it a computer program to execute, when executed in a processor, an audio decoding method of an encoded signal, the encoded signal comprising a first encoded signal, a second signal coded, an indication indicating the first coded signal and the second signal Petition 870190024003, of 03/13/2019, p. 28/176 16/17 encoded and time / frequency resolution information to be used to decode the first encoded signal and the second encoded audio signal, where the method comprises: decode, by a first decoding branch, the first signal encoded using a first controllable frequency / time converter, the first controllable frequency / time converter being configured to be controlled using the time resolution information / frequency for the first encoded signal to capture a first decoded signal; decode, by a second decoding branch, the second encoded signal using a second controllable frequency / time converter, and the second controllable frequency / time converter is configured to be controlled using the time resolution information / frequency for the second coded signal; control the first frequency / time converter and the second frequency / time converter using the time / frequency resolution information; generate, by a domain converter, a synthesis signal using the second decoded signal; and combining the first decoded signal and the synthesis signal to capture a decoded audio signal, where the control of the first frequency / time converter and the second frequency / time converter is so that, for the first frequency / time converter , the time / frequency resolution is selected from among Petition 870190024003, of 03/13/2019, p. 29/176 17/17 plurality of different window lengths, with different window lengths being at least two among 2304, 2048, 256, 1920, 2160, 240 samples, ie selected from a plurality of different transformed lengths, with the different transformed lengths comprising at least two of the group comprising 1152, 1024,1080, 960, 128, 120 coefficients per transformed block, or for the second frequency / time converter, the time / frequency resolution is selected as one of a plurality of different window lengths, with the plurality of different window lengths being at least two out of 640, 1152, 2304, 512, 1024 or 2048 samples, or is selected from a plurality of different transformed lengths, with the different transformed lengths comprising at least two of the group comprising 320, 576, 1152, 256, 512, 1024 spectral coefficients per transformed block.