CA3150683C - Downscaled decoding - Google Patents
Downscaled decoding Download PDFInfo
- Publication number
- CA3150683C CA3150683C CA3150683A CA3150683A CA3150683C CA 3150683 C CA3150683 C CA 3150683C CA 3150683 A CA3150683 A CA 3150683A CA 3150683 A CA3150683 A CA 3150683A CA 3150683 C CA3150683 C CA 3150683C
- Authority
- CA
- Canada
- Prior art keywords
- length
- window
- frame
- coefficients
- temporal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003786 synthesis reaction Methods 0.000 claims abstract description 81
- 230000015572 biosynthetic process Effects 0.000 claims abstract description 80
- 238000005070 sampling Methods 0.000 claims abstract description 54
- 238000000034 method Methods 0.000 claims abstract description 30
- 230000002123 temporal effect Effects 0.000 claims description 78
- 230000003595 spectral effect Effects 0.000 claims description 61
- 230000005236 sound signal Effects 0.000 claims description 46
- 230000006870 function Effects 0.000 claims description 35
- 230000008569 process Effects 0.000 claims description 11
- 238000012423 maintenance Methods 0.000 abstract description 2
- 238000004458 analytical method Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 7
- 230000006978 adaptation Effects 0.000 description 4
- 238000007493 shaping process Methods 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000000354 decomposition reaction Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 238000001228 spectrum Methods 0.000 description 3
- 239000000872 buffer Substances 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000005284 excitation Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 101100047216 Mus musculus Trnp1 gene Proteins 0.000 description 1
- 102100026436 Regulator of MON1-CCZ1 complex Human genes 0.000 description 1
- 101710180672 Regulator of MON1-CCZ1 complex Proteins 0.000 description 1
- 238000012952 Resampling Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 229910003460 diamond Inorganic materials 0.000 description 1
- 239000010432 diamond Substances 0.000 description 1
- 230000002349 favourable effect Effects 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000008450 motivation Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000010076 replication Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0212—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using orthogonal transformation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/022—Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
Abstract
A downscaled version of an audio decoding procedure may more effectively and/or at improved compliance maintenance be achieved if the synthesis window used for downscaled audio decoding is a downsampled version of a reference synthesis window involved in the non-downscaled audio decoding procedure by downsampling by the downsampling factor by which the downsampled sampling rate and the original sampling rate deviate, and downsampled using a segmental interpolation in segments of 1/4 of the frame length.
Description
Dow nscaled Decoding Dt,s.criptien The present application is concerned with a downscaled decoding concept.
The MPEG-4 Enhanced Low Delay AAC (AAC-ELD) usually operates at sample rates up to 48 kHz, which results in an algorithmic delay of 15ms. For some applications, e.g. lip-sync transmission of audio, an even lower delay is desirable. AAC-ELD already provides such an option by operating at higher sample rates, e.g. 96 kHz, and therefore provides operation modes with even lower delay, e.g. 7.5 ms. However, this operation mode comes along with an unnecessary high complexity due to the high sample rate.
The solution to this problem is to apply a downscaled version of the filter bank and therefore, to render the audio signal at a lower sample rate, e.g. 48kHz instead of 96 kHz.
The downscaling operation is already part of AAC-ELD as it Is inherited from the MPEG-4 AAC-LD codec, which serves as a basis for AAC-ELD.
The question which remains, however, is how to find the downscaled version of a specific filter bank. That is, the only uncertainty is the way the window coefficients are derived whilst enabling clear conformance testing of the downscaled operation modes of the AAC-ELD decoder.
In the following the principles of the down-scaled operation mode of tho AAC-(E)LD
codecs are described.
The downscaled operation mode or AAC-LD is described for AAC-LD in ISO/IEC
3:2009 in section 4.6.17.2.7 "Adaptation to systems using lower sampling rates" as follows:
"In certain applications it may be necessary to integrate the low delay decoder into an audio system running at lower sampling rates (e.g. 16 kHz) while the nominal sampling rate of the bitstream payload is much higher (e.g. 48 kHz, corresponding to an algorithmic codec delay of approx. 20 ms). In such cases, it is favorable to decode the output of the low delay codec directly at the target sampling rate rather than using an additional sampling rate conversion operation after decoding.
Date Recue/Date Received 2022-03-01
The MPEG-4 Enhanced Low Delay AAC (AAC-ELD) usually operates at sample rates up to 48 kHz, which results in an algorithmic delay of 15ms. For some applications, e.g. lip-sync transmission of audio, an even lower delay is desirable. AAC-ELD already provides such an option by operating at higher sample rates, e.g. 96 kHz, and therefore provides operation modes with even lower delay, e.g. 7.5 ms. However, this operation mode comes along with an unnecessary high complexity due to the high sample rate.
The solution to this problem is to apply a downscaled version of the filter bank and therefore, to render the audio signal at a lower sample rate, e.g. 48kHz instead of 96 kHz.
The downscaling operation is already part of AAC-ELD as it Is inherited from the MPEG-4 AAC-LD codec, which serves as a basis for AAC-ELD.
The question which remains, however, is how to find the downscaled version of a specific filter bank. That is, the only uncertainty is the way the window coefficients are derived whilst enabling clear conformance testing of the downscaled operation modes of the AAC-ELD decoder.
In the following the principles of the down-scaled operation mode of tho AAC-(E)LD
codecs are described.
The downscaled operation mode or AAC-LD is described for AAC-LD in ISO/IEC
3:2009 in section 4.6.17.2.7 "Adaptation to systems using lower sampling rates" as follows:
"In certain applications it may be necessary to integrate the low delay decoder into an audio system running at lower sampling rates (e.g. 16 kHz) while the nominal sampling rate of the bitstream payload is much higher (e.g. 48 kHz, corresponding to an algorithmic codec delay of approx. 20 ms). In such cases, it is favorable to decode the output of the low delay codec directly at the target sampling rate rather than using an additional sampling rate conversion operation after decoding.
Date Recue/Date Received 2022-03-01
2 This can be approximated by appropriate downscaling of both, the frame size and the sampling rate, by some integer factor (e.g. 2, 3), resulting in the same time/frequency resolution of the codec. For example, the codec output can be generated at 16 kHz sampling rate instead of the nominal 48 kHz by retaining only the lowest third (i.e. 480/3 =
160) of the spectral coefficients prior to the synthesis filterbank and reducing the inverse transform size to one third (i.e. window size 960/3 = 320).
As a consequence, decoding for lower sampling rates reduces both memory and computational requirements, but may not produce exactly the same output as a full-bandwidth decoding, followed by band limiting and sample rate conversion.
Please note that decoding at a lower sampling rate, as described above, does not affect the interpretation of levels, which refers to the nominal sampling rate of the AAC low delay bitstream payload."
Please note that AAC-LD works with a standard MDCT framework and two window shapes, i.e. sine-window and low-overlap-window. Both windows are fully described by formulas and therefore, window coefficients for any transformation lengths can be determined.
Compared to AAC-LD, the AAC-ELD codec shows two major differences:
= The Low Delay MDCT window (LD-MDCT) = The possibility of utilizing the Low Delay SBR tool The IMDCT algorithm using the low delay MDCT window is described in 4.6.20.2 in (11, which is very similar to the standard IMDCT version using e.g. the sine window. The coefficients of the low delay MDCT windows (480 and 512 samples frame size) are given in Table 4.A.15 and 4.A.16 in [11. Please note that the coefficients cannot be determined by a formula, as the coefficients are the result of an optimization algorithm.
Fig. 9 shows a plot of the window shape for frame size 512 In case the low delay SBR (LD-SBR) tool is used in conjunction with the AAC-ELD coder.
the filter banks of the LD-SBR module are downscaled as well. This ensures that the SBR
Date Recue/Date Received 2022-03-01
160) of the spectral coefficients prior to the synthesis filterbank and reducing the inverse transform size to one third (i.e. window size 960/3 = 320).
As a consequence, decoding for lower sampling rates reduces both memory and computational requirements, but may not produce exactly the same output as a full-bandwidth decoding, followed by band limiting and sample rate conversion.
Please note that decoding at a lower sampling rate, as described above, does not affect the interpretation of levels, which refers to the nominal sampling rate of the AAC low delay bitstream payload."
Please note that AAC-LD works with a standard MDCT framework and two window shapes, i.e. sine-window and low-overlap-window. Both windows are fully described by formulas and therefore, window coefficients for any transformation lengths can be determined.
Compared to AAC-LD, the AAC-ELD codec shows two major differences:
= The Low Delay MDCT window (LD-MDCT) = The possibility of utilizing the Low Delay SBR tool The IMDCT algorithm using the low delay MDCT window is described in 4.6.20.2 in (11, which is very similar to the standard IMDCT version using e.g. the sine window. The coefficients of the low delay MDCT windows (480 and 512 samples frame size) are given in Table 4.A.15 and 4.A.16 in [11. Please note that the coefficients cannot be determined by a formula, as the coefficients are the result of an optimization algorithm.
Fig. 9 shows a plot of the window shape for frame size 512 In case the low delay SBR (LD-SBR) tool is used in conjunction with the AAC-ELD coder.
the filter banks of the LD-SBR module are downscaled as well. This ensures that the SBR
Date Recue/Date Received 2022-03-01
3 module operates with the same frequency resolution and therefore, no more adaptions are required.
Thus, the above description reveals that there is a need for downscaling decoding operations such as, for example, downscaling a decoding at an AAC-ELD. It would be feasible to find out the coefficients for the downscaled synthesis window function anew, but this is a cumbersome task, necessitates additional storage for storing the downscaled version and renders a conformity check between the non-downscaled decoding and the downscaled decoding more complicated or, from another perspective, does not comply with the manner of downscaling requested in the AAC-ELD, for example.
Depending on the downscale ratio, i.e. the ratio between the original sampling rate and the downscaled sampling rate, one could derive the downscaled synthesis window function simply by downsampling, i.e. picking out every second, third, ... window coefficient of the original synthesis window function, but this procedure does not result in a sufficient conformity of the non-downscaled decoding and downscaled decoding, respectively. Using more sophisticated decimating procedures applied to the synthesis window function, lead to unacceptable deviations from the original synthesis window function shape.
Therefore, there Is a need in the art for an improved downscaled decoding concept.
Accordingly, it is an object of the present invention to provide an audio decoding scheme which allows for such an improved downscaled decoding.
The present invention is based on the finding that a downscaled version of an audio decoding procedure may more effectively and/or at improved compliance maintenance be achieved if the synthesis window used for downscaled audio decoding is a downsampled version of a reference synthesis window involved in the non-downscaled audio decoding procedure by downsampling by the downsampling factor by which the downsampled sampling rate and the original sampling rate deviate, and downsampled using a segmental interpolation in segments of 1/4 of the frame length.
Preferred embodiments of the present application are described below with respect to the figures, among which:
Date Recue/Date Received 2022-03-01
Thus, the above description reveals that there is a need for downscaling decoding operations such as, for example, downscaling a decoding at an AAC-ELD. It would be feasible to find out the coefficients for the downscaled synthesis window function anew, but this is a cumbersome task, necessitates additional storage for storing the downscaled version and renders a conformity check between the non-downscaled decoding and the downscaled decoding more complicated or, from another perspective, does not comply with the manner of downscaling requested in the AAC-ELD, for example.
Depending on the downscale ratio, i.e. the ratio between the original sampling rate and the downscaled sampling rate, one could derive the downscaled synthesis window function simply by downsampling, i.e. picking out every second, third, ... window coefficient of the original synthesis window function, but this procedure does not result in a sufficient conformity of the non-downscaled decoding and downscaled decoding, respectively. Using more sophisticated decimating procedures applied to the synthesis window function, lead to unacceptable deviations from the original synthesis window function shape.
Therefore, there Is a need in the art for an improved downscaled decoding concept.
Accordingly, it is an object of the present invention to provide an audio decoding scheme which allows for such an improved downscaled decoding.
The present invention is based on the finding that a downscaled version of an audio decoding procedure may more effectively and/or at improved compliance maintenance be achieved if the synthesis window used for downscaled audio decoding is a downsampled version of a reference synthesis window involved in the non-downscaled audio decoding procedure by downsampling by the downsampling factor by which the downsampled sampling rate and the original sampling rate deviate, and downsampled using a segmental interpolation in segments of 1/4 of the frame length.
Preferred embodiments of the present application are described below with respect to the figures, among which:
Date Recue/Date Received 2022-03-01
4 Fig. 1 shows a schematic diagram illustrating perfect reconstruction requirements needed to be obeyed when downscaling decoding in order to preserve perfect reconstruction:
Fig. 2 shows a block diagram of an audio decoder for downscaied decoding according to an embodiment;
Fig. 3 shows a schematic diagram illustrating in the upper half the manner in which an audio signal has been coded at an original sampling rate into a data stream and, in the lower half separated from the upper half by a dashed horizontal line, a downscaled decoding operation for reconstructing the audio signal from the data stream at a reduced or downscaled sampling rate, so as to illustrate the mode of operation of the audio decoder of Fig.
2;
Fig. 4 shows a schematic diagram illustrating the cooperation of the windower and time domain aliasing canceler of Fig. 2;
Fig. 5 Illustrates a possible implementation for achieving the reconstruction according to Fig. 4 using a special treatment of the zero-weighted portions of the spectral-to-time modulated time portions;
Fig. 6 shows a schematic diagram illustrating the downsampling to obtain the downsampled synthesis window;
Fig. 7 shows a block diagram illustrating a downscaled operation of AAC-ELD
including the low delay SBR tool;
Fig. 8 shows a block diagram of an audio decoder for downscaled decoding according to an embodiment where modulator, windower and canceller are implemented according to a lifting implementation; and Fig. 9 shows a graph of the window coefficients of a low delay window according to AAC-ELD for 512 sample frame size as an example of a reference synthesis window to be downsampled.
Date Recue/Date Received 2022-03-01 The following description starts with an illustration of an embodiment for downscaled decoding with respect to the AAC-ELD codec. That is, the following description starts with an embodiment which could form a downscaled mode for AAC-El..D. This description concurrently forms a kind of explanation of the motivation underlying the embodiments of
Fig. 2 shows a block diagram of an audio decoder for downscaied decoding according to an embodiment;
Fig. 3 shows a schematic diagram illustrating in the upper half the manner in which an audio signal has been coded at an original sampling rate into a data stream and, in the lower half separated from the upper half by a dashed horizontal line, a downscaled decoding operation for reconstructing the audio signal from the data stream at a reduced or downscaled sampling rate, so as to illustrate the mode of operation of the audio decoder of Fig.
2;
Fig. 4 shows a schematic diagram illustrating the cooperation of the windower and time domain aliasing canceler of Fig. 2;
Fig. 5 Illustrates a possible implementation for achieving the reconstruction according to Fig. 4 using a special treatment of the zero-weighted portions of the spectral-to-time modulated time portions;
Fig. 6 shows a schematic diagram illustrating the downsampling to obtain the downsampled synthesis window;
Fig. 7 shows a block diagram illustrating a downscaled operation of AAC-ELD
including the low delay SBR tool;
Fig. 8 shows a block diagram of an audio decoder for downscaled decoding according to an embodiment where modulator, windower and canceller are implemented according to a lifting implementation; and Fig. 9 shows a graph of the window coefficients of a low delay window according to AAC-ELD for 512 sample frame size as an example of a reference synthesis window to be downsampled.
Date Recue/Date Received 2022-03-01 The following description starts with an illustration of an embodiment for downscaled decoding with respect to the AAC-ELD codec. That is, the following description starts with an embodiment which could form a downscaled mode for AAC-El..D. This description concurrently forms a kind of explanation of the motivation underlying the embodiments of
5 the present application. Later on, this description is generalized, thereby leading to a description of an audio decoder and audio decoding method in accordance with an embodiment of the present application.
As described in the introductory portion of the specification of the present application, AAC-ELD uses low delay MDCT windows. In order to generate downscaled versions thereof, i.e. downscaled low delay windows, the subsequently explained proposal for forming a downscaled mode for AAC-ELD uses a segmental spline interpolation algorithm which maintains the perfect reconstruction property (PR) of the LD-MDCT window with a very high precision. Therefore, the algorithm allows the generation of window coefficients in the direct form, as described in ISO/IEC 14496-3:2009, as well as in the lifting form, as described In [2], in a compatible way. This means both implementations generate 16bit-conform output.
The interpolation of Low Delay MDCT window is performed as follows.
In general a spline interpolation is to be used for generating the downscaled window coefficients to maintain the frequency response and mostly the perfect reconstruction property (around 170dB SNR). The interpolation needs to be constraint in certain segments to maintain the perfect reconstruction property. For the window coefficients c covering the DCT kernel of the transformation (see also Figure 1, c(1024)..c(2048)), the following constraint is required, 1 = Ksgn c(i) = c (2N ¨ 1 ¨ i) + c(N + 0 = c (N ¨ 1 OA for i = 0...N/2 ¨ 1 (1) where N denotes the frame size. Some implementation may use different signs to optimize the complexity, here, denoted by sgn. The requirement in (1) can be illustrated by Fig. 1. It should be recalled that simply in even in case of F=2, i.e.
halfening the sample rate, leaving-out every second window coefficient of the reference synthesis window to obtain the downscaled synthesis window does not fulfil the requirement.
Date Recue/Date Received 2022-03-01
As described in the introductory portion of the specification of the present application, AAC-ELD uses low delay MDCT windows. In order to generate downscaled versions thereof, i.e. downscaled low delay windows, the subsequently explained proposal for forming a downscaled mode for AAC-ELD uses a segmental spline interpolation algorithm which maintains the perfect reconstruction property (PR) of the LD-MDCT window with a very high precision. Therefore, the algorithm allows the generation of window coefficients in the direct form, as described in ISO/IEC 14496-3:2009, as well as in the lifting form, as described In [2], in a compatible way. This means both implementations generate 16bit-conform output.
The interpolation of Low Delay MDCT window is performed as follows.
In general a spline interpolation is to be used for generating the downscaled window coefficients to maintain the frequency response and mostly the perfect reconstruction property (around 170dB SNR). The interpolation needs to be constraint in certain segments to maintain the perfect reconstruction property. For the window coefficients c covering the DCT kernel of the transformation (see also Figure 1, c(1024)..c(2048)), the following constraint is required, 1 = Ksgn c(i) = c (2N ¨ 1 ¨ i) + c(N + 0 = c (N ¨ 1 OA for i = 0...N/2 ¨ 1 (1) where N denotes the frame size. Some implementation may use different signs to optimize the complexity, here, denoted by sgn. The requirement in (1) can be illustrated by Fig. 1. It should be recalled that simply in even in case of F=2, i.e.
halfening the sample rate, leaving-out every second window coefficient of the reference synthesis window to obtain the downscaled synthesis window does not fulfil the requirement.
Date Recue/Date Received 2022-03-01
6 The coefficients c(0) c(2N ¨ 1) are listed along the diamond shape. The N/4 zeros in the window coefficients, which are responsible for the delay reduction of the filter bank, are marked using a bold arrow. Fig. 1 shows the dependencies of the coefficients caused by the folding involved in the MDCT and also the points where the interpolation needs to be constraint in order to avoid any undesired dependencies.
= Every N/2 coefficient, the interpolation needs to stop to maintain (1) = Additionally, the interpolation algorithm needs to stop every N/4 coefficients due to the inserted zeros. This ensures that the zeros are maintained and the interpolation error is not spread which maintains the PR.
The second constraint is not only required for the segment containing the zeros but also for the other segments. Knowing that some coefficients in the DCT kernel were not determined by the optimization algorithm but were determined by formula (1) to enable PR, several discontinuities in the window shape can be explained, e.g. around c(1536+128) in Figure 1.
In order to minimize the PR error, the interpolation needs to stop at such points, which appear in a N/4 grid.
Due to that reason, the segment size of N/4 is chosen for the segmental spline interpolation to generate the downscaled window coefficients. The source window coefficients are always given by the coefficients used for N = 512, also for downscaling operations resulting in frame sizes of N = 240 or /V = 120. The basic algorithm is outlined very briefly in the following as MATLABO code:
FAC Downscaling factor % e.g. 0.5 sb = 128; segment size of source window w_down 1]; % downscaled window nSegments = Lenqth(W)/(sb);& number of segments; W=LD window coefficients for N=512 (FAC*sb-1)) +O. !3)/PAC-O.5, % spline init for i=1:nSegments, w_down= [w down, spline ( [0: (sb-1) ,W( (i-1) *sb+ (1: (sb))),xn)J;
end;
As the spline function may not be fully deterministic, the complete algorithm is exactly specified in the following section, which may be included into ISO/IEC 14496-3:2009, in order to form an improved downscaled mode in AAC-ELD.
Date Recue/Date Received 2022-03-01
= Every N/2 coefficient, the interpolation needs to stop to maintain (1) = Additionally, the interpolation algorithm needs to stop every N/4 coefficients due to the inserted zeros. This ensures that the zeros are maintained and the interpolation error is not spread which maintains the PR.
The second constraint is not only required for the segment containing the zeros but also for the other segments. Knowing that some coefficients in the DCT kernel were not determined by the optimization algorithm but were determined by formula (1) to enable PR, several discontinuities in the window shape can be explained, e.g. around c(1536+128) in Figure 1.
In order to minimize the PR error, the interpolation needs to stop at such points, which appear in a N/4 grid.
Due to that reason, the segment size of N/4 is chosen for the segmental spline interpolation to generate the downscaled window coefficients. The source window coefficients are always given by the coefficients used for N = 512, also for downscaling operations resulting in frame sizes of N = 240 or /V = 120. The basic algorithm is outlined very briefly in the following as MATLABO code:
FAC Downscaling factor % e.g. 0.5 sb = 128; segment size of source window w_down 1]; % downscaled window nSegments = Lenqth(W)/(sb);& number of segments; W=LD window coefficients for N=512 (FAC*sb-1)) +O. !3)/PAC-O.5, % spline init for i=1:nSegments, w_down= [w down, spline ( [0: (sb-1) ,W( (i-1) *sb+ (1: (sb))),xn)J;
end;
As the spline function may not be fully deterministic, the complete algorithm is exactly specified in the following section, which may be included into ISO/IEC 14496-3:2009, in order to form an improved downscaled mode in AAC-ELD.
Date Recue/Date Received 2022-03-01
7 In other words, the following section provides a proposal as to how the above-outlined idea could be applied to ER AAC ELD, i.e. as to how a low-complex decoder could decode a ER AAC ELD bitstream coded at a first data rate at a second data rate lower than the first data rate. It is emphasized however, that the definition of N
as used in the following adheres to the standard. Here, N corresponds to the length of the DCT kernel whereas hereinabove, in the claims, and the subsequently described generalized embodiments, N corresponds to the frame length, namely the mutual overlap length of the DCT kernels, i.e. the half of the DCT kernel length. Accordingly, while N was indicated to be 512 hereinabove, for example, it is indicated to be 1024 in the following.
The following paragraphs are proposed for inclusion to 14496-3:2009 via Amendment.
A.0 Adaptation to systems using lower sampling rates For certain applications, ER AAC LID can change the playout sample rate in order to avoid additional resampling steps (see 4.6.17.2.7). ER AAC ELD can apply similar clownscaling steps using the Low Delay MDCT window and the LD-SBR tool. In case AAC-ELD
operates with the LD-SBR tool, the downscaling factor is limited to multiples of 2. Without LD-SBR, the downscaled frame size needs to be an integer number.
A.I Downscaling of Low Delay MOGI' window The LD-MDCT window ww for N=1024 is downscaled by a factor F using a segmental spline interpolation. The number of leading zeros in the window coefficients, i.e. N/8, determines the segment size. The downscaled window coefficients we) .4 are used for the Inverse MDCT as described in 4Ø20.2 but with a downscaled window length No =
N / F.
Please note that the algorithm is also able to generate downscaled lifting coefficients of the LD-MDCT.
fs_window..si4e 2048; /*Number of fullscalc window coefficients. According to ISOIIEC 14496-3:2009, use 2048. For lifting impleruenations, please adjust this variable accordingly */
ds_window_size ¨ N * fs..window_size / (1024 * F); /* downscaled window coefficients; N determines the transformation length according to 4.6.20.2 41 fs...segmcnt_size 128;
num....segments ¨ fs._ window size / fs_segment_size;
ds_segment size tis._window. size / nuni_segrnents;
Mil:4128j, y[128); I* temporary buffers *I
/* loop over segments */
for (b 0; b <111.111)....segments;
I* copy current segment to unp */
copy(ScW. 1.-11)[b fs_scgment_size), tinp, Is segment_siz.e), I* apply cubic spline interpolation frx downscaling *I
Date Recue/Date Received 2022-03-01
as used in the following adheres to the standard. Here, N corresponds to the length of the DCT kernel whereas hereinabove, in the claims, and the subsequently described generalized embodiments, N corresponds to the frame length, namely the mutual overlap length of the DCT kernels, i.e. the half of the DCT kernel length. Accordingly, while N was indicated to be 512 hereinabove, for example, it is indicated to be 1024 in the following.
The following paragraphs are proposed for inclusion to 14496-3:2009 via Amendment.
A.0 Adaptation to systems using lower sampling rates For certain applications, ER AAC LID can change the playout sample rate in order to avoid additional resampling steps (see 4.6.17.2.7). ER AAC ELD can apply similar clownscaling steps using the Low Delay MDCT window and the LD-SBR tool. In case AAC-ELD
operates with the LD-SBR tool, the downscaling factor is limited to multiples of 2. Without LD-SBR, the downscaled frame size needs to be an integer number.
A.I Downscaling of Low Delay MOGI' window The LD-MDCT window ww for N=1024 is downscaled by a factor F using a segmental spline interpolation. The number of leading zeros in the window coefficients, i.e. N/8, determines the segment size. The downscaled window coefficients we) .4 are used for the Inverse MDCT as described in 4Ø20.2 but with a downscaled window length No =
N / F.
Please note that the algorithm is also able to generate downscaled lifting coefficients of the LD-MDCT.
fs_window..si4e 2048; /*Number of fullscalc window coefficients. According to ISOIIEC 14496-3:2009, use 2048. For lifting impleruenations, please adjust this variable accordingly */
ds_window_size ¨ N * fs..window_size / (1024 * F); /* downscaled window coefficients; N determines the transformation length according to 4.6.20.2 41 fs...segmcnt_size 128;
num....segments ¨ fs._ window size / fs_segment_size;
ds_segment size tis._window. size / nuni_segrnents;
Mil:4128j, y[128); I* temporary buffers *I
/* loop over segments */
for (b 0; b <111.111)....segments;
I* copy current segment to unp */
copy(ScW. 1.-11)[b fs_scgment_size), tinp, Is segment_siz.e), I* apply cubic spline interpolation frx downscaling *I
Date Recue/Date Received 2022-03-01
8 to calculate interpolating phase 41 phase - idOW_SiZe) i (2 * ds.. window ...size);
/4 calculate the coefficients c of the cubic spline given tmp */
Po array of precalculated constants *I
m [0.166666672, 0.25, 0.266666681, 0.267857134, 0.267942578, 0.267948717, 0.267949l64.;
ii = fs_segment_size; /4 for simplicity */
I* calculate vector r needed to calculate the coefficients c */' for (i n -3; i 0; i¨) r[i) ¨3 * ((ttrip[i 1. 21- tntpri .1- 1]) 1] -for (1= 1; i < 7; i++) r[ij -= m[i - 1]* r[i - 1];
for(i = 7; i < n - 4; i++) r[i.1-= 0.267949194 * rii - 1];
I* calculate coefficients c */
cin - 2] = r[n - 3]/ 6;
cfn - 31 = (r[n - 4] - c[n - 21) * 0.25;
for (i n - 4; > 7; i--) c[i]--(Ii - I) - c[i 1])* 0.267949194;
for (i = 7; i > 1; 1--) c[i]¨(rii-1)-c[iI-1))*m[i-11;
011 =11'01 * mr0];
c[0] ¨2 *41] -c[2];
c[n-l] = 2 * c[n - 2] - c[n - 3];
/* keep original samples in temp buffer y because samples of tmp will be replaced with interpolated samples 41 copy(trnp, y, fs_..segment_size);
/* generate downscaied points and do interriolation */
for (k =0; k< ds.. segment size; ki-+) {
step phase+ k * fs,õ.segment_size / ds_pegment_size;
dx floor(step);
duff'- step - idx;
di ¨ (c[idic 1] - c(idx]) / 3;
bi (y[idx + 1] - y[idx]) - (14itlx + 11+ 2 * c[itlx1) / 3;
t* calculate downscaled values and store in tmp */
tmp[k] y[idx] diff * (bi + cliff* (c[idx] + cliff" di));
/* assemble downsealed window 'V
copy(tmp, &W_LV_dfb * ds_segment_size), ds_segment_sizel;
A.2 Downscaling of Low Delay SBR tool In case the Low Delay SBR tool is used in conjunction with ELD, this tool can be downscaled to lower sample rates, at least for downscaling factors of a multiple of 2. The downscaie factor F controls the number of bands used for the CLDFB analysis and synthesis filter bank. The following two paragraphs describe a downscaled CLDFB
analysis and synthesis filter bank, see also 4.6.19.4.
Date Recue/Date Received 2022-03-01
/4 calculate the coefficients c of the cubic spline given tmp */
Po array of precalculated constants *I
m [0.166666672, 0.25, 0.266666681, 0.267857134, 0.267942578, 0.267948717, 0.267949l64.;
ii = fs_segment_size; /4 for simplicity */
I* calculate vector r needed to calculate the coefficients c */' for (i n -3; i 0; i¨) r[i) ¨3 * ((ttrip[i 1. 21- tntpri .1- 1]) 1] -for (1= 1; i < 7; i++) r[ij -= m[i - 1]* r[i - 1];
for(i = 7; i < n - 4; i++) r[i.1-= 0.267949194 * rii - 1];
I* calculate coefficients c */
cin - 2] = r[n - 3]/ 6;
cfn - 31 = (r[n - 4] - c[n - 21) * 0.25;
for (i n - 4; > 7; i--) c[i]--(Ii - I) - c[i 1])* 0.267949194;
for (i = 7; i > 1; 1--) c[i]¨(rii-1)-c[iI-1))*m[i-11;
011 =11'01 * mr0];
c[0] ¨2 *41] -c[2];
c[n-l] = 2 * c[n - 2] - c[n - 3];
/* keep original samples in temp buffer y because samples of tmp will be replaced with interpolated samples 41 copy(trnp, y, fs_..segment_size);
/* generate downscaied points and do interriolation */
for (k =0; k< ds.. segment size; ki-+) {
step phase+ k * fs,õ.segment_size / ds_pegment_size;
dx floor(step);
duff'- step - idx;
di ¨ (c[idic 1] - c(idx]) / 3;
bi (y[idx + 1] - y[idx]) - (14itlx + 11+ 2 * c[itlx1) / 3;
t* calculate downscaled values and store in tmp */
tmp[k] y[idx] diff * (bi + cliff* (c[idx] + cliff" di));
/* assemble downsealed window 'V
copy(tmp, &W_LV_dfb * ds_segment_size), ds_segment_sizel;
A.2 Downscaling of Low Delay SBR tool In case the Low Delay SBR tool is used in conjunction with ELD, this tool can be downscaled to lower sample rates, at least for downscaling factors of a multiple of 2. The downscaie factor F controls the number of bands used for the CLDFB analysis and synthesis filter bank. The following two paragraphs describe a downscaled CLDFB
analysis and synthesis filter bank, see also 4.6.19.4.
Date Recue/Date Received 2022-03-01
9 4.6.20.5.2.1 Downscaled analyses CLDFB filter bank = Define number of dowriscalect CLDFB bands B =32/F.
= Shift the samples in the array x by B positions. The oldest B samples are discarded and 6 new samples are stored in positions 0 to B 1.
= Multiply the samples of array x by the coefficient of window ci to get array z. The window coefficients ci are obtained by linear interpolation of the coefficients c, i.e.
through the equation ci(i) = :11c(2F = i + 1 + p) + c(21" = i + p)1, 0 S <(108). p = int (25 ¨
0.5).
The window coefficients of c can be found in Table 4.A.90.
.. = Sum the samples to create the 2B-element array u:
u(n) = z(n) z(n+ 26) + z(n + 48)+ z(n + 68) + z(n. 4- 88). 0 5 n < (213).
= Calculate B new subband samples by the matrix operation Mu, where OSk<B
M(k,n)= 2 = exp __________________________ 0 < n <
In the equation, exp() denotes the complex exponential function and j is the imaginary unit.
4.6.20.5.2.2 Downscaled synthesis CLDFB filter bank . Define number of downscaled CLDFB bands El = 64/F.
= Shift the samples in the array v by 28 positions. The oldest 2B samples are discarded.
= The B new complex-valued subband samples are multiplied by the matrix N, where N(k,n) = exp (pir.(k +0.5).(2.r1-(B- I))) f 0 < k <B
A4 ZB P t0 n.<213^
in the equation, exp() denotes the complex exponential function and) is the imaginary unit. The real part of the output from this operation is stored in the positions 0 to 21/
1 of array v.
= Extract samples from v to create the 1.0B-element array 9.
g(28 = rt. + k) = v(46 = n + k) r 0 < n < 4 g(2B=n+B+k).-t-v(41:1 = n 3B + k)' 105k<E1 Date Recue/Date Received 2022-03-01 = Multiply the samples of array g by the coefficient of window ci to produce array w. The window coefficients ci are obtained by linear interpolation of the coefficients c, i.e.
through the equation 5 ci(i) =Dc(2/7 = i t- 1 1 p) I c(2P = i p)], 0 i < (108),p = int ¨
0.5).
The window coefficients of c can be found in Table 4.A.90.
= Calculate B new output samples by summation of samples from array w according to output(n) = Ero w(Bi + n), 0 5 n < B.
Please note that setting F = 2 provides the downsampled synthesis filter bank according to 4.6.19.4.3. Therefore, to process a downsampled LD-SBR bit stream with an additional downscale factor F, P needs to be multiplied by 2.
4.6.20.5.2.3 Downscaled real-valued CLDFB filter bank The downscaling of the CLDFB can be applied for the real valued versions of the low power SBR mode as well. For illustration, please also consider 4.6.19.5.
For the downscaled real-valued analysis and synthesis filter bank, follow the description in 4.6.20.5.2.1 and 4.6.20.2.2 and exchange the exp() modulator in Ai by a cos() modulator.
A.3 Low Delay MDCT Analysis This subciause describes the Low Delay MDCT filter bank utilized in the AAC
ELD
encoder. The core MDCT algorithm is mostly unchanged, but with a longer window, such that n is now running from ¨N to N-1 (rather than from 0 to N-1) The spectral coefficient, Xi,k, are defined as follows:
N 27r X i.k = --2 = E z,..õ cos( __________ v2 + no)(k ¨I)) for 0 k < N/2 where:
zin = windowed input sequence N = sample index K = spectral coefficient index I = block index N = window length no (-N / 2 + 1) / 2 Date Recue/Date Received 2022-03-01 The window length N (based on the sine window) is 1024 or 960.
The window length of the low-delay window is 2*N. The windowing is extended to the past in the following way:
zi,(N ¨1¨ n) x'for for n=-N,...,N-1, with the synthesis window w used as the analysis window by inverting the order.
A.4 Low Delay MOM" Synthesis The synthesis filter bank is modified compared to the standard IMDCT algorithm using a sine window in order to adopt a low-delay filter bank. The core IMDCT
algorithm is mostly unchanged, but with a longer window, such that n is now running up to 2N-1 (rather than up to N-1).
2 -2-µ71 xf,õ =.= 4.it specti][k]cosi 7¨(n + no) k ,for 0 n< 2N
where:
n = sample index i = window index k = spectral coefficient index N = window length / twice the frame length no = (-N / 2 + 1) / 2 with N = 960 or 1024.
The windowing and overlap-add is Conducted in the following way:
The length N window is replaced by a length 2N window with more overlap in the past, and less overlap to the future (N/8 values are actually zero).
Windowing for the Low Delay Window:
zem = W1(n) =
Date Recue/Date Received 2022-03-01 Where the window now has a length of 2N, hence n=0,...,2N-1.
Overlap and add:
uuL1,, +z ,v = N
for 0<=n<N/2 Here, the paragraphs proposed for being included into 14496-3:2009 via amendment end.
Naturally, the above description of a possible downscaled mode for AAC-ELD
merely represents one embodiment of the present application and several modifications are feasible. Generally, embodiments of the present application are not restricted to an audio decoder performing a downscaled version of AAC-ELD decoding. In other words, embodiments of the present application may, for instance, be derived by forming an audio decoder capable of performing the inverse transformation process in a downscaled manner only without supporting or using the various AAC-ELD specific further tasks such as, for instance, the scale factor-based transmission of the spectral envelope, TNS
(temporal noise shaping) filtering, spectral band replication (SBR) or the like.
Subsequently, a more general embodiment for an audio decoder is described. The above-outlined example for an AAC-ELD audio decoder supporting the described downscaled mode could thus represent an implementation of the subsequently described audio decoder. In particular, the subsequently explained decoder is shown in Fig. 2 while Fig. 3 illustrates the steps performed by the decoder of Fig. 2.
The audio decoder of Fig. 2, which is generally indicated using reference sign
= Shift the samples in the array x by B positions. The oldest B samples are discarded and 6 new samples are stored in positions 0 to B 1.
= Multiply the samples of array x by the coefficient of window ci to get array z. The window coefficients ci are obtained by linear interpolation of the coefficients c, i.e.
through the equation ci(i) = :11c(2F = i + 1 + p) + c(21" = i + p)1, 0 S <(108). p = int (25 ¨
0.5).
The window coefficients of c can be found in Table 4.A.90.
.. = Sum the samples to create the 2B-element array u:
u(n) = z(n) z(n+ 26) + z(n + 48)+ z(n + 68) + z(n. 4- 88). 0 5 n < (213).
= Calculate B new subband samples by the matrix operation Mu, where OSk<B
M(k,n)= 2 = exp __________________________ 0 < n <
In the equation, exp() denotes the complex exponential function and j is the imaginary unit.
4.6.20.5.2.2 Downscaled synthesis CLDFB filter bank . Define number of downscaled CLDFB bands El = 64/F.
= Shift the samples in the array v by 28 positions. The oldest 2B samples are discarded.
= The B new complex-valued subband samples are multiplied by the matrix N, where N(k,n) = exp (pir.(k +0.5).(2.r1-(B- I))) f 0 < k <B
A4 ZB P t0 n.<213^
in the equation, exp() denotes the complex exponential function and) is the imaginary unit. The real part of the output from this operation is stored in the positions 0 to 21/
1 of array v.
= Extract samples from v to create the 1.0B-element array 9.
g(28 = rt. + k) = v(46 = n + k) r 0 < n < 4 g(2B=n+B+k).-t-v(41:1 = n 3B + k)' 105k<E1 Date Recue/Date Received 2022-03-01 = Multiply the samples of array g by the coefficient of window ci to produce array w. The window coefficients ci are obtained by linear interpolation of the coefficients c, i.e.
through the equation 5 ci(i) =Dc(2/7 = i t- 1 1 p) I c(2P = i p)], 0 i < (108),p = int ¨
0.5).
The window coefficients of c can be found in Table 4.A.90.
= Calculate B new output samples by summation of samples from array w according to output(n) = Ero w(Bi + n), 0 5 n < B.
Please note that setting F = 2 provides the downsampled synthesis filter bank according to 4.6.19.4.3. Therefore, to process a downsampled LD-SBR bit stream with an additional downscale factor F, P needs to be multiplied by 2.
4.6.20.5.2.3 Downscaled real-valued CLDFB filter bank The downscaling of the CLDFB can be applied for the real valued versions of the low power SBR mode as well. For illustration, please also consider 4.6.19.5.
For the downscaled real-valued analysis and synthesis filter bank, follow the description in 4.6.20.5.2.1 and 4.6.20.2.2 and exchange the exp() modulator in Ai by a cos() modulator.
A.3 Low Delay MDCT Analysis This subciause describes the Low Delay MDCT filter bank utilized in the AAC
ELD
encoder. The core MDCT algorithm is mostly unchanged, but with a longer window, such that n is now running from ¨N to N-1 (rather than from 0 to N-1) The spectral coefficient, Xi,k, are defined as follows:
N 27r X i.k = --2 = E z,..õ cos( __________ v2 + no)(k ¨I)) for 0 k < N/2 where:
zin = windowed input sequence N = sample index K = spectral coefficient index I = block index N = window length no (-N / 2 + 1) / 2 Date Recue/Date Received 2022-03-01 The window length N (based on the sine window) is 1024 or 960.
The window length of the low-delay window is 2*N. The windowing is extended to the past in the following way:
zi,(N ¨1¨ n) x'for for n=-N,...,N-1, with the synthesis window w used as the analysis window by inverting the order.
A.4 Low Delay MOM" Synthesis The synthesis filter bank is modified compared to the standard IMDCT algorithm using a sine window in order to adopt a low-delay filter bank. The core IMDCT
algorithm is mostly unchanged, but with a longer window, such that n is now running up to 2N-1 (rather than up to N-1).
2 -2-µ71 xf,õ =.= 4.it specti][k]cosi 7¨(n + no) k ,for 0 n< 2N
where:
n = sample index i = window index k = spectral coefficient index N = window length / twice the frame length no = (-N / 2 + 1) / 2 with N = 960 or 1024.
The windowing and overlap-add is Conducted in the following way:
The length N window is replaced by a length 2N window with more overlap in the past, and less overlap to the future (N/8 values are actually zero).
Windowing for the Low Delay Window:
zem = W1(n) =
Date Recue/Date Received 2022-03-01 Where the window now has a length of 2N, hence n=0,...,2N-1.
Overlap and add:
uuL1,, +z ,v = N
for 0<=n<N/2 Here, the paragraphs proposed for being included into 14496-3:2009 via amendment end.
Naturally, the above description of a possible downscaled mode for AAC-ELD
merely represents one embodiment of the present application and several modifications are feasible. Generally, embodiments of the present application are not restricted to an audio decoder performing a downscaled version of AAC-ELD decoding. In other words, embodiments of the present application may, for instance, be derived by forming an audio decoder capable of performing the inverse transformation process in a downscaled manner only without supporting or using the various AAC-ELD specific further tasks such as, for instance, the scale factor-based transmission of the spectral envelope, TNS
(temporal noise shaping) filtering, spectral band replication (SBR) or the like.
Subsequently, a more general embodiment for an audio decoder is described. The above-outlined example for an AAC-ELD audio decoder supporting the described downscaled mode could thus represent an implementation of the subsequently described audio decoder. In particular, the subsequently explained decoder is shown in Fig. 2 while Fig. 3 illustrates the steps performed by the decoder of Fig. 2.
The audio decoder of Fig. 2, which is generally indicated using reference sign
10, comprises a receiver 12, a grabber 14, a spectral-to-time modulator 16, a windower 18 and a time domain aliasing canceler 20, all of which are connected in series to each other in the order of their mentioning. The interaction and functionality of blocks 12 to 20 of audio decoder 10 are described in the following with respect to Fig. 3. As described at the end of the description of the present application. blocks 12 to 20 may be implemented in software, programmable hardware or hardware such as in the form of a computer program. an FPGA or appropriately programmed computer, programmed microprocessor Date Recue/Date Received 2022-03-01 or application specific integrated circuit with the blocks 12 to 20 representing respective subroutines, circuit paths or the like.
In a manner outlined in more details below, the audio decoder 10 of Fig. 2 is configured to, - and the elements of the audio decoder 10 are configured to appropriately cooperate - in order to decode an audio signal 22 from a data stream 24 with a noteworthiness that audio decoder 10 decodes signal 22 at a sampling rate being 1/Ph of the sampling rate at which the audio signal 22 has been transform coded into data stream 24 at the encoding side. F may, for instance, be any rational number greater than one. The audio decoder may be configured to operate at different or varying downscaling factors F or at a fixed one. Alternatives are described in more detail below.
The manner in which the audio signal 22 is transform coded at the encoding or original sampling rate into the data stream is illustrated in Fig. 3 in the upper half.
At 26 Fig. 3 illustrates the spectral coefficients using small boxes or squares 28 arranged in a spectrotemporal manner along a time axis 30 which runs horizontally In Fig. 3, and a frequency axis 32 which runs vertically in Fig. 3, respectively. The spectral coefficients 28 are transmitted within data stream 24. The manner in which the spectral coefficients 28 have been obtained, and thus the manner via which the spectral coefficients 28 represent the audio signal 22, is illustrated in Fig. 3 at 34, which illustrates for a portion of time axis how the spectral coefficients 28 belonging to, or representing the respective time portion, have been obtained from the audio signal.
In particular, coefficients 28 as transmitted within data stream 24 are coefficients of a 25 lapped transform of the audio signal 22 so that the audio signal 22, sampled at the original or encoding sampling rate, is partitioned into immediately temporally consecutive and non-overlapping frames of a predetermined length N, wherein N spectral coefficients are transmitted in data stream 24 for each frame 36. That is, transform coefficients 28 are obtained from the audio signal 22 using a critically sampled lapped transform.
In the 30 spectrotemporal spectrogram representation 26, each column of the temporal sequence of columns of spectral coefficients 28 corresponds to a respective one of frames 36 of the sequence of frames. The N spectral coefficients 28 are obtained for the corresponding frame 36 by a spectrally decomposing transform or time-to-spectral modulation, the modulation functions of which temporally extend, however, not only across the frame 36 to which the resulting spectral coefficients 28 belong, but also across E + 1 previous frames, wherein E may be any integer or any even numbered integer greater than zero.
That is, Date Recue/Date Received 2022-03-01 the spectral coefficients 28 of one column of the spectrogram at 26 which belonged to a certain frame 36 are obtained by applying a transform onto a transform window, which in addition the respective frame comprises E
1 frames lying in the past relative to the current frame. The spectral decomposition of the samples of the audio signal within this transform window 38, which is illustrated in Fig. 3 for !he column of transform coefficients 28 belonging to the middle frame 36 of the portion shown at 34 is achieved using a low delay unirriodal analysis window function 40 using which the spectral samples within the transform window 38 are weighted prior to subjecting same to an MDCT or MDST
or other spectral decomposition transform. In order to lower the encoder-side delay, the analysis window 40 comprises a zero-interval 42 at the temporal leading end thereof so that the encoder does not need to await the corresponding portion of newest samples within the current frame 36 so as to compute the spectral coefficients 28 for this current frame 36.
That is, within the zero-interval 42 the low delay window function 40 is zero or has zero window coefficients so that the co-located audio samples of the current frame 36 do not, owing to the window weighting 40, contribute to the transform coefficients 28 transmitted for that frame and a data stream 24. That is, summarizing the above, transform coefficients 28 belonging to a current frame 36 are obtained by windowing and spectral decomposition of samples of the audio signal within a transform window 38 which comprises the current frame as well as temporally preceding frames and which temporally overlaps with the corresponding transform windows used for determining the spectral coefficients 28 belonging to temporally neighboring frames.
Before resuming the description of the audio decoder 10, it should be noted that the description of the transmission of the spectral coefficients 28 within the data stream 24 as provided so far has been simplified with respect to the manner in which the spectral coefficients 28 are quantized or coded into data stream 24 and/or the manner in which the audio signal 22 has boon preprocessed before subjecting the audio signal to the lapped transform. For example, the audio encoder having transform coded audio signal 22 into data stream 24 may be controlled via a psychoacoustic model or may use a psychoacoustic model to keep the quantization noise and quantizing the spectral coefficients 28 unperceivable for the hearer and/or below a masking threshold function, thereby determining scale factors for spectral bands using which the quantized and transmitted spectral coefficients 28 are scaled. The scale factors would also be signaled in data stream 24. Alternatively, the audio encoder may have been a TCX
(transform coded excitation) type of encoder. Then, the audio signal would have had subject to a linear prediction analysis filtering before forming the spectrotemporal representation 26 of Date Recue/Date Received 2022-03-01 spectral coefficients 28 by applying the lapped transform onto the excitation signal. i.e. the linear prediction residual signal. For example, the linear prediction coefficients could be signaled in data stream 24 as well, and a spectral uniform quantization could be applied in order to obtain the spectral coefficients 28.
Furthermore, the description brought forward so far has also been simplified with respect to the frame length of frames 36 and/or with respect to the low delay window function 40.
In fact, the audio signal 22 may have been coded into data stream 24 in a manner using varying frame sizes and/or different windows 40. However, the description brought 10 forward in the following concentrates on one window 40 and one frame length, although the subsequent description may easily be extended to a case where the entropy encoder changes these parameters during coding the audio signal into the data stream.
Returning back to the audio decoder 10 of Fig. 2 and its description, receiver 12 receives 15 data stream 24 and receives thereby, for each frame 36, N spectral coefficients 28, i.e. a respective column of coefficients 28 shown in Fig. 3. It should be recalled that the temporal length of the frames 36, measured in samples of the original or encoding sampling rate, is N as indicated in Fig. 3 at 34, but the audio decoder 10 of Fig. 2 is configured to decode the audio signal 22 at a reduced sampling rate. The audio decoder 10 supports, for example, merely this downscaled decoding functionality described in the following. Alternatively, audio decoder 10 would be able to reconstruct the audio signal at the original or encoding sampling rate, but may be switched between the downscaled decoding mode and a non-downscaled decoding mode with the downscaled decoding mode coinciding with the audio decoder's 10 mode of operation as subsequently explained. For example, audio encoder 10 could be switched to a downscaled decoding mode in the case of a low battery level, reduced reproduction environment capabilities or the like. Whenever the situation changes the audio decoder 10 could, for instance, switch back from the downscaled decoding mode to the non-downscaled one. In any case, in accordance with the downscaled decoding process of decoder 10 as described in the following, the audio signal 22 is reconstructed at a sampling rate at which frames 36 have, at the reduced sampling rate, a lower length measured in samples of this reduced sampling rate, namely a length of N/F samples at the reduced sampling rate.
The output of receiver 12 is the sequence of N spectral coefficients, namely one set of N
spectral coefficients, i.e. one column in Fig. 3, per frame 36. It already turned out from the above brief description of the transform coding process for forming data stream 24 that Date Recue/Date Received 2022-03-01 receiver 12 may apply various tasks in obtaining the N spectral coefficients per frame 36.
For example, receiver 12 may use entropy decoding in order to read the spectral coefficients 28 from the data stream 24 Receiver 12 may also spectrally shape the spectral coefficients read from the data stream with scale factors provided in the data stream and/or scale factors derived by linear prediction coefficients conveyed within data stream 24. For example, receiver 12 may obtain scale factors from the data stream 24, namely on a per frame and per subband basis, and use these scale factors in order to scale the scale factors conveyed within the data stream 24. Alternatively, receiver 12 may derive scale factors from linear prediction coefficients conveyed within the data stream 24, for each frame 36, and use these scale factors in order to scale the transmitted spectral coefficients 28. Optionally, receiver 12 may perform gap filling in order to synthetically fill zero-quantized portions within the sets of N spectral coefficients 18 per frame. Additionally or alternatively, receiver 12 may apply a TNS-synthesis filter onto a transmitted TNS filter coefficient per frame to assist the reconstruction of the spectral coefficients 28 from the data stream with the TNS coefficients also being transmitted within the data stream 24.
The just outlined possible tasks of receiver 12 shall be understood as a non-exclusive list of possible measures and receiver 12 may perform further or other tasks in connection with the reading of the spectral coefficients 28 from data stream 24.
Grabber 14 thus receives from receiver 12 the spectrogram 26 of spectral coefficients 28 and grabs, for each frame 36, a low frequency fraction 44 of the N spectral coefficients of the respective frame 36, namely the N/F lowest-frequency spectral coefficients.
That is, spectral-to-time modulator 16 receives from grabber 14 a stream or sequence 46 of N/F spectral coefficients 28 per frame 36, corresponding to a low-frequency slice out of the spectrogram 26, spectrally registered to the lowest frequency spectral coefficients illustrated using index "0" in Fig. 3, and extending till the spectral coefficients of index N/F
¨1.
The spectral-to-time modulator 16 subjects, for each frame 36, the corresponding low-frequency fraction 44 of spectral coefficients 28 to an inverse transform 48 having modulation functions of length (E 2) = N/F temporally extending over the respective frame and E 1 previous frames as illustrated at 50 in Fig. 3. thereby obtaining a temporal portion of length (E + 2) = N/F, i.e. a not-yet windowed time segment 52. That is, the spectral-to-time modulator may obtain a temporal time segment of (E 2) -N/F
samples of reduced sampling rate by weighting and summing modulation functions of the Date Recue/Date Received 2022-03-01 same length using, for instance, the first formulae of the proposed replacement section A.4 indicated above. The newest N/F samples of time segment 52 belong to the current frame 36. The modulation functions may, as indicated, be cosine functions in case of the inverse transform being an inverse MDCT, or sine functions in case of the inverse transform being an inverse MDCT, for instance Thus, windower 52 receives, for each frame, a temporal portion 52, the N/F
samples at the leading end thereof temporally corresponding to the respective frame while the other samples of the respective temporal portion 52 belong to the corresponding temporally preceding frames_ Windower 18 windows, for each frame 36, the temporal portion using a unirnodal synthesis window 54 of length (E + 2) - N/F comprising a zero-portion 56 of length 1/4 = N/F at a leading end thereof, i.e. 1/F = N/F zero-valued window coefficients, and having a peak 58 within its temporal interval succeeding, temporally, the zero-portion 56, i.e. the temporal interval of temporal portion 52 not covered by the zero-portion 52.
The latter temporal interval may be called the non-zero portion of window 58 and has a length of 7/4 = N/F measured in samples of the reduced sampling rate, i.e. 7/4 = N/F
window coefficients. The windower 18 weights, for instance, the temporal portion 52 using window 58. This weighting or multiplying 58 of each temporal portion 52 with window 54 results in a windowed temporal portion 60, one for each frame 36, and coinciding with the respective temporal portion 52 as far as the temporal coverage is concerned.
In the above proposed section A.4, the windowing processing which may be used by window 18 is described by the formulae relating zi,,, to where Xim corresponds to the aforementioned temporal portions 52 not yet windowed and Zi,n corresponds to the windowed temporal portions 60 with i indexing the sequence of frames/windows, and n indexing, within each temporal portion 52/60, the samples or values of the respective portions 52/60 in accordance with a reduced sampling rate.
Thus, the time domain aliasing canceler 20 receives from windower 18 a sequence of windowed temporal portions 60, namely one per frame 36. Canceler 20 subjects the .. windowed temporal portions 60 of frames 36 to an overlap-add process 62 by registering each windowed temporal portion 60 with its leading N/F values to coincide with the corresponding frame 36. By this measure, a trailing-end fraction of length (E
+ 1)/(E + 2) of the windowed temporal portion 60 of a current frame, i.e. the remainder having length (E + 1). N/F, overlaps with a corresponding equally long leading end of the temporal .. portion of the immediately preceding frame. In formulae, the time domain aliasing canceler 20 may operate as shown in the last formula of the above proposed version of section Date Recue/Date Received 2022-03-01 A.4, where out:, corresponds to the audio samples of the reconstructed audio signal 22 at the reduced sampling rate.
The processes of windowing 58 and overlap-adding 62 as performed by windower 18 and time domain aliasing canceler 20 are illustrated in more detail below with respect to Fig. 4.
Fig. 4 uses both the nomenclature applied in the above-proposed section A.4 and the reference signs applied in Figs. 3 and 4. X
0,0 to - le, N/F-1 represents the O'h temporal portion 52 obtained by the spatial-to-temporal-modulator 16 for the 01h frame 36. The first index of x indexes the frames 36 along the temporal order, and the second index of x .. orders the samples of the temporal along the temporal order, the inter-sample pitch belonging to the reduced sample rate. Then, in Fig. 4, -o to w w -(E4-2) N/F-1 indicate the window coefficients of window 54. Like the second index of x, i.e. the temporal portion 52 as output by modulator 16, the index of w is such that index 0 corresponds to the oldest and index (E + 2) N/F - 1 corresponds to the newest sample value when the window 54 is applied to the respective temporal portion 52. Windower 18 windows the temporal portion 52 using window 54 to obtain the windowed temporal portion 60 so that z0.0 to ZO.(E+2).N/F-1, which denotes the windowed temporal portion 60 for the Ow frame, is obtained according to z0,0 zr- x0.0 = wo, .=., 20.(E+2).N/F-1 = X0.(E+2)+1/F-1 W(E+2)-N/F-1. The indices of z have the same meaning as for x. In this manner, modulator 16 and windower 18 act for each frame indexed by the first index of x and z. Canceler 20 sums up E + 2 windowed temporal portions 60 of E + 2 immediately consecutive frames with offsetting the samples of the windowed temporal portions 60 relative to each other by one frame, i.e. by the number of samples per frame 36, namely N/F, so as to obtain the samples u of one current frame, here u4E.e.0
In a manner outlined in more details below, the audio decoder 10 of Fig. 2 is configured to, - and the elements of the audio decoder 10 are configured to appropriately cooperate - in order to decode an audio signal 22 from a data stream 24 with a noteworthiness that audio decoder 10 decodes signal 22 at a sampling rate being 1/Ph of the sampling rate at which the audio signal 22 has been transform coded into data stream 24 at the encoding side. F may, for instance, be any rational number greater than one. The audio decoder may be configured to operate at different or varying downscaling factors F or at a fixed one. Alternatives are described in more detail below.
The manner in which the audio signal 22 is transform coded at the encoding or original sampling rate into the data stream is illustrated in Fig. 3 in the upper half.
At 26 Fig. 3 illustrates the spectral coefficients using small boxes or squares 28 arranged in a spectrotemporal manner along a time axis 30 which runs horizontally In Fig. 3, and a frequency axis 32 which runs vertically in Fig. 3, respectively. The spectral coefficients 28 are transmitted within data stream 24. The manner in which the spectral coefficients 28 have been obtained, and thus the manner via which the spectral coefficients 28 represent the audio signal 22, is illustrated in Fig. 3 at 34, which illustrates for a portion of time axis how the spectral coefficients 28 belonging to, or representing the respective time portion, have been obtained from the audio signal.
In particular, coefficients 28 as transmitted within data stream 24 are coefficients of a 25 lapped transform of the audio signal 22 so that the audio signal 22, sampled at the original or encoding sampling rate, is partitioned into immediately temporally consecutive and non-overlapping frames of a predetermined length N, wherein N spectral coefficients are transmitted in data stream 24 for each frame 36. That is, transform coefficients 28 are obtained from the audio signal 22 using a critically sampled lapped transform.
In the 30 spectrotemporal spectrogram representation 26, each column of the temporal sequence of columns of spectral coefficients 28 corresponds to a respective one of frames 36 of the sequence of frames. The N spectral coefficients 28 are obtained for the corresponding frame 36 by a spectrally decomposing transform or time-to-spectral modulation, the modulation functions of which temporally extend, however, not only across the frame 36 to which the resulting spectral coefficients 28 belong, but also across E + 1 previous frames, wherein E may be any integer or any even numbered integer greater than zero.
That is, Date Recue/Date Received 2022-03-01 the spectral coefficients 28 of one column of the spectrogram at 26 which belonged to a certain frame 36 are obtained by applying a transform onto a transform window, which in addition the respective frame comprises E
1 frames lying in the past relative to the current frame. The spectral decomposition of the samples of the audio signal within this transform window 38, which is illustrated in Fig. 3 for !he column of transform coefficients 28 belonging to the middle frame 36 of the portion shown at 34 is achieved using a low delay unirriodal analysis window function 40 using which the spectral samples within the transform window 38 are weighted prior to subjecting same to an MDCT or MDST
or other spectral decomposition transform. In order to lower the encoder-side delay, the analysis window 40 comprises a zero-interval 42 at the temporal leading end thereof so that the encoder does not need to await the corresponding portion of newest samples within the current frame 36 so as to compute the spectral coefficients 28 for this current frame 36.
That is, within the zero-interval 42 the low delay window function 40 is zero or has zero window coefficients so that the co-located audio samples of the current frame 36 do not, owing to the window weighting 40, contribute to the transform coefficients 28 transmitted for that frame and a data stream 24. That is, summarizing the above, transform coefficients 28 belonging to a current frame 36 are obtained by windowing and spectral decomposition of samples of the audio signal within a transform window 38 which comprises the current frame as well as temporally preceding frames and which temporally overlaps with the corresponding transform windows used for determining the spectral coefficients 28 belonging to temporally neighboring frames.
Before resuming the description of the audio decoder 10, it should be noted that the description of the transmission of the spectral coefficients 28 within the data stream 24 as provided so far has been simplified with respect to the manner in which the spectral coefficients 28 are quantized or coded into data stream 24 and/or the manner in which the audio signal 22 has boon preprocessed before subjecting the audio signal to the lapped transform. For example, the audio encoder having transform coded audio signal 22 into data stream 24 may be controlled via a psychoacoustic model or may use a psychoacoustic model to keep the quantization noise and quantizing the spectral coefficients 28 unperceivable for the hearer and/or below a masking threshold function, thereby determining scale factors for spectral bands using which the quantized and transmitted spectral coefficients 28 are scaled. The scale factors would also be signaled in data stream 24. Alternatively, the audio encoder may have been a TCX
(transform coded excitation) type of encoder. Then, the audio signal would have had subject to a linear prediction analysis filtering before forming the spectrotemporal representation 26 of Date Recue/Date Received 2022-03-01 spectral coefficients 28 by applying the lapped transform onto the excitation signal. i.e. the linear prediction residual signal. For example, the linear prediction coefficients could be signaled in data stream 24 as well, and a spectral uniform quantization could be applied in order to obtain the spectral coefficients 28.
Furthermore, the description brought forward so far has also been simplified with respect to the frame length of frames 36 and/or with respect to the low delay window function 40.
In fact, the audio signal 22 may have been coded into data stream 24 in a manner using varying frame sizes and/or different windows 40. However, the description brought 10 forward in the following concentrates on one window 40 and one frame length, although the subsequent description may easily be extended to a case where the entropy encoder changes these parameters during coding the audio signal into the data stream.
Returning back to the audio decoder 10 of Fig. 2 and its description, receiver 12 receives 15 data stream 24 and receives thereby, for each frame 36, N spectral coefficients 28, i.e. a respective column of coefficients 28 shown in Fig. 3. It should be recalled that the temporal length of the frames 36, measured in samples of the original or encoding sampling rate, is N as indicated in Fig. 3 at 34, but the audio decoder 10 of Fig. 2 is configured to decode the audio signal 22 at a reduced sampling rate. The audio decoder 10 supports, for example, merely this downscaled decoding functionality described in the following. Alternatively, audio decoder 10 would be able to reconstruct the audio signal at the original or encoding sampling rate, but may be switched between the downscaled decoding mode and a non-downscaled decoding mode with the downscaled decoding mode coinciding with the audio decoder's 10 mode of operation as subsequently explained. For example, audio encoder 10 could be switched to a downscaled decoding mode in the case of a low battery level, reduced reproduction environment capabilities or the like. Whenever the situation changes the audio decoder 10 could, for instance, switch back from the downscaled decoding mode to the non-downscaled one. In any case, in accordance with the downscaled decoding process of decoder 10 as described in the following, the audio signal 22 is reconstructed at a sampling rate at which frames 36 have, at the reduced sampling rate, a lower length measured in samples of this reduced sampling rate, namely a length of N/F samples at the reduced sampling rate.
The output of receiver 12 is the sequence of N spectral coefficients, namely one set of N
spectral coefficients, i.e. one column in Fig. 3, per frame 36. It already turned out from the above brief description of the transform coding process for forming data stream 24 that Date Recue/Date Received 2022-03-01 receiver 12 may apply various tasks in obtaining the N spectral coefficients per frame 36.
For example, receiver 12 may use entropy decoding in order to read the spectral coefficients 28 from the data stream 24 Receiver 12 may also spectrally shape the spectral coefficients read from the data stream with scale factors provided in the data stream and/or scale factors derived by linear prediction coefficients conveyed within data stream 24. For example, receiver 12 may obtain scale factors from the data stream 24, namely on a per frame and per subband basis, and use these scale factors in order to scale the scale factors conveyed within the data stream 24. Alternatively, receiver 12 may derive scale factors from linear prediction coefficients conveyed within the data stream 24, for each frame 36, and use these scale factors in order to scale the transmitted spectral coefficients 28. Optionally, receiver 12 may perform gap filling in order to synthetically fill zero-quantized portions within the sets of N spectral coefficients 18 per frame. Additionally or alternatively, receiver 12 may apply a TNS-synthesis filter onto a transmitted TNS filter coefficient per frame to assist the reconstruction of the spectral coefficients 28 from the data stream with the TNS coefficients also being transmitted within the data stream 24.
The just outlined possible tasks of receiver 12 shall be understood as a non-exclusive list of possible measures and receiver 12 may perform further or other tasks in connection with the reading of the spectral coefficients 28 from data stream 24.
Grabber 14 thus receives from receiver 12 the spectrogram 26 of spectral coefficients 28 and grabs, for each frame 36, a low frequency fraction 44 of the N spectral coefficients of the respective frame 36, namely the N/F lowest-frequency spectral coefficients.
That is, spectral-to-time modulator 16 receives from grabber 14 a stream or sequence 46 of N/F spectral coefficients 28 per frame 36, corresponding to a low-frequency slice out of the spectrogram 26, spectrally registered to the lowest frequency spectral coefficients illustrated using index "0" in Fig. 3, and extending till the spectral coefficients of index N/F
¨1.
The spectral-to-time modulator 16 subjects, for each frame 36, the corresponding low-frequency fraction 44 of spectral coefficients 28 to an inverse transform 48 having modulation functions of length (E 2) = N/F temporally extending over the respective frame and E 1 previous frames as illustrated at 50 in Fig. 3. thereby obtaining a temporal portion of length (E + 2) = N/F, i.e. a not-yet windowed time segment 52. That is, the spectral-to-time modulator may obtain a temporal time segment of (E 2) -N/F
samples of reduced sampling rate by weighting and summing modulation functions of the Date Recue/Date Received 2022-03-01 same length using, for instance, the first formulae of the proposed replacement section A.4 indicated above. The newest N/F samples of time segment 52 belong to the current frame 36. The modulation functions may, as indicated, be cosine functions in case of the inverse transform being an inverse MDCT, or sine functions in case of the inverse transform being an inverse MDCT, for instance Thus, windower 52 receives, for each frame, a temporal portion 52, the N/F
samples at the leading end thereof temporally corresponding to the respective frame while the other samples of the respective temporal portion 52 belong to the corresponding temporally preceding frames_ Windower 18 windows, for each frame 36, the temporal portion using a unirnodal synthesis window 54 of length (E + 2) - N/F comprising a zero-portion 56 of length 1/4 = N/F at a leading end thereof, i.e. 1/F = N/F zero-valued window coefficients, and having a peak 58 within its temporal interval succeeding, temporally, the zero-portion 56, i.e. the temporal interval of temporal portion 52 not covered by the zero-portion 52.
The latter temporal interval may be called the non-zero portion of window 58 and has a length of 7/4 = N/F measured in samples of the reduced sampling rate, i.e. 7/4 = N/F
window coefficients. The windower 18 weights, for instance, the temporal portion 52 using window 58. This weighting or multiplying 58 of each temporal portion 52 with window 54 results in a windowed temporal portion 60, one for each frame 36, and coinciding with the respective temporal portion 52 as far as the temporal coverage is concerned.
In the above proposed section A.4, the windowing processing which may be used by window 18 is described by the formulae relating zi,,, to where Xim corresponds to the aforementioned temporal portions 52 not yet windowed and Zi,n corresponds to the windowed temporal portions 60 with i indexing the sequence of frames/windows, and n indexing, within each temporal portion 52/60, the samples or values of the respective portions 52/60 in accordance with a reduced sampling rate.
Thus, the time domain aliasing canceler 20 receives from windower 18 a sequence of windowed temporal portions 60, namely one per frame 36. Canceler 20 subjects the .. windowed temporal portions 60 of frames 36 to an overlap-add process 62 by registering each windowed temporal portion 60 with its leading N/F values to coincide with the corresponding frame 36. By this measure, a trailing-end fraction of length (E
+ 1)/(E + 2) of the windowed temporal portion 60 of a current frame, i.e. the remainder having length (E + 1). N/F, overlaps with a corresponding equally long leading end of the temporal .. portion of the immediately preceding frame. In formulae, the time domain aliasing canceler 20 may operate as shown in the last formula of the above proposed version of section Date Recue/Date Received 2022-03-01 A.4, where out:, corresponds to the audio samples of the reconstructed audio signal 22 at the reduced sampling rate.
The processes of windowing 58 and overlap-adding 62 as performed by windower 18 and time domain aliasing canceler 20 are illustrated in more detail below with respect to Fig. 4.
Fig. 4 uses both the nomenclature applied in the above-proposed section A.4 and the reference signs applied in Figs. 3 and 4. X
0,0 to - le, N/F-1 represents the O'h temporal portion 52 obtained by the spatial-to-temporal-modulator 16 for the 01h frame 36. The first index of x indexes the frames 36 along the temporal order, and the second index of x .. orders the samples of the temporal along the temporal order, the inter-sample pitch belonging to the reduced sample rate. Then, in Fig. 4, -o to w w -(E4-2) N/F-1 indicate the window coefficients of window 54. Like the second index of x, i.e. the temporal portion 52 as output by modulator 16, the index of w is such that index 0 corresponds to the oldest and index (E + 2) N/F - 1 corresponds to the newest sample value when the window 54 is applied to the respective temporal portion 52. Windower 18 windows the temporal portion 52 using window 54 to obtain the windowed temporal portion 60 so that z0.0 to ZO.(E+2).N/F-1, which denotes the windowed temporal portion 60 for the Ow frame, is obtained according to z0,0 zr- x0.0 = wo, .=., 20.(E+2).N/F-1 = X0.(E+2)+1/F-1 W(E+2)-N/F-1. The indices of z have the same meaning as for x. In this manner, modulator 16 and windower 18 act for each frame indexed by the first index of x and z. Canceler 20 sums up E + 2 windowed temporal portions 60 of E + 2 immediately consecutive frames with offsetting the samples of the windowed temporal portions 60 relative to each other by one frame, i.e. by the number of samples per frame 36, namely N/F, so as to obtain the samples u of one current frame, here u4E.e.0
11-(E+1),N/F-1). Here, again, the first index of u indicates the frame number and the second index orders the samples of this frame along the temporal order.
The canceller Joins the reconstructed frames thus obtained so that the samples of the reconstructed audio signal 22 within the consecutive frames 36 follow each other according to uiee1).0 U..(E+1),1/F-1. U-E,N/F-1, UAE-1).0, . the canceler 22 computes each sample of the audio signal 22 within the -(E+1)th frame according to UlEtt),t) = Z0,0 Z.1,14/F + =
Z-(E+1),(E+1) NW. = = = .
U{E+1)N/F-1 = ZO.N/F-1 Z-1,2=N/F-1 1 "I' Z-(E+1)4E+2)No--1, i.e. summing up (e+2) addends per samples u of the current frame.
Fig. 5 illustrates a possible exploitation of the fact that, among the just windowed samples contributing to the audio samples u of frame -(E + 1), the ones corresponding to, or having been windowed using, the zero-portion 56 of window 54, namely z (p...1).(e.7/1).e1F -Z-(E. 1).(E-1-2), are zero valued. Thus, instead of obtaining all N/F samples within the -Date Recue/Date Received 2022-03-01 (E+1)th frame 36 of the audio signal u using E+2 addends, canceler 20 may compute the leading end quarter thereof, namely u-go1me=7/4) NH' U.(Esl).(li),IF.I merely using E+1 addends according to ii-ge+ime.,n4)-NtP = Zo.314. NW 4* Z-1.7/4 Nft + = = = +
Z-E.(E.3/4)14/F. = = =
r,//t,.1 ZO.N/F-1 Z-1,2=14/F-1 In this manner, the windower could even leave out, effectively, the performance of the weighting 58 with respect to the zero-portion 56. Samples u.(E.1),(s-,7/4) 1+1/F
IME*2) Nif.) of current --(E+1)"' frame would, thus, be obtained using E.+1 addends only, while u4e, 1),(E.7/4)-N,F-1 would be obtained using E+2 addends.
Thus, in the manner outlined above, the audio decoder 10 of Fig. 2 reproduces, in a downscaled manner, the audio signal coded into data stream 24. To this end, the audio decoder 10 uses a window function 54 which is itself a downsampled version of a reference synthesis window of length (E+2)-N. As explained with respect to Fig. 6, this downsampled version, i.e. window 54, is obtained by downsampling the reference synthesis window by a factor of F, i.e. the downsampling factor, using a segmental interpolation, namely in segments of length 1/4-N when measured in the not yet downscalecl regime, in segments of length 1/4=N/F in the downsampled regime, in segments of quarters of a frame length of frames 36, measured temporally and expressed independently from the sampling rate. In 4 = (E+2) the interpolation is, thus, performed, thus yielding 4 = (E+2) times 1/4=N/F long segments which, concatenated, represent the downsampled version of the reference synthesis window of length (E+2)-N. See Fig. 6 for illustration. Fig. 6 shows the synthesis window 54 which is unimodal and used by the audio decoder 10 in accordance with a downsampled audio decoding procedure underneath the reference synthesis window 70 which his of length (El-2)-N.
That is, by the downsampling procedure 72 leading from the reference synthesis window 70 to the synthesis window 54 actually used by the audio decoder 10 for downsampled decoding, the number of window coefficients is reduced by a factor of F. In Fig. 6, the nomenclature of Figs. 5 and 6 has been adhered to, i.e. w is used in order to denote the downsampled version window 54, while w' has been used to denote the window coefficients of the reference synthesis window 70.
As just mentioned, in order to perform the downsampling 72, the reference synthesis window 70 is processed in segments 74 of equal length. In number, there are (E+2)=4 such segments 74. Measured in the original sampling rate, i.e. in the number of window coefficients of the reference synthesis window 70, each segment 74 is 1/4 = N
window Date Recue/Date Received 2022-03-01 coefficients w' long, and measured in the reduced or downsampled sampling rate, each segment 74 is 1/4-N/F window coefficients w long.
Naturally, it would be possible to perform the downsampling 72 for each downsarnple.d 5 window coefficient wi coinciding accidentally with any of the window coefficients w; of the reference synthesis window 70 by simply setting w, = w; with the sample time of w1 coinciding with that of wl, and/or by linearly interpolating any window coefficients wi residing, temporally, between two window coefficients w; and w1+2 by linear interpolation, but this procedure would result in a poor approximation of the reference synthesis window 10 70, i.e. the synthesis window 54 used by audio decoder 10 for the downsampled decoding would represent a poor approximation of the reference synthesis window 70, thereby not fulfilling the request for guaranteeing conformance testing of the downscaled decoding relative to the non-downscaled decoding of the audio signal from data stream 24. Thus, the downsampling 72 involves an interpolation procedure according to which the majority 15 of the window coefficients %xi of the downsampled window 54, namely the ones positioned offset from the borders of segments 74, depend by way of the downsampling procedure 72 on more than two window coefficients w' of the reference window 70. In particular, while the majority of the window coefficients wi of the downsampled window 54 depend on more than two window coefficients w; of the reference window 70 in order to increase the 20 quality of the interpolation/downsampling result, i.e. the approximation quality, for every window coefficient wi of the downsampled version 54 it holds true that same does not depend in window coefficients w; belonging to different segments 74. Rather, the downsampling procedure 72 is a segmental interpolation procedure.
For example, the synthesis window 54 may be a concatenation of spline functions of length 1/4 N/F. Cubic spline functions may be used. Such an example has been outlined above in section A.1 where the outer for-next loop sequentially looped over segments 74 wherein, in each segment 74, the downsampling or Interpolation 72 involved a mathematical combination of consecutive window coefficients w' within the current :30 segment 74 at, for example, the first for next clause in the section "calculate vector r needed to calculate the coefficients c". The interpolation applied in segments, may, however, also be chosen differently. That is, the interpolation is not restricted to splines or cubic splines. Rather, linear interpolation or any other interpolation method may be used as well. In any case, the segmental implementation of the interpolation would cause the computation of samples of the downscaled synthesis window, i.e. the outmost samples of Date Recue/Date Received 2022-03-01 the segments of the downscaled synthesis window, neighboring another segment, to not depend on window coefficients of the reference synthesis window residing in different segments.
it may be that windower 18 obtains the downsampled synthesis window 54 from a storage whore the window coefficients wi of this downsampled synthesis window 54 have been stored after having been obtained using the downsampling 72. Alternatively, as illustrated in Fig. 2, the audio decoder 10 may comprise a segmental clownsampler 76 performing the downsampling 72 of Fig. 6 on the basis of the reference synthesis window 70.
It should be noted that the audio decoder 10 of Fig. 2 may be configured to support merely one fixed downsampling factor F or may support different values. In that case, the audio decoder 10 may be responsive to an input value for F as illustrated in Fig. 2 at 78.
The grabber 14, for instance, may be responsive to this value F in order to grab, as mentioned above, the N/F spectral values per frame spectrum. In a like manner, the optional segmental downsampler 76 may also be responsive to this value of F an operate as indicated above. The SIT modulator 16 may be responsive to F either in order to, for example, computationally derive clownscaled/downsampled versions of the modulation functions, downscaled/downsampled relative to the ones used in not-downscaled operation mode where the reconstruction leads to the full audio sample rate.
Naturally, the modulator 16 would also be responsive to F input 78, as modulator 16 would use appropriately downsampled versions of the modulation functions and the same holds true for the windower 18 and canceler 20 with respect to an adaptation of the actual length of the frames in the reduced or downsampled sampling rate.
For example, F may lie between 1.5 and 10, both inclusively.
It should be noted that the decoder of Fig. 2 and 3 or any modification thereof outlined herein, may be implemented so as to perform the spectral-to-time transition using a lifting implementation of the Low Delay MDCT as taught in, for example, EP 2 378 516 81.
Fig. 8 illustrates an implementation of the decoder using the lifting concept.
The SIT
modulator 16 performs exemplarily an inverse DCT-IV and is shown as followed by a block representing the concatenation of the windower 18 and the time domain aliasing canceller 20. In the example of Fig. 8 E is 2, i.e. E=2.
Date Recue/Date Received 2022-03-01 The modulator 16 comprises an inverse type-iv discrete cosine transform frequency/time converter. Instead of outputing sequences of (E+2)N/F long temporal portions 52, it merely outputs temporal portions 52 of length 2=N/F, all derived from the sequence of N/F
long spectra 46, these shortened portions 52 corresponding to the DCT kernel, i.e. the 2-N/F newest samples of the erstwhile described portions.
The windower 18 acts as described previously and generates a windowed temporal portion 60 for each temporal portion 52, but it operates merely on the DCT
kernel. To this end, windower 18 uses window function 0.), with i=0...2N/F-1, having the kernel size. The relationship between wi with 10. ..(E+2)-N/F-1 is described later, just as the relationship between the subsequently mentioned lifting coefficients and wi with i=0 ...(E+2)-N/F-1 is.
Using the nomenclature applied above, the process described so far yields:
2k,r; = Wn = Xk,n for n = 0....,2M-1 with redefining M = N/F, so that M corresponds to the frame size expressed in the downscaled domain and using the nomenclature of Fig. 2-6, wherein, however, zk,n and xk,a shall contain merely the samples of the windowed temporal portion and the not-yet windowed temporal portion within the DOT kernel having size 2.M and temporally corresponding to samples EN/F...(E-1-2)-N/F-1 in Fig. 4. That is, n is an integer indicating a sample index and ren is a real-valued window function coefficient corresponding to the sample index n.
The overlap/add process of the canceller 20 operates in a manner different compared to the above description. It generates intermediate temporal portions mk(0),...mk(M-1) based on the equation or expression mk.n = Zk,n Zk-1,n+h4 for n = 0,...,M-1 In the implementation of Fig. 8, the apparatus further comprises a lifter 80 which may be interpreted as a part of the modulator 16 and windower 18 since the lifter 80 compensates the fact the modulator and the windower restricted their processing to the DCT
kernel instead of processing the extension of the modulation functions and the synthesis window beyond the kernel towards the past which extension was introduced to compensate for the Date Recue/Date Received 2022-03-01 zero portion 56. The lifter 80 produces, using a framework of the delayers and multipliers 82 and adders 84, the finally reconstructed temporal portions or frames of length M in pairs of immediately consecutive frames based on the equation or expression LIN.n=-= mk,n "4' 1n-M/2 Mk -1 .01-1 for n = M/2,... ,M-1 , and uk,õ= Mic,n 1m-1-n = OUltic-I.M-1-n for n=0,...,M/2-1 wherein la with n = 0...M-1 arc real-valued lifting coefficients related to the downscaled synthesis window in a manner described in more detail below.
In other words, for the extended overlap of E frames into the past, only M
additional multiplier-add operations are required, as can be seen in the framework of the lifter 80.
These additional operations are sometimes also referred to as "zero-delay matrices".
Sometimes these operations are also known as "lifting steps'. The efficient implementation shown in Fig. 8 may under some circumstances be more efficient as a straightforward implementation. To be more precise, depending on the concrete .. implementation, such a more efficient implementation might result in saving M operations, as in the case of a straightforward implementation for M operations, it might be advisable to implement, as the implementation shown in Fig. 19, requires in principle, 2M operations in the framework of the module 820 and M operations in the framework of the lifter 830.
As to the dependency of con with n=0...2M-1 and In with n = 0...M-1 on the synthesis window vvi with i = 0...(E+2)M-1 (it is recalled that here E=2), the following formulae describe the relationship between them with displacing, however, the subscript indices used so far into the parenthesis following the respective variable:
w(i) = (-2 ¨ 1¨ n) = l(M ¨1 ¨n) = w(M + n) w(M/2 + = 1(n) =1(M/2 + n) = ba(3M/2 + n) w(M + 0 / (¨ ¨ 1¨ n) = 03(it4 + n) w(3M/2 + = ¨1(n) - (0(3 M / 2 + n) w (2M + i) = --(.1)(M + n) 1(M ¨ 1 n) = w(n) w(5M/2 + = ¨(1)(3M/2 + n) 1(M/2 + n) = w(M/2 + n) w(3/i/ 4- = ¨1.&)(n) Date Recue/Date Received 2022-03-01 til(7M/ 2 + i) = w(M 4 n) for i, n = 0 ¨2 ¨ I
Please note that the window we contains the peak values on the right side in this formulation, i.e. between the indices 2M and 4M 1. The above formulae relate coefficients in with n = 0...M-1 and co. n = 0,...,2M-1 to the coefficients wn with n =
0... (E+2)M-1 of the downscaled synthesis window. As can be seen, in with n =
0...M-1 actually merely depend on % of the coefficients of the downsam pled synthesis window, namely on wõ with n = 0...(E+1)M-1, while m, n = 0....,2M-1 depend on all wn with n =
0...(F+2)M-1.
As stated above, it might be that windower 18 obtains the downsampted synthesis window 54 wn with n = 0...(E+2)M-1 from a storage where the window coefficients wi of this downsampled synthesis window 54 have been stored after having been obtained using the downsampling 72, and from where same are read to compute coefficients in with n =
0...M-1 and con n = 0,...,2M-1 using the above relation, but alternatively, winder 18 may retrieve the coefficients In with n = 0...M-1 and con n = 0,...,2M-1, thus computed from the pre-downsampled synthesis window, from the storage directly. Alternatively, as stated above, the audio decoder 10 may comprise the segmental downsampler 76 performing the downsampling 72 of Fig. 6 on the basis of the reference synthesis window 70, thereby yielding wn with n = 0...(E+2)M-1 on the basis of which the windower 18 computes coefficients In with n = 0...M-1 and o.)n n = 0,... ,2M-1 using above relation/formulae. Even using the lifting implementation, more than one value for F may be supported.
Briefly summarizing the lifting implementation, same results in an audio decoder 10 configured to decode an audio signal 22 at a first sampling rate from a data stream 24 into which the audio signal is transform coded at a second sampling rate, the first sampling rate being 1/Fth of the second sampling rate, the audio decoder 10 comprising the receiver
The canceller Joins the reconstructed frames thus obtained so that the samples of the reconstructed audio signal 22 within the consecutive frames 36 follow each other according to uiee1).0 U..(E+1),1/F-1. U-E,N/F-1, UAE-1).0, . the canceler 22 computes each sample of the audio signal 22 within the -(E+1)th frame according to UlEtt),t) = Z0,0 Z.1,14/F + =
Z-(E+1),(E+1) NW. = = = .
U{E+1)N/F-1 = ZO.N/F-1 Z-1,2=N/F-1 1 "I' Z-(E+1)4E+2)No--1, i.e. summing up (e+2) addends per samples u of the current frame.
Fig. 5 illustrates a possible exploitation of the fact that, among the just windowed samples contributing to the audio samples u of frame -(E + 1), the ones corresponding to, or having been windowed using, the zero-portion 56 of window 54, namely z (p...1).(e.7/1).e1F -Z-(E. 1).(E-1-2), are zero valued. Thus, instead of obtaining all N/F samples within the -Date Recue/Date Received 2022-03-01 (E+1)th frame 36 of the audio signal u using E+2 addends, canceler 20 may compute the leading end quarter thereof, namely u-go1me=7/4) NH' U.(Esl).(li),IF.I merely using E+1 addends according to ii-ge+ime.,n4)-NtP = Zo.314. NW 4* Z-1.7/4 Nft + = = = +
Z-E.(E.3/4)14/F. = = =
r,//t,.1 ZO.N/F-1 Z-1,2=14/F-1 In this manner, the windower could even leave out, effectively, the performance of the weighting 58 with respect to the zero-portion 56. Samples u.(E.1),(s-,7/4) 1+1/F
IME*2) Nif.) of current --(E+1)"' frame would, thus, be obtained using E.+1 addends only, while u4e, 1),(E.7/4)-N,F-1 would be obtained using E+2 addends.
Thus, in the manner outlined above, the audio decoder 10 of Fig. 2 reproduces, in a downscaled manner, the audio signal coded into data stream 24. To this end, the audio decoder 10 uses a window function 54 which is itself a downsampled version of a reference synthesis window of length (E+2)-N. As explained with respect to Fig. 6, this downsampled version, i.e. window 54, is obtained by downsampling the reference synthesis window by a factor of F, i.e. the downsampling factor, using a segmental interpolation, namely in segments of length 1/4-N when measured in the not yet downscalecl regime, in segments of length 1/4=N/F in the downsampled regime, in segments of quarters of a frame length of frames 36, measured temporally and expressed independently from the sampling rate. In 4 = (E+2) the interpolation is, thus, performed, thus yielding 4 = (E+2) times 1/4=N/F long segments which, concatenated, represent the downsampled version of the reference synthesis window of length (E+2)-N. See Fig. 6 for illustration. Fig. 6 shows the synthesis window 54 which is unimodal and used by the audio decoder 10 in accordance with a downsampled audio decoding procedure underneath the reference synthesis window 70 which his of length (El-2)-N.
That is, by the downsampling procedure 72 leading from the reference synthesis window 70 to the synthesis window 54 actually used by the audio decoder 10 for downsampled decoding, the number of window coefficients is reduced by a factor of F. In Fig. 6, the nomenclature of Figs. 5 and 6 has been adhered to, i.e. w is used in order to denote the downsampled version window 54, while w' has been used to denote the window coefficients of the reference synthesis window 70.
As just mentioned, in order to perform the downsampling 72, the reference synthesis window 70 is processed in segments 74 of equal length. In number, there are (E+2)=4 such segments 74. Measured in the original sampling rate, i.e. in the number of window coefficients of the reference synthesis window 70, each segment 74 is 1/4 = N
window Date Recue/Date Received 2022-03-01 coefficients w' long, and measured in the reduced or downsampled sampling rate, each segment 74 is 1/4-N/F window coefficients w long.
Naturally, it would be possible to perform the downsampling 72 for each downsarnple.d 5 window coefficient wi coinciding accidentally with any of the window coefficients w; of the reference synthesis window 70 by simply setting w, = w; with the sample time of w1 coinciding with that of wl, and/or by linearly interpolating any window coefficients wi residing, temporally, between two window coefficients w; and w1+2 by linear interpolation, but this procedure would result in a poor approximation of the reference synthesis window 10 70, i.e. the synthesis window 54 used by audio decoder 10 for the downsampled decoding would represent a poor approximation of the reference synthesis window 70, thereby not fulfilling the request for guaranteeing conformance testing of the downscaled decoding relative to the non-downscaled decoding of the audio signal from data stream 24. Thus, the downsampling 72 involves an interpolation procedure according to which the majority 15 of the window coefficients %xi of the downsampled window 54, namely the ones positioned offset from the borders of segments 74, depend by way of the downsampling procedure 72 on more than two window coefficients w' of the reference window 70. In particular, while the majority of the window coefficients wi of the downsampled window 54 depend on more than two window coefficients w; of the reference window 70 in order to increase the 20 quality of the interpolation/downsampling result, i.e. the approximation quality, for every window coefficient wi of the downsampled version 54 it holds true that same does not depend in window coefficients w; belonging to different segments 74. Rather, the downsampling procedure 72 is a segmental interpolation procedure.
For example, the synthesis window 54 may be a concatenation of spline functions of length 1/4 N/F. Cubic spline functions may be used. Such an example has been outlined above in section A.1 where the outer for-next loop sequentially looped over segments 74 wherein, in each segment 74, the downsampling or Interpolation 72 involved a mathematical combination of consecutive window coefficients w' within the current :30 segment 74 at, for example, the first for next clause in the section "calculate vector r needed to calculate the coefficients c". The interpolation applied in segments, may, however, also be chosen differently. That is, the interpolation is not restricted to splines or cubic splines. Rather, linear interpolation or any other interpolation method may be used as well. In any case, the segmental implementation of the interpolation would cause the computation of samples of the downscaled synthesis window, i.e. the outmost samples of Date Recue/Date Received 2022-03-01 the segments of the downscaled synthesis window, neighboring another segment, to not depend on window coefficients of the reference synthesis window residing in different segments.
it may be that windower 18 obtains the downsampled synthesis window 54 from a storage whore the window coefficients wi of this downsampled synthesis window 54 have been stored after having been obtained using the downsampling 72. Alternatively, as illustrated in Fig. 2, the audio decoder 10 may comprise a segmental clownsampler 76 performing the downsampling 72 of Fig. 6 on the basis of the reference synthesis window 70.
It should be noted that the audio decoder 10 of Fig. 2 may be configured to support merely one fixed downsampling factor F or may support different values. In that case, the audio decoder 10 may be responsive to an input value for F as illustrated in Fig. 2 at 78.
The grabber 14, for instance, may be responsive to this value F in order to grab, as mentioned above, the N/F spectral values per frame spectrum. In a like manner, the optional segmental downsampler 76 may also be responsive to this value of F an operate as indicated above. The SIT modulator 16 may be responsive to F either in order to, for example, computationally derive clownscaled/downsampled versions of the modulation functions, downscaled/downsampled relative to the ones used in not-downscaled operation mode where the reconstruction leads to the full audio sample rate.
Naturally, the modulator 16 would also be responsive to F input 78, as modulator 16 would use appropriately downsampled versions of the modulation functions and the same holds true for the windower 18 and canceler 20 with respect to an adaptation of the actual length of the frames in the reduced or downsampled sampling rate.
For example, F may lie between 1.5 and 10, both inclusively.
It should be noted that the decoder of Fig. 2 and 3 or any modification thereof outlined herein, may be implemented so as to perform the spectral-to-time transition using a lifting implementation of the Low Delay MDCT as taught in, for example, EP 2 378 516 81.
Fig. 8 illustrates an implementation of the decoder using the lifting concept.
The SIT
modulator 16 performs exemplarily an inverse DCT-IV and is shown as followed by a block representing the concatenation of the windower 18 and the time domain aliasing canceller 20. In the example of Fig. 8 E is 2, i.e. E=2.
Date Recue/Date Received 2022-03-01 The modulator 16 comprises an inverse type-iv discrete cosine transform frequency/time converter. Instead of outputing sequences of (E+2)N/F long temporal portions 52, it merely outputs temporal portions 52 of length 2=N/F, all derived from the sequence of N/F
long spectra 46, these shortened portions 52 corresponding to the DCT kernel, i.e. the 2-N/F newest samples of the erstwhile described portions.
The windower 18 acts as described previously and generates a windowed temporal portion 60 for each temporal portion 52, but it operates merely on the DCT
kernel. To this end, windower 18 uses window function 0.), with i=0...2N/F-1, having the kernel size. The relationship between wi with 10. ..(E+2)-N/F-1 is described later, just as the relationship between the subsequently mentioned lifting coefficients and wi with i=0 ...(E+2)-N/F-1 is.
Using the nomenclature applied above, the process described so far yields:
2k,r; = Wn = Xk,n for n = 0....,2M-1 with redefining M = N/F, so that M corresponds to the frame size expressed in the downscaled domain and using the nomenclature of Fig. 2-6, wherein, however, zk,n and xk,a shall contain merely the samples of the windowed temporal portion and the not-yet windowed temporal portion within the DOT kernel having size 2.M and temporally corresponding to samples EN/F...(E-1-2)-N/F-1 in Fig. 4. That is, n is an integer indicating a sample index and ren is a real-valued window function coefficient corresponding to the sample index n.
The overlap/add process of the canceller 20 operates in a manner different compared to the above description. It generates intermediate temporal portions mk(0),...mk(M-1) based on the equation or expression mk.n = Zk,n Zk-1,n+h4 for n = 0,...,M-1 In the implementation of Fig. 8, the apparatus further comprises a lifter 80 which may be interpreted as a part of the modulator 16 and windower 18 since the lifter 80 compensates the fact the modulator and the windower restricted their processing to the DCT
kernel instead of processing the extension of the modulation functions and the synthesis window beyond the kernel towards the past which extension was introduced to compensate for the Date Recue/Date Received 2022-03-01 zero portion 56. The lifter 80 produces, using a framework of the delayers and multipliers 82 and adders 84, the finally reconstructed temporal portions or frames of length M in pairs of immediately consecutive frames based on the equation or expression LIN.n=-= mk,n "4' 1n-M/2 Mk -1 .01-1 for n = M/2,... ,M-1 , and uk,õ= Mic,n 1m-1-n = OUltic-I.M-1-n for n=0,...,M/2-1 wherein la with n = 0...M-1 arc real-valued lifting coefficients related to the downscaled synthesis window in a manner described in more detail below.
In other words, for the extended overlap of E frames into the past, only M
additional multiplier-add operations are required, as can be seen in the framework of the lifter 80.
These additional operations are sometimes also referred to as "zero-delay matrices".
Sometimes these operations are also known as "lifting steps'. The efficient implementation shown in Fig. 8 may under some circumstances be more efficient as a straightforward implementation. To be more precise, depending on the concrete .. implementation, such a more efficient implementation might result in saving M operations, as in the case of a straightforward implementation for M operations, it might be advisable to implement, as the implementation shown in Fig. 19, requires in principle, 2M operations in the framework of the module 820 and M operations in the framework of the lifter 830.
As to the dependency of con with n=0...2M-1 and In with n = 0...M-1 on the synthesis window vvi with i = 0...(E+2)M-1 (it is recalled that here E=2), the following formulae describe the relationship between them with displacing, however, the subscript indices used so far into the parenthesis following the respective variable:
w(i) = (-2 ¨ 1¨ n) = l(M ¨1 ¨n) = w(M + n) w(M/2 + = 1(n) =1(M/2 + n) = ba(3M/2 + n) w(M + 0 / (¨ ¨ 1¨ n) = 03(it4 + n) w(3M/2 + = ¨1(n) - (0(3 M / 2 + n) w (2M + i) = --(.1)(M + n) 1(M ¨ 1 n) = w(n) w(5M/2 + = ¨(1)(3M/2 + n) 1(M/2 + n) = w(M/2 + n) w(3/i/ 4- = ¨1.&)(n) Date Recue/Date Received 2022-03-01 til(7M/ 2 + i) = w(M 4 n) for i, n = 0 ¨2 ¨ I
Please note that the window we contains the peak values on the right side in this formulation, i.e. between the indices 2M and 4M 1. The above formulae relate coefficients in with n = 0...M-1 and co. n = 0,...,2M-1 to the coefficients wn with n =
0... (E+2)M-1 of the downscaled synthesis window. As can be seen, in with n =
0...M-1 actually merely depend on % of the coefficients of the downsam pled synthesis window, namely on wõ with n = 0...(E+1)M-1, while m, n = 0....,2M-1 depend on all wn with n =
0...(F+2)M-1.
As stated above, it might be that windower 18 obtains the downsampted synthesis window 54 wn with n = 0...(E+2)M-1 from a storage where the window coefficients wi of this downsampled synthesis window 54 have been stored after having been obtained using the downsampling 72, and from where same are read to compute coefficients in with n =
0...M-1 and con n = 0,...,2M-1 using the above relation, but alternatively, winder 18 may retrieve the coefficients In with n = 0...M-1 and con n = 0,...,2M-1, thus computed from the pre-downsampled synthesis window, from the storage directly. Alternatively, as stated above, the audio decoder 10 may comprise the segmental downsampler 76 performing the downsampling 72 of Fig. 6 on the basis of the reference synthesis window 70, thereby yielding wn with n = 0...(E+2)M-1 on the basis of which the windower 18 computes coefficients In with n = 0...M-1 and o.)n n = 0,... ,2M-1 using above relation/formulae. Even using the lifting implementation, more than one value for F may be supported.
Briefly summarizing the lifting implementation, same results in an audio decoder 10 configured to decode an audio signal 22 at a first sampling rate from a data stream 24 into which the audio signal is transform coded at a second sampling rate, the first sampling rate being 1/Fth of the second sampling rate, the audio decoder 10 comprising the receiver
12 which receives, per frame of length N of the audio signal, N spectral coefficients 28, the grabber 14 which grabs-out for each frame, a low-frequency fraction of length N/F out of the N spectral coefficients 28, a spectral-to-time modulator 16 configured to subject, for each frame 36, the low-frequency fraction to an inverse transform having modulation functions c.4 length 2. N/F temporally extending over the respective frame and a previous frame so as to obtain a temporal portion of length 2-N/F, and a windower 18 which windows, for each frame 30, the temporal portion xn according to zecr, = con =
xki, for n =
Date Recue/Date Received 2022-03-01 O,... ,2M-1 so as to obtain a windowed temporal portion zk.0 with with n =
0...2M-1. The time domain aliasing canceler 20 generates intermediate temporal portions rnk(0),...mk(M-1) according to mk.n = 7km + zk-1.n.k4 for n = 0,...,M-1. Finally, the lifter 80 computes frames uk,n of the audio signal with n = 0...M-1 according to Ukm = Mkm in=M/2 = Mic-1,M-1-n for 5 n M12,... ,M-1, and uk,n mk,õ outka.k4-1.n for n=0,... ,M/2-1, wherein In with n =
0._ M-1 are lifting coefficients, wherein the inverse transform is an inverse MDCT or inverse MDST, and wherein in with n 0...M-1 and 6.)n n 0,...,2M-1 depend on coefficients wn with n = O... (E+2)M-1 of a synthesis window, and the synthesis Window is a downsampled version of a reference synthesis window of length 4 = N, downsampled by a 10 factor of F by a segmental interpolation in segments of length 1/4 = N.
It already turned out from the above discussion of a proposal for an extension of AAC-ELD with respect to a downscaled decoding mode that the audio decoder of Fig.
2 may be accompanied with a low delay SBR tool. The following outlines, for instance, how the 15 AAC-ELD coder extended to support the above-proposed downscaled operating mode, would operate when using the low delay SBR tool As already mentioned in the introductory portion of the specification of the present application, in case the low delay SBR tool is used in connection with the AAC-ELD coder, the filter banks of the low delay SBR module are downscaled as well. This ensures that the SBR module operates with the 20 same frequency resolution and therefore no more adaptations are required. Fig. 7 outlines the signal path of the AAC-ELD decoder operating at 96 kHz, with frame size of samples, in down-sampled SBR mode and with a downscaling factor F of 2.
In Fig. 7, the bitstream arriving as processed by a sequence of blocks, namely an AAC
25 decoder, an inverse LD-MDCT block, a CLDFB analysis block, an SBR decoder and a CLDFB synthesis block (CLDFB = complex low delay filter bank). The bitstream equals the data stream 24 discussed previously with respect to Figs. 3 to 6, but is additionally accompanied by parametric SBR data assisting the spectral shaping of a spectral replicate of a spectral extension band extending the spectra frequency of the audio signal .. obtained by the downscaled audio decoding at the output of the inverse low delay (VOGT
block, the spectral shaping being performed by the SBR decoder. In particular, the AAC
decoder retrieves all of the necessary syntax elements by appropriate parsing and entropy decoding. The AAC decoder may partially coincide with the receiver 12 of the audio decoder 10 which, in Fig, 7, is embodied by the inverse low delay MDCT block.
In Fig. 7, F is exemplarily equal to 2. That is, the inverse low delay MDCT block of Fig.
7 outputs, as an example for the reconstructed audio signal 22 of Fig. 2, a 48 kHz time signal Date Recue/Date Received 2022-03-01 downsampled at half the rate at which the audio signal was originally coded into the arriving bitstroam. The CLDFB analysis block subdivides this 48 kHz time signal, i.e. the audio signal obtained by downscaled audio decoding, into N bands, here N = 16, and the SBR decoder computes re-shaping coefficients for those bands, re-shapes the N
bands accordingly ¨ controlled via the SBR data in the input bitstream arriving at the input of the AAC decoder, and the CLDFB synthesis block re-transitions from spectral domain to time domain with obtaining, thereby, a high frequency extension signal to be added to the original decoded audio signals output by the inverse low delay MDCT block.
Please note, that the standard operation of SBR utilizes a 32 band CLDFB. The interpolation algorithm for the 32 band CLDFB window coefficients ci32 is already given in 4.6.19.4.1 in En ci32(i) = ¨2tc64(2i + + c64(21)). 0 < I < 320, where c" are the window coefficients of the 64 band window given in Table 4.A.90 in 11).
This formula can be further generalized to define window coefficients for a lower number of bands B as well cip(i) .----- ¨2 it:64(2F = i + 1 + p) + c64.(21i = + p)1, OS. < (10B), p =
int (¨ ¨ 0.5) ZB
where F denotes the downscaling factor being F = 32/B. With this definition of the window coefficients, the CLDFB analysis and synthesis filter bank can be completely described as outlined in the above example of section A.2.
Thus, above examples provided some missing definitions for the AAC-ELD codec in order to adapt the codec to systems with lower sample rates. These definitions may be included in the ISO/IEC 14496-3:2009 standard.
Thus, in the above discussion it has, inter alias, been described:
An audio decoder may be configured to decode an audio signal at a first sampling rate from a data stream into which the audio signal is transform coded at a second sampling rate, the first sampling rate being 1/F6' of the second sampling rate, the audio decoder comprising: a receiver configured to receive, per frame of length N of the audio signal, N
Date Recue/Date Received 2022-03-01 spectral coefficients; a grabber configured to grab-out for each frame, a low-frequency fraction of length N/F out of the N spectral coefficients; a spectral-to-time modulator configured to subject, for each frame, the low-frequency fraction to an inverse transform having modulation functions of length (E
2) = N/F temporally extending over the respective frame and E+1 previous frames so as to obtain a temporal portion of length (E
+ 2) = N/F; a windower configured to window, for each frame, the lemporal portion using a unimodal synthesis window of length (E + 2) = N/F comprising a zero-portion of length 1/4 =
N/F at a leading end thereof and having a peak within a temporal interval of the unimodal synthesis window, the temporal interval succeeding the zero-portion and having length 7/4 = N/F so that the windower obtains a windowed temporal portion of length (E +
2) = N/F:
and a time domain abasing canceler configured to subject the windowed temporal portion of the frames to an overlap-add process so that a trailing-end fraction of length (E + 1)/(E
+ 2) of the windowed temporal portion of a current frame overlaps a leading end of length (E + 1)/(E + 2) of the windowed temporal portion of a preceding frame, wherein the inverse transform is an inverse MDCT or inverse MDST, and wherein the unimodal synthesis window is a downsampled version of a reference unimodal synthesis window of length (E + 2) = N, downsampled by a factor of F by a segmental interpolation in segments of length 114 = N/F.
Audio decoder according to an embodiment, wherein the unirnodal synthesis window is a concatenation of spline functions of length 1/4 = N/F.
Audio decoder according to an embodiment, wherein the unimodal synthesis window is a concatenation of cubic spline functions of length 114- N/F.
Audio decoder according to any of the previous embodiments, wherein E =, 2.
Audio decoder according to any of the previous embodiments, wherein the inverse transform is an inverse MDCT.
Audio decoder according to any of the previous embodiments, wherein more than 80% of a mass of the unimodal synthesis window is comprised within the temporal interval succeeding the zero-portion and having length 7/4 = N/F.
Date Recue/Date Received 2022-03-01 Audio decoder according to any of the previous embodiments, wherein the audio decoder is configured to perform the interpolation or to derive the unimodat synthesis window from a storage.
Audio decoder according to any of the previous embodiments, wherein the audio decoder is configured to support different values for F.
Audio decoder according to any of the previous embodiments, wherein F is between 1.5 and 10, both inclusively.
A method performed by an audio decoder according to any of the previous embodiments.
A computer program having a program code for performing, when running on a computer, a method according to an embodiment.
As far as the term "of ...length" is concerned it should be noted that this term is to be interpreted as measuring the length in samples. As far as the length of the zero portion and the segments is concerned it should be noted that same may be integer valued.
Alternatively, same may be non-integer valued.
As to the temporal interval within which the peak is positioned it is noted that Fig. 1 shows this peak as well as the temporal interval illustratively for an example of the reference unimodal synthesis window with E -,- 2 and N 512; The peak has its maximum at approximately sample No. 1408 and the temporal interval extends from sample No. 1024 to sample No. 1920. The temporal interval is, thus, 7/8 of the DCT kernel long.
As to the term "downsampled version" it is noted that in the above specification, instead of this term, "downscaled version" has synonymously been used.
As to the term "mass of a function within a certain interval" it is noted that same shall = denote the definite integral of the respective function within the respective interval.
In case of the audio decoder supporting different values for F, same may comprise a storage having accordingly segmentally interpolated versions of the reference unimodal synthesis window or may perform the segmental interpolation for a currently active value of F. The different segmentally interpolated versions have in common that the Date Recue/Date Received 2022-03-01 interpolation does not negatively affect the discontinuities at the segment boundaries.
They may, as described above, spline functions.
By deriving the unimodal synthesis window by a segmental interpolation from the reference unimodal synthesis window such as the one shown in Fig. 1 above, the 4 - (E +
2) segments may be formed by spline approximation such as by cubic splines and despite the interpolation, the discontinuities which are to be present in the unimodal synthesis window at a pitch of 1/4 = N/F owing to the synthetically introduced zero-portion as a means for lowering the delay are conserved.
References [1] ISO/IEC 14496-3:2009 [2] M13958, "Proposal for an Enhanced Low Delay Coding Mode", October 2006, Hangzhou, China Date Recue/Date Received 2022-03-01
xki, for n =
Date Recue/Date Received 2022-03-01 O,... ,2M-1 so as to obtain a windowed temporal portion zk.0 with with n =
0...2M-1. The time domain aliasing canceler 20 generates intermediate temporal portions rnk(0),...mk(M-1) according to mk.n = 7km + zk-1.n.k4 for n = 0,...,M-1. Finally, the lifter 80 computes frames uk,n of the audio signal with n = 0...M-1 according to Ukm = Mkm in=M/2 = Mic-1,M-1-n for 5 n M12,... ,M-1, and uk,n mk,õ outka.k4-1.n for n=0,... ,M/2-1, wherein In with n =
0._ M-1 are lifting coefficients, wherein the inverse transform is an inverse MDCT or inverse MDST, and wherein in with n 0...M-1 and 6.)n n 0,...,2M-1 depend on coefficients wn with n = O... (E+2)M-1 of a synthesis window, and the synthesis Window is a downsampled version of a reference synthesis window of length 4 = N, downsampled by a 10 factor of F by a segmental interpolation in segments of length 1/4 = N.
It already turned out from the above discussion of a proposal for an extension of AAC-ELD with respect to a downscaled decoding mode that the audio decoder of Fig.
2 may be accompanied with a low delay SBR tool. The following outlines, for instance, how the 15 AAC-ELD coder extended to support the above-proposed downscaled operating mode, would operate when using the low delay SBR tool As already mentioned in the introductory portion of the specification of the present application, in case the low delay SBR tool is used in connection with the AAC-ELD coder, the filter banks of the low delay SBR module are downscaled as well. This ensures that the SBR module operates with the 20 same frequency resolution and therefore no more adaptations are required. Fig. 7 outlines the signal path of the AAC-ELD decoder operating at 96 kHz, with frame size of samples, in down-sampled SBR mode and with a downscaling factor F of 2.
In Fig. 7, the bitstream arriving as processed by a sequence of blocks, namely an AAC
25 decoder, an inverse LD-MDCT block, a CLDFB analysis block, an SBR decoder and a CLDFB synthesis block (CLDFB = complex low delay filter bank). The bitstream equals the data stream 24 discussed previously with respect to Figs. 3 to 6, but is additionally accompanied by parametric SBR data assisting the spectral shaping of a spectral replicate of a spectral extension band extending the spectra frequency of the audio signal .. obtained by the downscaled audio decoding at the output of the inverse low delay (VOGT
block, the spectral shaping being performed by the SBR decoder. In particular, the AAC
decoder retrieves all of the necessary syntax elements by appropriate parsing and entropy decoding. The AAC decoder may partially coincide with the receiver 12 of the audio decoder 10 which, in Fig, 7, is embodied by the inverse low delay MDCT block.
In Fig. 7, F is exemplarily equal to 2. That is, the inverse low delay MDCT block of Fig.
7 outputs, as an example for the reconstructed audio signal 22 of Fig. 2, a 48 kHz time signal Date Recue/Date Received 2022-03-01 downsampled at half the rate at which the audio signal was originally coded into the arriving bitstroam. The CLDFB analysis block subdivides this 48 kHz time signal, i.e. the audio signal obtained by downscaled audio decoding, into N bands, here N = 16, and the SBR decoder computes re-shaping coefficients for those bands, re-shapes the N
bands accordingly ¨ controlled via the SBR data in the input bitstream arriving at the input of the AAC decoder, and the CLDFB synthesis block re-transitions from spectral domain to time domain with obtaining, thereby, a high frequency extension signal to be added to the original decoded audio signals output by the inverse low delay MDCT block.
Please note, that the standard operation of SBR utilizes a 32 band CLDFB. The interpolation algorithm for the 32 band CLDFB window coefficients ci32 is already given in 4.6.19.4.1 in En ci32(i) = ¨2tc64(2i + + c64(21)). 0 < I < 320, where c" are the window coefficients of the 64 band window given in Table 4.A.90 in 11).
This formula can be further generalized to define window coefficients for a lower number of bands B as well cip(i) .----- ¨2 it:64(2F = i + 1 + p) + c64.(21i = + p)1, OS. < (10B), p =
int (¨ ¨ 0.5) ZB
where F denotes the downscaling factor being F = 32/B. With this definition of the window coefficients, the CLDFB analysis and synthesis filter bank can be completely described as outlined in the above example of section A.2.
Thus, above examples provided some missing definitions for the AAC-ELD codec in order to adapt the codec to systems with lower sample rates. These definitions may be included in the ISO/IEC 14496-3:2009 standard.
Thus, in the above discussion it has, inter alias, been described:
An audio decoder may be configured to decode an audio signal at a first sampling rate from a data stream into which the audio signal is transform coded at a second sampling rate, the first sampling rate being 1/F6' of the second sampling rate, the audio decoder comprising: a receiver configured to receive, per frame of length N of the audio signal, N
Date Recue/Date Received 2022-03-01 spectral coefficients; a grabber configured to grab-out for each frame, a low-frequency fraction of length N/F out of the N spectral coefficients; a spectral-to-time modulator configured to subject, for each frame, the low-frequency fraction to an inverse transform having modulation functions of length (E
2) = N/F temporally extending over the respective frame and E+1 previous frames so as to obtain a temporal portion of length (E
+ 2) = N/F; a windower configured to window, for each frame, the lemporal portion using a unimodal synthesis window of length (E + 2) = N/F comprising a zero-portion of length 1/4 =
N/F at a leading end thereof and having a peak within a temporal interval of the unimodal synthesis window, the temporal interval succeeding the zero-portion and having length 7/4 = N/F so that the windower obtains a windowed temporal portion of length (E +
2) = N/F:
and a time domain abasing canceler configured to subject the windowed temporal portion of the frames to an overlap-add process so that a trailing-end fraction of length (E + 1)/(E
+ 2) of the windowed temporal portion of a current frame overlaps a leading end of length (E + 1)/(E + 2) of the windowed temporal portion of a preceding frame, wherein the inverse transform is an inverse MDCT or inverse MDST, and wherein the unimodal synthesis window is a downsampled version of a reference unimodal synthesis window of length (E + 2) = N, downsampled by a factor of F by a segmental interpolation in segments of length 114 = N/F.
Audio decoder according to an embodiment, wherein the unirnodal synthesis window is a concatenation of spline functions of length 1/4 = N/F.
Audio decoder according to an embodiment, wherein the unimodal synthesis window is a concatenation of cubic spline functions of length 114- N/F.
Audio decoder according to any of the previous embodiments, wherein E =, 2.
Audio decoder according to any of the previous embodiments, wherein the inverse transform is an inverse MDCT.
Audio decoder according to any of the previous embodiments, wherein more than 80% of a mass of the unimodal synthesis window is comprised within the temporal interval succeeding the zero-portion and having length 7/4 = N/F.
Date Recue/Date Received 2022-03-01 Audio decoder according to any of the previous embodiments, wherein the audio decoder is configured to perform the interpolation or to derive the unimodat synthesis window from a storage.
Audio decoder according to any of the previous embodiments, wherein the audio decoder is configured to support different values for F.
Audio decoder according to any of the previous embodiments, wherein F is between 1.5 and 10, both inclusively.
A method performed by an audio decoder according to any of the previous embodiments.
A computer program having a program code for performing, when running on a computer, a method according to an embodiment.
As far as the term "of ...length" is concerned it should be noted that this term is to be interpreted as measuring the length in samples. As far as the length of the zero portion and the segments is concerned it should be noted that same may be integer valued.
Alternatively, same may be non-integer valued.
As to the temporal interval within which the peak is positioned it is noted that Fig. 1 shows this peak as well as the temporal interval illustratively for an example of the reference unimodal synthesis window with E -,- 2 and N 512; The peak has its maximum at approximately sample No. 1408 and the temporal interval extends from sample No. 1024 to sample No. 1920. The temporal interval is, thus, 7/8 of the DCT kernel long.
As to the term "downsampled version" it is noted that in the above specification, instead of this term, "downscaled version" has synonymously been used.
As to the term "mass of a function within a certain interval" it is noted that same shall = denote the definite integral of the respective function within the respective interval.
In case of the audio decoder supporting different values for F, same may comprise a storage having accordingly segmentally interpolated versions of the reference unimodal synthesis window or may perform the segmental interpolation for a currently active value of F. The different segmentally interpolated versions have in common that the Date Recue/Date Received 2022-03-01 interpolation does not negatively affect the discontinuities at the segment boundaries.
They may, as described above, spline functions.
By deriving the unimodal synthesis window by a segmental interpolation from the reference unimodal synthesis window such as the one shown in Fig. 1 above, the 4 - (E +
2) segments may be formed by spline approximation such as by cubic splines and despite the interpolation, the discontinuities which are to be present in the unimodal synthesis window at a pitch of 1/4 = N/F owing to the synthetically introduced zero-portion as a means for lowering the delay are conserved.
References [1] ISO/IEC 14496-3:2009 [2] M13958, "Proposal for an Enhanced Low Delay Coding Mode", October 2006, Hangzhou, China Date Recue/Date Received 2022-03-01
Claims (3)
1. Audio decoder configured to decode an audio signal at a first sampling rate from a data stream into which the audio signal is transform coded at a second sampling rate, the first sampling rate being 1/Fth of the second sampling rate, with F
being a downscaling factor, the audio decoder comprising:
a receiver configured to receive, per frame of length N of the audio signal. N
spectral coefficients;
a grabber configured to grab-out for each frame, a low-frequency fraction of length N/F out of the N spectral coefficients;
a spectral-to-time modulator configured to subject, for each frame, the low-frequency fraction to an inverse transform having modulation functions of length 4 N/F
temporally extending over the respective frame and three previous frames so as to obtain a temporal portion of length 4 = N/F;
a windower configured to window, for each frame, the temporal portion using a synthesis window of length 4 N/F comprising a zero-portion of length 1/4=NIF
at a leading end thereof and having a peak within a temporal interval of the synthesis window, the temporal interval succeeding the zero-portion and having length 7/4 =
N/F so that the windower obtains a windowed temporal portion of length 4 =
N/F; and a time domain aliasing canceler configured to subject the windowed temporal portion of the frames to an overlap-add process so that a trailing-end fraction of length 3/4 of the windowed temporal portion of a current frame overlaps a leading end of length 3/4 of the windowed temporal portion of a preceding frame, wherein the inverse transforrn is an inverse MDCT (Modified Discrete Cosine Transform) or inverse MDST (Modified Discrete Sine Transform), and wherein the synthesis window is a downsampled version of a reference synthesis window of length 4 = N, clownsampled by a factor of F by a segmental interpolation in segments of length 1/4 N.
Date Recue/Date Received 2022-03-01
being a downscaling factor, the audio decoder comprising:
a receiver configured to receive, per frame of length N of the audio signal. N
spectral coefficients;
a grabber configured to grab-out for each frame, a low-frequency fraction of length N/F out of the N spectral coefficients;
a spectral-to-time modulator configured to subject, for each frame, the low-frequency fraction to an inverse transform having modulation functions of length 4 N/F
temporally extending over the respective frame and three previous frames so as to obtain a temporal portion of length 4 = N/F;
a windower configured to window, for each frame, the temporal portion using a synthesis window of length 4 N/F comprising a zero-portion of length 1/4=NIF
at a leading end thereof and having a peak within a temporal interval of the synthesis window, the temporal interval succeeding the zero-portion and having length 7/4 =
N/F so that the windower obtains a windowed temporal portion of length 4 =
N/F; and a time domain aliasing canceler configured to subject the windowed temporal portion of the frames to an overlap-add process so that a trailing-end fraction of length 3/4 of the windowed temporal portion of a current frame overlaps a leading end of length 3/4 of the windowed temporal portion of a preceding frame, wherein the inverse transforrn is an inverse MDCT (Modified Discrete Cosine Transform) or inverse MDST (Modified Discrete Sine Transform), and wherein the synthesis window is a downsampled version of a reference synthesis window of length 4 = N, clownsampled by a factor of F by a segmental interpolation in segments of length 1/4 N.
Date Recue/Date Received 2022-03-01
2. Method for decoding an audio signal at a first sampling rate from a data stream into which the audio signal is transform coded at a second sampling rate, the first sampling rate being 1/Fth of the second sampling rateõ with F being a downscaling factor, the rnethod comprising:
receiving, per frame of length N of the audio signal, N spectral coefficients;
grabbing-out for each frame, a low-frequency fraction of length N/F out of the N
spectral coefficients;
performing a spectral-to-time modulation by subjecting, for each frame, the low-frequency fraction to an inverse transform having modulation functions of length 4 -N/F temporally extending over the respective frame and three previous frames so as to obtain a temporal portion of length 4 = N/F;
windowing, for each frame, the temporal portion using a synthesis window of length 4 = N/F comprising a zero-portion of length 1/4=NIF at a leading end thereof and having a peak within a temporal interval of the synthesis window, the temporal interval succeeding the zero-portion and having length 7/4 = N/F so that the windower obtains a windowed temporal portion of length 4 = N/F; and performing a time domain aliasing cancellation by subjecting the windowed temporal portion of the frames to an overlap-add process so that a trailing-end fraction of length 3/4 of the windowed temporal portion of a current frame overlaps a leading end of length 3/4 of the windowed temporal portion of a preceding frame, wherein the inverse transform is an inverse MDCT or inverse MDST, and wherein the synthesis window is a downsampled version of a reference synthesis window of length 4 = N, downsampled by a factor of F by a segmental interpolation in segments of length 1/4 = N.
receiving, per frame of length N of the audio signal, N spectral coefficients;
grabbing-out for each frame, a low-frequency fraction of length N/F out of the N
spectral coefficients;
performing a spectral-to-time modulation by subjecting, for each frame, the low-frequency fraction to an inverse transform having modulation functions of length 4 -N/F temporally extending over the respective frame and three previous frames so as to obtain a temporal portion of length 4 = N/F;
windowing, for each frame, the temporal portion using a synthesis window of length 4 = N/F comprising a zero-portion of length 1/4=NIF at a leading end thereof and having a peak within a temporal interval of the synthesis window, the temporal interval succeeding the zero-portion and having length 7/4 = N/F so that the windower obtains a windowed temporal portion of length 4 = N/F; and performing a time domain aliasing cancellation by subjecting the windowed temporal portion of the frames to an overlap-add process so that a trailing-end fraction of length 3/4 of the windowed temporal portion of a current frame overlaps a leading end of length 3/4 of the windowed temporal portion of a preceding frame, wherein the inverse transform is an inverse MDCT or inverse MDST, and wherein the synthesis window is a downsampled version of a reference synthesis window of length 4 = N, downsampled by a factor of F by a segmental interpolation in segments of length 1/4 = N.
3. A computer-readable medium having computer-readable code stored thereon for performing the method according to claim 2, when the computer-readable code is run by a computer.
Date Recue/Date Received 2022-03-01
Date Recue/Date Received 2022-03-01
Applications Claiming Priority (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP15172282.4 | 2015-06-16 | ||
| EP15172282 | 2015-06-16 | ||
| EP15189398.9 | 2015-10-12 | ||
| EP15189398.9A EP3107096A1 (en) | 2015-06-16 | 2015-10-12 | Downscaled decoding |
| CA2989252A CA2989252C (en) | 2015-06-16 | 2016-06-10 | Downscaled decoding |
Related Parent Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA2989252A Division CA2989252C (en) | 2015-06-16 | 2016-06-10 | Downscaled decoding |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA3150683A1 CA3150683A1 (en) | 2016-12-22 |
| CA3150683C true CA3150683C (en) | 2023-10-31 |
Family
ID=53483698
Family Applications (6)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA3150643A Pending CA3150643A1 (en) | 2015-06-16 | 2016-06-10 | Downscaled decoding |
| CA3150683A Active CA3150683C (en) | 2015-06-16 | 2016-06-10 | Downscaled decoding |
| CA3150666A Active CA3150666C (en) | 2015-06-16 | 2016-06-10 | Downscaled decoding |
| CA2989252A Active CA2989252C (en) | 2015-06-16 | 2016-06-10 | Downscaled decoding |
| CA3150675A Active CA3150675C (en) | 2015-06-16 | 2016-06-10 | Downscaled decoding |
| CA3150637A Active CA3150637C (en) | 2015-06-16 | 2016-06-10 | Downscaled decoding |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA3150643A Pending CA3150643A1 (en) | 2015-06-16 | 2016-06-10 | Downscaled decoding |
Family Applications After (4)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA3150666A Active CA3150666C (en) | 2015-06-16 | 2016-06-10 | Downscaled decoding |
| CA2989252A Active CA2989252C (en) | 2015-06-16 | 2016-06-10 | Downscaled decoding |
| CA3150675A Active CA3150675C (en) | 2015-06-16 | 2016-06-10 | Downscaled decoding |
| CA3150637A Active CA3150637C (en) | 2015-06-16 | 2016-06-10 | Downscaled decoding |
Country Status (20)
| Country | Link |
|---|---|
| US (10) | US10431230B2 (en) |
| EP (8) | EP3107096A1 (en) |
| JP (10) | JP6637079B2 (en) |
| KR (10) | KR102131183B1 (en) |
| CN (6) | CN114255772A (en) |
| AR (5) | AR105006A1 (en) |
| AU (1) | AU2016278717B2 (en) |
| BR (1) | BR112017026724B1 (en) |
| CA (6) | CA3150643A1 (en) |
| ES (1) | ES2950408T3 (en) |
| FI (1) | FI3311380T3 (en) |
| HK (1) | HK1247730A1 (en) |
| MX (1) | MX2017016171A (en) |
| MY (1) | MY178530A (en) |
| PL (1) | PL3311380T3 (en) |
| PT (1) | PT3311380T (en) |
| RU (1) | RU2683487C1 (en) |
| TW (1) | TWI611398B (en) |
| WO (1) | WO2016202701A1 (en) |
| ZA (1) | ZA201800147B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017129270A1 (en) * | 2016-01-29 | 2017-08-03 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and method for improving a transition from a concealed audio signal portion to a succeeding audio signal portion of an audio signal |
Family Cites Families (39)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5729556A (en) * | 1993-02-22 | 1998-03-17 | Texas Instruments | System decoder circuit with temporary bit storage and method of operation |
| US6092041A (en) * | 1996-08-22 | 2000-07-18 | Motorola, Inc. | System and method of encoding and decoding a layered bitstream by re-applying psychoacoustic analysis in the decoder |
| KR100335611B1 (en) | 1997-11-20 | 2002-10-09 | 삼성전자 주식회사 | Scalable stereo audio encoding/decoding method and apparatus |
| US6351730B2 (en) * | 1998-03-30 | 2002-02-26 | Lucent Technologies Inc. | Low-complexity, low-delay, scalable and embedded speech and audio coding with adaptive frame loss concealment |
| EP0957580B1 (en) * | 1998-05-15 | 2008-04-02 | Thomson | Method and apparatus for sampling-rate conversion of audio signals |
| KR20050021484A (en) * | 2002-07-16 | 2005-03-07 | 코닌클리케 필립스 일렉트로닉스 엔.브이. | Audio coding |
| KR100602975B1 (en) * | 2002-07-19 | 2006-07-20 | 닛본 덴끼 가부시끼가이샤 | Audio decoding apparatus and decoding method and computer-readable recording medium |
| FR2852172A1 (en) * | 2003-03-04 | 2004-09-10 | France Telecom | Audio signal coding method, involves coding one part of audio signal frequency spectrum with core coder and another part with extension coder, where part of spectrum is coded with both core coder and extension coder |
| US20050047793A1 (en) * | 2003-08-28 | 2005-03-03 | David Butler | Scheme for reducing low frequency components in an optical transmission network |
| CN1677492A (en) * | 2004-04-01 | 2005-10-05 | 北京宫羽数字技术有限责任公司 | Intensified audio-frequency coding-decoding device and method |
| JP4626261B2 (en) * | 2004-10-21 | 2011-02-02 | カシオ計算機株式会社 | Speech coding apparatus and speech coding method |
| US7720677B2 (en) * | 2005-11-03 | 2010-05-18 | Coding Technologies Ab | Time warped modified transform coding of audio signals |
| US8036903B2 (en) | 2006-10-18 | 2011-10-11 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. | Analysis filterbank, synthesis filterbank, encoder, de-coder, mixer and conferencing system |
| PT2109098T (en) | 2006-10-25 | 2020-12-18 | Fraunhofer Ges Forschung | Apparatus and method for generating audio subband values and apparatus and method for generating time-domain audio samples |
| WO2008056775A1 (en) * | 2006-11-10 | 2008-05-15 | Panasonic Corporation | Parameter decoding device, parameter encoding device, and parameter decoding method |
| EP2077551B1 (en) | 2008-01-04 | 2011-03-02 | Dolby Sweden AB | Audio encoder and decoder |
| ES2683077T3 (en) * | 2008-07-11 | 2018-09-24 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Audio encoder and decoder for encoding and decoding frames of a sampled audio signal |
| MX2011000375A (en) | 2008-07-11 | 2011-05-19 | Fraunhofer Ges Forschung | Audio encoder and decoder for encoding and decoding frames of sampled audio signal. |
| KR101381513B1 (en) | 2008-07-14 | 2014-04-07 | 광운대학교 산학협력단 | Apparatus for encoding and decoding of integrated voice and music |
| WO2010098112A1 (en) | 2009-02-26 | 2010-09-02 | パナソニック株式会社 | Encoder, decoder, and method therefor |
| TWI556227B (en) * | 2009-05-27 | 2016-11-01 | 杜比國際公司 | Systems and methods for generating a high frequency component of a signal from a low frequency component of the signal, a set-top box, a computer program product and storage medium thereof |
| AU2010305383B2 (en) | 2009-10-08 | 2013-10-03 | Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. | Multi-mode audio signal decoder, multi-mode audio signal encoder, methods and computer program using a linear-prediction-coding based noise shaping |
| PL2473995T3 (en) | 2009-10-20 | 2015-06-30 | Fraunhofer Ges Forschung | Audio signal encoder, audio signal decoder, method for providing an encoded representation of an audio content, method for providing a decoded representation of an audio content and computer program for use in low delay applications |
| CN102884574B (en) | 2009-10-20 | 2015-10-14 | 弗兰霍菲尔运输应用研究公司 | Audio signal encoder, audio signal decoder, use aliasing offset the method by audio-frequency signal coding or decoding |
| TW201214415A (en) * | 2010-05-28 | 2012-04-01 | Fraunhofer Ges Forschung | Low-delay unified speech and audio codec |
| MY156027A (en) * | 2010-08-12 | 2015-12-31 | Fraunhofer Ges Forschung | Resampling output signals of qmf based audio codecs |
| CN103282958B (en) * | 2010-10-15 | 2016-03-30 | 华为技术有限公司 | Signal analyzer, signal analysis method, signal synthesizer, signal synthesis method, transducer and inverted converter |
| CN102419978B (en) * | 2011-08-23 | 2013-03-27 | 展讯通信（上海）有限公司 | Audio decoder and frequency spectrum reconstructing method and device for audio decoding |
| ES2575693T3 (en) * | 2011-11-10 | 2016-06-30 | Nokia Technologies Oy | A method and apparatus for detecting audio sampling rate |
| WO2013142650A1 (en) * | 2012-03-23 | 2013-09-26 | Dolby International Ab | Enabling sampling rate diversity in a voice communication system |
| CN104885149B (en) * | 2012-09-24 | 2017-11-17 | 三星电子株式会社 | Method and apparatus for the method and apparatus of concealment frames mistake and for being decoded to audio |
| EP2720222A1 (en) * | 2012-10-10 | 2014-04-16 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and method for efficient synthesis of sinusoids and sweeps by employing spectral patterns |
| EP2959481B1 (en) * | 2013-02-20 | 2017-04-26 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and method for generating an encoded audio or image signal or for decoding an encoded audio or image signal in the presence of transients using a multi overlap portion |
| CN104078048B (en) * | 2013-03-29 | 2017-05-03 | 北京天籁传音数字技术有限公司 | Acoustic decoding device and method thereof |
| TWI557727B (en) * | 2013-04-05 | 2016-11-11 | 杜比國際公司 | An audio processing system, a multimedia processing system, a method of processing an audio bitstream and a computer program product |
| EP2981956B1 (en) * | 2013-04-05 | 2022-11-30 | Dolby International AB | Audio processing system |
| CN103632674B (en) * | 2013-12-17 | 2017-01-04 | 魅族科技（中国）有限公司 | A kind of processing method and processing device of audio signal |
| EP2980795A1 (en) | 2014-07-28 | 2016-02-03 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Audio encoding and decoding using a frequency domain processor, a time domain processor and a cross processor for initialization of the time domain processor |
| CN107112024B (en) | 2014-10-24 | 2020-07-14 | 杜比国际公司 | Encoding and decoding of audio signals |
-
2015
- 2015-10-12 EP EP15189398.9A patent/EP3107096A1/en not_active Withdrawn
-
2016
- 2016-06-03 TW TW105117582A patent/TWI611398B/en active
- 2016-06-10 CA CA3150643A patent/CA3150643A1/en active Pending
- 2016-06-10 JP JP2017565693A patent/JP6637079B2/en active Active
- 2016-06-10 PT PT167307776T patent/PT3311380T/en unknown
- 2016-06-10 KR KR1020177036140A patent/KR102131183B1/en active IP Right Grant
- 2016-06-10 FI FIEP16730777.6T patent/FI3311380T3/en active
- 2016-06-10 CN CN202111617877.1A patent/CN114255772A/en active Pending
- 2016-06-10 PL PL16730777.6T patent/PL3311380T3/en unknown
- 2016-06-10 CN CN202111617514.8A patent/CN114255768A/en active Pending
- 2016-06-10 CA CA3150683A patent/CA3150683C/en active Active
- 2016-06-10 KR KR1020227020912A patent/KR102503707B1/en active IP Right Grant
- 2016-06-10 EP EP16730777.6A patent/EP3311380B1/en active Active
- 2016-06-10 CN CN201680047160.9A patent/CN108028046B/en active Active
- 2016-06-10 WO PCT/EP2016/063371 patent/WO2016202701A1/en active Application Filing
- 2016-06-10 RU RU2018101193A patent/RU2683487C1/en active
- 2016-06-10 CN CN202111617731.7A patent/CN114255771A/en active Pending
- 2016-06-10 KR KR1020237034198A patent/KR102660437B1/en active IP Right Grant
- 2016-06-10 EP EP23174592.8A patent/EP4239631A3/en active Pending
- 2016-06-10 KR KR1020237034199A patent/KR102660438B1/en active IP Right Grant
- 2016-06-10 KR KR1020237034196A patent/KR102660436B1/en active IP Right Grant
- 2016-06-10 EP EP23174595.1A patent/EP4235658A3/en active Pending
- 2016-06-10 AU AU2016278717A patent/AU2016278717B2/en active Active
- 2016-06-10 MX MX2017016171A patent/MX2017016171A/en active IP Right Grant
- 2016-06-10 EP EP23174596.9A patent/EP4239633A3/en active Pending
- 2016-06-10 BR BR112017026724-1A patent/BR112017026724B1/en active IP Right Grant
- 2016-06-10 CN CN202111617610.2A patent/CN114255770A/en active Pending
- 2016-06-10 KR KR1020227020911A patent/KR102502644B1/en active IP Right Grant
- 2016-06-10 KR KR1020227020910A patent/KR102588135B1/en active IP Right Grant
- 2016-06-10 ES ES16730777T patent/ES2950408T3/en active Active
- 2016-06-10 CA CA3150666A patent/CA3150666C/en active Active
- 2016-06-10 CA CA2989252A patent/CA2989252C/en active Active
- 2016-06-10 CN CN202111617515.2A patent/CN114255769A/en active Pending
- 2016-06-10 KR KR1020237034197A patent/KR20230145251A/en not_active Application Discontinuation
- 2016-06-10 MY MYPI2017001760A patent/MY178530A/en unknown
- 2016-06-10 EP EP23174598.5A patent/EP4231287A1/en active Pending
- 2016-06-10 KR KR1020227020909A patent/KR102502643B1/en active IP Right Grant
- 2016-06-10 KR KR1020207019023A patent/KR102412485B1/en active IP Right Grant
- 2016-06-10 CA CA3150675A patent/CA3150675C/en active Active
- 2016-06-10 EP EP23174593.6A patent/EP4239632A3/en active Pending
- 2016-06-10 CA CA3150637A patent/CA3150637C/en active Active
- 2016-06-10 EP EP24165639.6A patent/EP4365895A2/en active Pending
- 2016-06-15 AR ARP160101779A patent/AR105006A1/en unknown
-
2017
- 2017-12-15 US US15/843,358 patent/US10431230B2/en active Active
-
2018
- 2018-01-09 ZA ZA2018/00147A patent/ZA201800147B/en unknown
- 2018-05-30 HK HK18107099.5A patent/HK1247730A1/en unknown
-
2019
- 2019-08-23 US US16/549,914 patent/US11062719B2/en active Active
- 2019-12-19 JP JP2019228825A patent/JP6839260B2/en active Active
-
2020
- 2020-07-30 AR ARP200102148A patent/AR119537A2/en unknown
- 2020-07-30 AR ARP200102150A patent/AR119541A2/en unknown
- 2020-11-19 AR ARP200103208A patent/AR120507A2/en unknown
- 2020-11-19 AR ARP200103207A patent/AR120506A2/en unknown
-
2021
- 2021-02-12 JP JP2021020355A patent/JP7089079B2/en active Active
- 2021-07-02 US US17/367,037 patent/US11670312B2/en active Active
- 2021-10-29 US US17/515,286 patent/US11341980B2/en active Active
- 2021-10-29 US US17/515,242 patent/US11341978B2/en active Active
- 2021-10-29 US US17/515,267 patent/US11341979B2/en active Active
-
2022
- 2022-06-09 JP JP2022093393A patent/JP7322248B2/en active Active
- 2022-06-09 JP JP2022093394A patent/JP7322249B2/en active Active
- 2022-06-09 JP JP2022093395A patent/JP7323679B2/en active Active
-
2023
- 2023-04-25 US US18/139,252 patent/US20240005931A1/en active Pending
- 2023-05-09 US US18/195,220 patent/US20230360657A1/en active Pending
- 2023-05-09 US US18/195,250 patent/US20230360658A1/en active Pending
- 2023-05-09 US US18/195,213 patent/US20230360656A1/en active Pending
- 2023-07-27 JP JP2023122204A patent/JP2023159096A/en active Pending
- 2023-08-29 JP JP2023139247A patent/JP2023164895A/en active Pending
- 2023-08-29 JP JP2023139246A patent/JP2023164894A/en active Pending
- 2023-08-29 JP JP2023139245A patent/JP2023164893A/en active Pending
Also Published As
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230360658A1 (en) | Downscaled decoding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request |
Effective date: 20220301 |
|
| EEER | Examination request |
Effective date: 20220301 |
|
| EEER | Examination request |
Effective date: 20220301 |
|
| EEER | Examination request |
Effective date: 20220301 |
|
| EEER | Examination request |
Effective date: 20220301 |
|
| EEER | Examination request |
Effective date: 20220301 |
|
| EEER | Examination request |
Effective date: 20220301 |