RU2813317C1 - Improved harmonic transformation based on block of sub-bands - Google Patents

Abstract

FIELD: audio systems.

SUBSTANCE: invention relates to source audio coding systems that use a harmonic transformation method for high frequency reconstruction (HFR), as well as to digital effects processors, such as exciters, in which the generation of harmonic distortion adds brightness to the processed signal, and to time expanders, in which the duration signal increases while maintaining the spectral composition. A system and method are described, which are configured to generate a time-stretched and/or frequency-converted signal from an input signal. The device includes an analysis filter unit (101) configured to create an analysis subband signal from an input signal, where the analysis subband signal includes a number of complex-valued analysis discrete values, each of which has a phase and an amplitude. In addition, the device includes a subband processing module (102) configured to determine a synthesis subband signal from an analysed subband signal using a subband transform factor Q and a subband stretch factor S. The subband processing module (102) performs non-linear processing on a unit basis, where the amplitude of the discrete values of the synthesized subband signal is determined from the amplitude of the corresponding samples of the analysed subband signal and the predetermined discrete value of the analysed subband signal. In addition, the device includes a synthesis filter unit (103) configured to generate a time-stretched and/or frequency-converted signal from a synthesis subband signal.

EFFECT: providing an HFR method based on harmonic transformation with reduced computational complexity, while ensuring good audio signal quality for various types of audio signals.

19 cl, 7 dwg

Description

TECHNICAL APPLICATION AREA

This document applies to source audio encoding systems that use a harmonic transform technique for High Frequency Reconstruction (HFR), as well as to digital effects processors such as exciters, in which the generation of harmonic distortion adds brightness to the processed signal, and to time expanders, in which in which the signal duration increases while maintaining the spectral composition.

BACKGROUND OF THE INVENTION

WO 98/57436 created the concept of conversion as a method of recreating a high frequency band from a lower frequency band of an audio signal. By using this concept to encode an audio signal, significant bit rate savings can be achieved. In an HFR-based audio encoding system, a low-bandwidth signal is passed to a base waveform encoder and the high frequencies are regenerated using a transform and an extremely low bit rate of additional information describing the target spectrum shape at the decoder side. For low bit rates, where the bandwidth of the underlying encoded signal is narrow, recreating the high frequency band with visually pleasing characteristics becomes increasingly important. The harmonic transformation defined in WO 98/57436 works well for complex musical material in low transition frequency situations. WO 98/57436 is incorporated by reference into this specification. The principle of harmonic transformation is that a sinusoid with a frequency displayed as a sine wave with frequency , Where - an integer that specifies the conversion order. In contrast, HFR based on single sideband (SSB) modulation displays a sine wave with a frequency into a sine wave with frequency , Where - fixed frequency shift. For a given low-frequency bandwidth signal, the SSB conversion typically results in a dissonant ringing artifact. Because of these artifacts, harmonic transform-based HFR is generally preferred over SSB-based HFR.

In order to achieve improved audio signal quality, high-quality harmonic transform based HFR techniques typically use complex banks of complex modulated filters with high frequency resolution and high oversampling to achieve the required audio signal quality. High frequency resolution is typically used to avoid unwanted intermodulation distortion that occurs during nonlinear processing or processing of different subband signals that can be considered as sums of a series of sinusoids. For fairly narrow subranges, i.e. With sufficiently high frequency resolution, HFR methods based on high-quality harmonic transformation strive to ensure that each subband contains no more than one sinusoid. As a result, intermodulation distortion caused by nonlinear processing can be avoided. On the other hand, a high degree of time oversampling can be useful to avoid aliasing distortion that can be caused by filter banks and nonlinear processing. Additionally, a certain degree of frequency oversampling may be necessary for short duration non-periodic signals to avoid leading echoes caused by non-linear processing of sub-band signals.

Moreover, harmonic transform based HFR methods generally use processing based on two groups of filter banks. The first part of harmonic transform-based HFR typically uses a high frequency and time resolution and/or frequency oversampling analysis/synthesis filter bank to generate the high frequency component of the signal from the low frequency component of the signal. The second part of the harmonic transform based HFR typically uses a filter bank with a relatively low frequency resolution, such as a QMF filter bank, which is used to apply additional spectral or HFR information to the high frequency component, i.e. to perform the so-called HFR processing, in order to generate a high-frequency component with the required spectrum shape. The second part of the filter banks is also used to combine the low frequency component of the signal with the modified high frequency component of the signal to create a decoded audio signal.

As a result of using a sequence of two banks of filter banks and using analysis/synthesis filter banks with high frequency and time resolution and/or frequency resampling, the computational complexity of harmonic transform-based HFR can be relatively high. Accordingly, there is a need to provide harmonic transform based HFR methods with reduced computational complexity while still providing good audio signal quality for various types of audio signals (eg, short non-periodic and stationary audio signals).

BRIEF DESCRIPTION OF THE INVENTION

According to one of the features, the so-called. subband block based harmonic transform can be used to suppress combinational components caused by nonlinear processing of subband signals, i.e. the combinational components in the subbands can be suppressed or reduced by performing nonlinear block-based processing of the subband signals of the harmonic converter. As a result, a harmonic transform can be applied that uses an analysis/synthesis filter bank with a relatively low frequency resolution and/or a relatively low oversampling rate. For example, a block of QMF filters may be used.

Nonlinear block-based processing of a subband block-based harmonic transform system involves processing a time block of complex subband discrete values. Processing a block of complex subband samples may include a well-known modification of the phase of the complex subband samples and a superposition of several modified samples to form an output subband sample. This block-based processing has the net effect of suppressing or reducing the combinational components that would otherwise occur with subband input signals comprising multiple sinusoids.

Because subband-based harmonic transform may use analysis/synthesis filter banks with relatively low frequency resolution, and because a lower degree of oversampling may be required, harmonic transform based on block-based subband processing may have lower computational complexity compared to high-quality harmonic converters, i.e. with harmonic converters having high frequency resolution and using discrete value processing. At the same time, it has been experimentally shown that for many types of audio signals, the quality of the audio signal that can be achieved using a subband block-based harmonic transform is almost the same as using a discrete value-based harmonic transform. However, it has been observed that the audio signal quality obtained for short duration non-periodic audio signals is generally degraded compared to the audio signal quality that can be achieved by high quality harmonic converters, i.e. harmonic converters using high frequency resolution. It was found that the reduced quality for short-term non-periodic signals may be due to time dilution caused by block processing.

In addition to the quality issues raised above, the complexity of harmonic mapping based on subband blocks still remains higher than the complexity of the simplest SSB-based HFR methods. This is so because synthesizing the required bandwidth usually requires multiple signals with different orders transformations. As a rule, each order block-based harmonic transform requires a different structure of analysis and synthesis filter blocks.

In view of the above analysis, there is a particular need to improve the quality of subband block-based harmonic transform for short duration non-periodic and voice signals while maintaining the quality of stationary signals. As will be described below, quality improvement can be achieved by fixed, or signal-adaptive, modification of nonlinear block processing. In addition, there is a need to further reduce the complexity of the harmonic transform based on a block of subbands. As will be described below, a reduction in computational complexity can be achieved by efficiently implementing a multi-order transform based on a block of subbands within a single pair of analysis and synthesis filter banks. As a result, for the harmonic transformation several orders, one block of analyzing/synthesizing filters can be used, for example, a block of QMF filters. In addition, the same pair of analyzer/synthesis blocks can be used for the harmonic transform (i.e., the first part of the HFR based on the harmonic transform) and for the HFR processing (i.e., the second part of the HFR based on the harmonic transform). filters, and thus the full harmonic transform-based HFR can be loaded onto a single analysis/synthesis filter bank. In other words, on the input side, only one single analysis filter bank can be used to generate a number of analysis subband signals, which are then subject to harmonic transformation and HFR processing. Ultimately, only one single synthesis filter bank can be used to generate the decoded signal on the output side.

In one aspect, a system is described configured to generate a time-stretched and/or frequency-converted signal from an input signal. The system may include an analysis filter bank configured to create an analysis subband signal from the input signal. The analyzed subband can be associated with one of the frequency bands of the input signal. The analyzed subband signal may include a number of complex-valued analyzed discrete values, each of which has a phase and amplitude. The analysis filter bank may be a quadrature mirror filter bank, a windowed discrete Fourier transform, or a wavelet transform. In particular, the analysis filter bank may be a 64-point quadrature mirror filter bank. Thus, the analysis filter bank may have low frequency resolution.

The analysis filter bank can apply a step to the input signal analysis over time, and/or the block of analyzing filters may have a spacing analyzed frequencies so that the frequency band associated with the signal of the analyzed subband has a nominal width , and/or the block of analyzing filters could contain a number N of analyzed subranges, N > 1, where n is the index of the analyzed subrange, . It should be noted that due to the overlap of adjacent frequency bands, the actual spectral width of the analyzed subband may be greater . However, the frequency spacing between adjacent analyzed subbands, as a rule, has the form separation of analyzed frequencies.

The system may include a subband processing module configured to determine a synthesis subband signal from an analysis subband signal using a coefficient subband transformation and subband stretch factor S. At least one of the coefficients, Q or S , may be greater than one. The subband processing module may include a block extractor configured to extract a frame consisting of L input samples from a set of complex-valued parsed samples. The frame length L may be greater than one, however, in some embodiments of the invention, the frame length L may be equal to one. Alternatively or in addition, the block extractor may be configured to apply a block jump value of p samples to a set of samples to be analyzed before extracting the next frame consisting of L input samples. By repeatedly applying the block jump value to a number of samples being analyzed, a set of frames consisting of the input samples can be generated.

It should be noted that the frame length L and/or block hop size p may be arbitrary numbers and are not necessarily integers. In this or other cases, the block extractor may be configured to interpolate two or more parsed samples to produce a frame input sample consisting of L input samples. For example, if the frame length and/or block jump amount are fractional numbers, the input sample value of the frame consisting of the input samples can be obtained by interpolating two or more adjacent samples being analyzed.

Alternatively or in addition, the block extractor may be configured to downsample a number of analyzed samples to produce a frame input sample consisting of L input samples. In particular, the block extractor may be configured to downsample a number of analyzed samples with a subband conversion factor Q. Thus, the block extractor can contribute to harmonic transformation and/or time stretching by performing a downsampling operation.

The system, and in particular the subband processing module, may include a nonlinear frame processing module configured to determine a frame of processed samples based on the frame of input samples. The definition may be repeated over a set of frames consisting of the input samples, and thus generating a set of frames consisting of the processed samples. The determination can be performed by determining, for each processed sample value frame, the phase of the processed sample value - by shifting the phase of the corresponding input sample value. In particular, the nonlinear frame processing unit may be configured to determine the phase of the sample being processed by shifting the phase of the corresponding input sample by a phase shift amount that is based on a predetermined input sample from a frame consisting of the input samples, a transform factor Q , and a factor S sub-range stretching. The amount of phase shift may be based on a predetermined input sample value multiplied by . In particular, the amount of phase shift can be specified by a predetermined input discrete value multiplied by , plus parameter phase correction. Parameter phase correction can be determined experimentally for a range of input signals having specific acoustic properties.

In one preferred embodiment of the invention, the predetermined input sample value is the same for each frame sample value being processed. In particular, the predetermined input sample may be the central sample of a frame consisting of the input samples.

Alternatively or in addition, the determination may be performed by determining, for each processed sample frame, the amplitude of the processed sample based on the amplitude of the corresponding input sample and the amplitude of the predetermined input sample. In particular, the nonlinear frame processing unit may be configured to determine the amplitude of the processed sample as the average of the amplitude of the corresponding input sample and the amplitude of the predetermined input sample. The amplitude of the processed discrete value can be determined as the geometric average of the amplitude of the corresponding input discrete value and the amplitude of the predetermined input discrete value. More specifically, the geometric mean can be defined as the amplitude of the corresponding input sample raised to the power , multiplied by the amplitude of the predetermined input discrete value raised to the power . Typically, the geometric amplitude weighting parameter is . In addition, the geometric amplitude weighting parameter may be a function of the subband transform factor Q and the subband stretch factor S. In particular, the geometric amplitude weighting parameter can be , which results in reduced computational complexity.

It should be noted that the predetermined input sample used to determine the amplitude of the processed sample may be different from the predetermined input sample used to determine the phase of the processed sample. However, in a preferred embodiment of the invention, both predetermined input samples are the same.

In general, a nonlinear frame processing module can be used in a system to control the degree of harmonic transformation and/or time stretching. It can be shown that by determining the amplitude of the processed sample from the amplitude of the corresponding input sample and the amplitude of the predetermined input sample, system performance with respect to transient non-periodic and/or voice signals can be improved.

The system, and in particular the nonlinear subband processing module, may include an overlay and addition module configured to determine a synthesis subband signal by overlaying and adding samples from a set of frames of processed samples. The stacker and adder can apply the jump amount to successive frames of processed samples. Said jump amount may be equal to the block jump amount p multiplied by the subband stretch factor S. Thus, the stacker and adder module can be used in the system to control the degree of time stretching and/or harmonic transformation.

The system, and in particular the subband processing module, may include a window processing module located upstream of the overlay and addition module. The windowing module may be configured to apply a windowing function to a frame consisting of the samples being processed. In this way, the windowing function can be applied to a set of frames consisting of processed discrete values before the overlay and addition operation. The window function may have a length that corresponds to the length L of the frame. The window function may be a Gaussian window, a cosine window, a raised cosine window, a Hamming window, a Hann window, a rectangular window, a Bartlett window, and/or a Blackman window. Typically, a window function includes a series of discrete window function values, and superimposed and stacked discrete window function values from a series of window functions, shifted by an amount jump, can provide a set of discrete values that have a largely constant value of K.

The system may include a synthesis filter bank configured to generate a time-stretched and/or frequency-converted signal from the synthesis subband signal. The synthesized subband can be associated with the frequency band of the time-stretched and/or frequency-converted signal. The synthesis filter bank may be the corresponding filter bank, or the inverse transform, of the filter bank, or transform, of the analysis filter bank. In particular, the synthesis filter bank may be a 64-point inverse quadrature mirror filter bank. In one embodiment of the invention, the synthesis filter bank applies a step to the synthesis subband signal synthesis in time, and/or the block of synthesis filters has a spacing synthesized frequencies, and/or the block of synthesis filters contains the number M of synthesized subbands, , where m is the index of the synthesized subband, .

It should be noted that, typically, the analysis filter bank is configured to generate a number of analysis subband signals; the subband processing module is configured to determine a number of synthesized subband signals from the number of analyzed subband signals; and the synthesis filter bank is configured to generate a time-stretched and/or frequency-converted signal from a number of synthesis subband signals.

In one embodiment of the invention, the system may be configured to generate a signal that is time-stretched by a factor physical stretching in time and/or converted in frequency with a coefficient physical frequency conversion. In this case, the subband stretching coefficient can have the form , the subband conversion coefficient can have the form ; and/or the analysis subband index n associated with the analysis subband signal and the synthesis subband index m associated with the synthesis subband signal may be related by the relation . If the expression is not an integer, n can be chosen as the closest one, i.e. nearest lesser or greater integer to the term .

The system may include a control data receiving module configured to receive control data indicative of instantaneous acoustic properties of the input signal. These instantaneous acoustic properties can, for example, be reflected by classifying the input signal into different classes of acoustic properties. These classes may include a class of transient properties for a short-term non-periodic signal, and/or a class of stationary properties for a stationary signal. The system may include a signal classifier or may receive control data from the signal classifier. The signal classifier may be configured to analyze the instantaneous acoustic properties of the input signal and/or configured to specify control data reflecting the instantaneous acoustic properties.

The subband processing module may be configured to determine a synthesis subband signal based on the control data. In particular, the block extractor may be configured to set the frame length L in accordance with the control data. In one embodiment of the invention, a short frame length L is set if the control data reflects a short-term non-periodic signal; and/or a larger frame length L is set if the control data reflects a stationary signal. In other words, the frame length L may be reduced for short non-periodic portions of the signal compared to the frame length L used for stationary portions of the signal. In this way, the subband processing module can take into account the instantaneous acoustic properties of the input signal. As a result, the efficiency of the system with respect to short non-periodic and/or voice signals can be increased.

As described above, the analysis filter bank is typically configured to create a number of analysis subband signals. In particular, the analysis filter bank may be configured to create a second analysis subband signal from the input signal. Said second analysis subband signal is typically associated with a different frequency band of the input signal than the analysis subband signal. The second analysis subband signal may include a number of complex-valued second analysis samples.

The subband processing module may include a second block extractor configured to extract a set of second input samples by applying a block jump value p to the set of second analyzed samples, i.e. in one preferred embodiment of the invention, the second block extractor applies a frame length L =1. Typically, every second input sample corresponds to a frame consisting of input samples. This correspondence may relate to timing and/or sampling features. In particular, the second input sample and the corresponding frame consisting of the input samples may refer to the same times of the input signal.

The subband processing module may include a second nonlinear frame processing module configured to determine a frame consisting of second samples to be processed, from a frame consisting of input samples, and a corresponding second input sample. Determining a frame consisting of second processed samples may be accomplished by determining, for each second processed sample frame, the phase of the second processed sample by shifting the phase of the corresponding input sample by a phase shift amount that is based on the corresponding second input sample, the transformation factor Q and subband stretch factor S. In particular, the phase shift may be performed as described herein, where the second processed sample value is taken in place of a predetermined input sample value. In addition, determining a frame consisting of second processed samples may be performed by determining, for each second processed sample of the frame, the amplitude of the second processed sample based on the amplitude of the corresponding input sample and the amplitude of the corresponding second input sample. In particular, the amplitude may be determined as described herein, where the second processed sample value is taken in place of a predetermined input sample value.

Thus, a second nonlinear frame processing module may be used to extract a frame, or set of frames, consisting of processed discrete values, from frames taken from two different analysis subband signals. In other words, a single synthesis subband signal may be derived from two or more different analysis subband signals. As described herein, this may be useful when a single pair of analysis and synthesis filter banks is used for a range of harmonic transform orders and/or degrees of time stretching.

In order to determine one or two analysis subbands that should contribute to the synthesized subband with index m , the relationship between the frequency resolution of the analysis and synthesis filter banks can be taken into account. In particular, it can be stipulated that if a member is an integer n , then the synthesized subband signal can be determined on the basis of a frame consisting of processed discrete values, i.e. the signal of the synthesized subband can be determined from a single signal of the analyzed subband, corresponding to the integer index n . Alternatively or in addition, it may be stipulated that if a member is not an integer, where n is the nearest integer, then the synthesis subband signal can be determined based on the frame consisting of the second processed discrete values, i.e. the synthesized subband signal can be determined from two analyzed subband signals corresponding to the nearest integer index value n and the adjacent integer index value. In particular, the second analysis subband signal may correspond to an analysis subband index n +1 or n −1.

According to the following feature, a system configured to generate a time-stretched and/or frequency-converted signal from an input signal is described. This system is particularly adapted to generate a time-stretched and/or frequency-converted signal under the influence of a control signal and thus take into account the instantaneous acoustic properties of the input signal. This may be particularly well suited for improving the transient response of a system.

The system may include a control data receiving module configured to receive control data indicative of instantaneous acoustic properties of the input signal. In addition, the system may include an analysis filter bank configured to create an analysis subband signal from the input signal; where the signal of the analyzed subband includes a number of complex-valued analyzed discrete values, each of which has a phase and amplitude. In addition, the system may include a subband processing module configured to determine a synthesis subband signal from an analysis subband signal using a subband transform factor Q , a subband stretch factor S , and control data. Typically, at least one of the coefficients, Q or S , is greater than one.

The subband processing module may include a block extractor configured to extract a frame containing L input samples from a set of complex-valued parsed samples. The frame length L can be greater than one. In addition, the block extractor can be configured to set the frame length L in accordance with the control data. The block extractor may also be configured to apply a block jump value of p samples to a set of samples to be analyzed before extracting the next frame consisting of L input samples; thus, a set of frames consisting of input discrete values is generated.

As described above, the subband processing module may include a nonlinear frame processing module configured to determine a frame consisting of processed samples from a frame consisting of input samples. This may be accomplished by determining, for each processed sample value of the frame, the phase of the processed sample value by shifting the phase of the corresponding input sample value; and by determining, for each processed sample value of the frame, the amplitude of the processed sample value based on the amplitude of the corresponding input sample value.

Additionally, as described above, the system may include an overlay and addition module configured to determine a synthesis subband signal by overlaying and adding samples from a set of frames consisting of processed samples; and a synthesis filter bank configured to generate a time-stretched and/or frequency-converted signal from the synthesis subband signal.

In another aspect, a system is described configured to generate a time-stretched and/or frequency-converted signal from an input signal. The system may be particularly well adapted to perform a series of time stretching and/or frequency conversion operations in a single pair of analysis and synthesis filter banks. The system may include an analysis filter bank configured to generate first and second analysis subband signals from an input signal, wherein each of the first and second subband signals includes a number of complex valued analysis samples, referred to, respectively, as the first and second analysis samples, and each the analyzed discrete value has a phase and amplitude. Typically, the first and second signals of the analyzed subbands correspond to different frequency bands of the input signal.

The system may also include a subband processing module configured to determine a synthesis subband signal from the first and second analysis subband signals using a subband transform factor Q and a subband stretch factor S. Typically, at least one of the coefficients, Q or S , is greater than one. The subband processing module may include a first block extractor configured to extract a frame consisting of L first input samples from a set of first analyzed samples; frame length L is greater than one. The first block extractor may be configured to apply a block jump amount of p samples to a series of first analyzed samples before extracting the next frame consisting of the L first input samples; and thus a set of frames consisting of the L first input samples is generated. In addition, the subband processing module may include a second block extractor configured to extract a set of second input samples by applying a block jump value p to the set of second analyzed samples; where every second input sample corresponds to a frame consisting of the first input samples. The first and second block extractors may have any of their characteristics described herein.

The subband processing module may include a nonlinear frame processing module configured to determine a frame consisting of processed samples, a frame consisting of first input samples, and a corresponding second input sample. This may be accomplished by determining, for each processed sample value of the frame, the phase of the processed sample value - by shifting the phase of the corresponding first input sample value; and/or determining, for each processed sample value of the frame, the amplitude of the processed sample value based on the amplitude of the corresponding first input sample value and the amplitude of the corresponding second input sample value. In particular, the nonlinear frame processing module may be configured to determine the phase of the sample being processed by shifting the phase of the corresponding first input sample by a phase shift amount that is based on the corresponding second input sample, the transform factor Q , and the subband stretch factor S.

In addition, the subband processing module may include an overlay and addition module configured to determine a synthesized subband signal by overlaying and adding samples from a set of frames consisting of the processed samples, where the overlay and addition module may apply a jump amount to successive frames consisting of processed discrete values. The jump size may be equal to the block jump size p multiplied by the subband stretch factor S. Finally, the system may include a synthesis filter bank configured to generate a time-stretched and/or frequency-converted signal from the synthesis subband signal.

It should be noted that the various components of the systems described herein may include any or all of the features described herein with respect to those components. This applies in particular to the analysis and synthesis filter banks, subband processing module, nonlinear processing module, block extractors, overlay and addition module, and/or windowing module described in various parts of this document.

The systems described herein may include a number of subband processing modules. Each subband processing module may be configured to determine an intermediate synthesized subband signal using different subband transform coefficients Q and/or different subband stretch coefficients S. The systems may also include, downstream of a number of subband processing modules and upstream of a synthesis filter bank, a fusion module configured to merge corresponding intermediate synthesis subband signals into a synthesis subband signal. Thus, the systems can be used to perform a number of time stretching and/or harmonic transformation operations, despite using only a single pair of analysis and synthesis filter banks.

Systems may include, upstream of the analysis filter bank, a core decoder configured to decode the bit stream into an input signal. Systems may also include an HFR processing module downstream of the merge module (if the merge module is present) and upstream of the synthesis filter bank. The HFR processing module may be configured to apply spectral band information extracted from the bitstream to the synthesis subband signal.

According to another aspect, an external device is described for decoding a received signal including at least a low frequency component of an audio signal. The external device may include a system in accordance with any of the features and characteristics described herein for generating a high frequency component of an audio signal from a low frequency component of an audio signal.

According to the following feature, a method for generating a time-stretched and/or frequency-converted signal from an input signal is described. This method is particularly well suited to improving the transient response of time stretching and/or frequency conversion operations. The method may include the step of generating an analysis subband signal from an input signal, where the analysis subband signal includes a series of complex valued analysis samples, each of which has a phase and amplitude.

The method may generally include the step of determining a synthesis subband signal from an analysis subband signal using a transform factor Q and a subband stretch factor S. Typically, at least one of the coefficients, Q or S , is greater than one. In particular, the method may include the step of extracting a frame consisting of L input samples from a set of complex-valued parsed samples, where the frame length L is typically greater than one. In addition, a block jump value of p samples may be applied to the set of samples being analyzed before retrieving the next frame containing L input samples; thus, a set of frames consisting of the input discrete values is generated. In addition, the method may include the step of determining a frame consisting of processed samples from the frame consisting of input samples. This may be accomplished by determining, for each successive frame sample, the phase of the sample being processed by shifting the phase of the corresponding input sample.

Alternatively or in addition, for each frame sample processed, the amplitude of the processed sample may be determined based on the amplitude of the corresponding input sample and the amplitude of the predetermined input sample.

The method may also include the step of determining a synthesis subband signal by overlaying and adding samples from a set of frames consisting of their processed samples.

Ultimately, a time-stretched and/or frequency-converted signal can be generated from the synthesized subband signal.

In another aspect, a method for generating a time-stretched and/or frequency-converted signal from an input signal is described. This method is particularly well suited to improve the efficiency of time stretching and/or frequency conversion operations in combination with short-term non-periodic input signals. The method may include the step of receiving control data reflecting instantaneous acoustic properties of the input signal. The method may also include the step of generating an analysis subband signal from an input signal, where the analysis subband signal includes a number of complex valued analysis samples, each of which has a phase and amplitude.

In the next step, the signal of the synthesized subband can be determined from the signal of the analyzed subband using the subband transformation coefficient Q , the subband stretch coefficient S and control data. Typically, at least one of the coefficients, Q or S , is greater than one. In particular, the method may include the step of extracting a frame consisting of L input samples from a set of complex-valued parsed samples, where the frame length L is typically greater than one, and where the frame length L is set in accordance with the control data. In addition, the method may include the step of applying a jump value of a block of p samples to a number of analyzed subbands before extracting the next frame consisting of the L input samples, thereby generating a set of frames consisting of the input samples. Then, from a frame consisting of input samples, a frame consisting of process samples can be determined by determining for each process sample frame the phase of the process sample by shifting the phase of the corresponding input sample, and the amplitude of the process sample based on the amplitude of the corresponding input discrete value.

The synthesis subband signal may be determined by overlaying and adding samples from a set of frames consisting of the processed samples, and a time-stretched and/or frequency-converted signal may be generated from the synthesis subband signal.

According to the following feature, a method for generating a time-stretched and/or frequency-converted signal from an input signal is described. The technique may be particularly well adapted to perform a series of time stretching and/or frequency conversion operations using a single pair of analysis/synthesis filter banks. At the same time, the method is well adapted for processing short-term non-periodic input signals. The method may include the step of generating first and second analysis subband signals from an input signal, wherein each of the first and second analysis subband signals includes a number of complex valued analysis samples, referred to, respectively, as first and second analysis samples, wherein each analysis sample has phase and amplitude.

Additionally, the method may include the step of determining a synthesis subband signal from the first and second analysis subband signals using a subband transform coefficient Q and a subband stretch coefficient S , wherein at least one of the coefficients, Q or S , is typically greater than one. In particular, the method may include the step of extracting a frame consisting of L first input samples from a set of first analyzed samples, where the frame length L is typically greater than one. A block jump value of P samples may be applied to the set of first samples analyzed before extracting the next frame consisting of the L first input samples, thereby generating a set of frames consisting of the first input samples. The method may also include the step of retrieving a set of second input samples by applying a block jump value p to a set of second analyzed samples, where each second original sample corresponds to a frame consisting of the first input samples.

The method continues by defining a frame consisting of the samples to be processed, a frame consisting of the first input samples, and a corresponding second input sample. This can be done by determining, for each sampled value of the frame, the phase of the processed sample value - by shifting the phase of the corresponding first input sample value, and the amplitude of the processed discrete value - based on the amplitude of the corresponding first input discrete value and the amplitude of the corresponding second input discrete value.

The synthesized subband signal can then be determined by superimposing and adding the samples from a set of frames consisting of the processed samples. Ultimately, a time-stretched and/or frequency-converted signal can be generated from the synthesized subband signal.

According to another feature, a program implemented in software is described. The program, implemented in software, may be adapted to be executed on a processor and to perform the method steps and/or implement the features and characteristics described herein when implemented on a computing device.

According to another feature, a storage medium is described. The storage medium may include a program implemented in software adapted for execution on a processor and for performing the steps of the method and/or implementing the features and characteristics described herein when implemented on a computing device.

In another aspect, a computer program product is described. The computer program product may include executable instructions for performing method steps and/or for implementing the features and characteristics described herein when executed on a computer.

It should be noted that the methods and systems, including preferred embodiments thereof, as described in this patent application may be used alone or in combination with other methods and systems disclosed herein. In addition, all features of the methods and systems described in this patent application can be arbitrarily combined. In particular, the characteristic features of the claims can be combined with each other in any way.

BRIEF DESCRIPTION OF GRAPHIC MATERIALS

The present invention will be described below by way of illustrative examples, without limiting the scope or spirit of the invention, with reference to the accompanying drawings, wherein:

fig. 1 illustrates the principle of an example of harmonic transform based on a block of subbands;

fig. 2 illustrates the operating principle of an example of nonlinear processing of a subband block with one input subband;

fig. 3 illustrates the operating principle of an example of nonlinear processing of a subband block with two input subbands;

fig. 4 illustrates an example scenario for applying subband block based transform using multiple transform orders in an HFR enhanced audio codec;

fig. 5 illustrates an example of a multi-order transform operation scenario based on a subband block, applying a separate parsing filter block for each transform order;

fig. 6 illustrates an example scenario for efficient operation of a multi-order transform based on a subband block using a single 64-band QMF filter bank; And

fig. 7 illustrates the transient response for time-stretching an example audio signal by a factor of two based on a block of subbands.

DESCRIPTION OF PREFERRED OPTIONS FOR IMPLEMENTING THE INVENTION

The embodiments described below are solely illustrative of the principles of the present invention for improved harmonic mapping based on a subband block. It should be understood that modifications and changes to the arrangements and details described herein will be apparent to those skilled in the art. It is therefore intended to be limited only by the scope of the forthcoming claims and not by the specific details presented herein for the purpose of describing and explaining embodiments of the invention.

Fig. 1 illustrates the principle of an example of transform, time stretching, or a combination of transform and time stretching based on a block of subbands. The time domain input signal is fed to an analysis filter bank 101, which produces several, or a series, of complex-valued subband signals. This series of subband signals is supplied to a subband processing unit 102, the operation of which is influenced by the control data 104. Each output subband of the subband processing unit 102 can be obtained either by processing one or by processing two input subbands, or even by superposing the result for several specified processed subranges. Several, or a series, of complex-valued output subbands are supplied to a synthesis filter bank 103, which in turn outputs a modified time domain signal. Control data 104 helps improve the quality of the modified time domain signal for some signal types. Control data 104 may be associated with a time domain signal. In particular, the control data 104 may be related to the type of the time domain signal, or dependent on the type of the time domain signal supplied to the analysis filter block 101. For example, control data 104 may indicate whether a time domain signal, or an instantaneous portion of a time domain signal, is a stationary signal, or whether the time domain signal is a transient non-periodic signal.

Fig. 2 illustrates the operating principle of an example nonlinear subband block processing 102 with a single input subband. Given the target values of the physical time stretching and/or transformation, and the physical parameters of the analysis and synthesis filter blocks 101 and 103, it is possible to derive the subband stretching time and transformation parameters, as well as the source subband index, which may also be referred to as the analyzed subband index, for each target subband index, which may also be referred to as the synthesized subband index. The purpose of subband block processing is to implement appropriate transformation, time stretching, or a combination of transformation and time stretching of a complex-valued source subband signal to generate a target subband signal.

In nonlinear subband block processing 102, block extractor 201 samples a finite-size discrete frame from a complex-valued input signal. The frame may be determined by the position of the input pointer and the subband conversion factor. The frame is subjected to nonlinear processing in the nonlinear processing module 202, and then processed by a finite length window method in the module 203. The window 203 may be a Gaussian window, a cosine window, a Hamming window, a Hann window, a rectangular window, a Bartlett window, a Blackman window, etc. d. The resulting samples are added to the previous output samples in an overlay and addition module 204, where the position of the output frame may be determined by the position of the output pointer. The input pointer has a fixed increment amount, also called the block jump value, and the output pointer has an increment equal to the increment of the original pointer, with a factor corresponding to the subband stretch factor, i.e. The block jump value is multiplied by the subband stretch factor. Repeating this chain of operations will generate an output signal with a duration equal to the duration of the input subband signal, with a factor corresponding to the subband stretch factor (up to the length of the synthesis window), and with complex frequencies converted with the subband conversion factor.

Control data 104 may influence non-linear block-based processing 102 of any of the blocks 201, 202, 203, 204 being processed. In particular, control data 104 may control the length of blocks extracted by block extractor 201. In one embodiment of the invention, the block length is reduced if control data 104 indicates that the time domain signal is a short-term non-periodic signal, while the block length is increased or maintained at a higher level if control data 104 indicates that that the time domain signal is a stationary signal. Alternatively or in addition, the control data 104 may affect the nonlinear processing unit 202, such as the parameters used in the nonlinear processing unit 202, and/or the window processing unit 203, such as the window used by the method processing unit 203. window.

Fig. 3 illustrates the operating principle of an example nonlinear subband block processing 102 with two input subbands. Given the physical time stretch and transform target values, and the physical parameters of the analysis and synthesis filter blocks 102 and 103, the time stretch and subband transform parameters are output, as well as two source subband indices for each target subband index. The purpose of subband block processing is to implement appropriate transform, time stretching, or a combination of transform and time stretching to combine two complex-valued source subband signals to generate a target subband signal. Block extractor 301-1 samples a finite-size frame consisting of samples from the first complex-valued source subband, and block extractor 301-2 samples a finite-size frame consisting of samples from the second complex-valued source subrange. In one embodiment, one of the block extractors 301-1 and 301-2 may generate a single discrete subband value, i.e. one of the block extractors 301-1 and 301-2 may apply a block length equal to one discrete value. Frames may be determined by the overall position of the input pointer and the subband conversion factor. The two frames extracted by block extractors 301-1 and 301-2, respectively, are subject to non-linear processing in module 302. Non-linear processing module 302 typically generates a single output frame from two input frames. The output frame is then processed by a finite length window method in module 203. The above process is repeated for a set of frames that are generated from a set of frames extracted from the two subband signals using the block hop value. The set of output frames is superimposed and folded in the superposition and addition module 204. Repeating this chain of operations will generate an output signal with a duration that is equal to the duration of the longest of the two input subband signals with a subband stretch factor (up to the length of the synthesis window). If two input subband signals carry the same frequencies, the output signal will have complex frequencies converted by the subband conversion factor.

As described in the context of FIG. 2, control data 104 may be used to modify the operation of various nonlinear processing blocks 102, such as the operation of extractor blocks 301-1 and 301-2. In addition, it should be noted that the operations described above are generally performed for all analysis subband signals produced by the analysis filter bank 101, and for all synthesis subband signals that are input to the synthesis filter bank 103.

In the following description with reference to FIGS. 1 - 3 and by adding appropriate mathematical terminology, a description of the principles of time stretching and subband block based transformation will be given.

The two main configuration parameters for a harmonic converter and/or time stretch device in general are:

• - required coefficient of physical stretching over time; And

• - required physical conversion coefficient.

Filter banks 101 and 103 may be of any complex exponential modulated type, such as, for example, QMF or windowed DFT, or wavelet transform. The analysis filter bank 101 and the synthesis filter bank 103 in modulation can be arranged uniformly or non-uniformly and can be determined from a wide range of prototype filters and/or windows. Although all of these second-order choices have an impact on subsequent design details such as phase corrections and subband display control, the major system design parameters for subband processing can generally be inferred from knowledge of two relationships - And - for the four filter block parameters given below, each of which is measured in physical units. In the above respects:

• - time step of the discrete value of the sub-range, or time step of the block 101 of analyzing filters (for example, measured in seconds [s]);

• - the frequency spacing of the sub-bands of the block 101 of the analyzing filters (for example, measured in Hertz [1/s]);

• - time step of the discrete subband value, or time step of the synthesis filter block 103 (for example, measured in seconds [s]); And

• is the frequency spacing of the subbands of the synthesis filter block 103 (for example, measured in Hertz [1/s]).

To configure the subband processing module 102, the following parameters must be calculated:

• S - subband stretching coefficient, i.e. a stretch factor that is applied in the subband processing module 102 to achieve an overall physical time stretch of the time domain signal by a factor ;

• Q - subband conversion coefficient, i.e. a conversion factor that is applied in the subband processing module 102 to achieve an overall physical conversion of the time domain signal with a factor ; And

• correspondence between the indices of the source and target subbands, where n denotes the index of the analyzed subband included in the subband processing module 102, and m denotes the corresponding synthesized subband at the output of the subband processing module 102.

When determining the subband stretching coefficient S, it was observed that the input signal to the synthesizing filter block 101 with a physical duration D corresponds to the number discrete values of the analyzed subranges at the input to the subrange processing module 102. These discrete values will be stretched into discrete values by subband processing module 102, which applies the subband stretch factor S. At the output of the synthesis filter block 102, these discrete values lead to an output signal having a physical duration . Since the last specified duration must match the specified value , i.e. because the duration of the output time domain signal must be time stretched compared to the input time domain signal by a physical time stretch factor , the following design rule is obtained:

. (1)

In order to determine the subband conversion factor Q which is applied in the subband processing unit 102 to achieve physical conversion , it is observed that the input sinusoid to the analyzing filter block 101 with a physical frequency will lead to a complex signal of the analyzed subband with a frequency dependent on discrete time , and the main contribution is made to the analyzed subrange with the index . Output sinusoid at the output of synthesis filter block 103 with the required converted physical frequency will be generated by supplying the synthesized subrange with an index with a complex subband signal with a discrete frequency . In this context, care should be taken to avoid synthesizing stepped output frequencies other than . Typically, this can be avoided by creating appropriate second order selections as discussed above, for example by selecting appropriate analysis/synthesis filter banks. Discrete frequency at the output of the subband processing module 102 must correspond to a discrete time-dependent frequency at the input of the subband processing module 102, multiplied by the subband conversion factor Q. Those. by equating And it is possible to determine the relationship between the coefficient physical conversion and subband conversion coefficient Q :

. (2)

Likewise, the corresponding index of the source, or analyzed, subband n of the subband processing module 102 for the index of a given target, or synthesized, subband m must obey the relation:

. (3)

In one embodiment of the invention it is assumed that , i.e. the frequency spacing of the synthesis filter bank 103 corresponds to the frequency spacing of the analysis filter bank 101 multiplied by the physical conversion factor, and a one-to-one mapping of the analysis subband index to the synthesis subband index, n = m , can be applied. In other embodiments of the invention, the subband index display may depend on details of the filter bank parameters. In particular, if the ratio of the frequency spacings of the synthesis filter bank 103 and the analysis filter bank 101 differs from the coefficient physical transformation, one or two source subbands can be specified for a given target subband. In the case of two source subbands, it may be preferable to use two adjacent source subbands, respectively, with indexes n and n +1. That is, the first and second source subbands are specified as , or .

Processing of subbands according to FIG. 2 with a single source subband will be described below as a function of the subband processing parameters S and Q. Let is the input signal to the extractor 201, blocks, and let p be the input step of the block. Those. is the complex-valued subband signal for the analyzed subband with index n . The block extracted by block extractor 201 can be considered, without loss of generality, to be defined by L =2 R +1 discrete values:

, (4)

where the integer l is the counting index of the block, L is the length of the block, and R is the integer R >0. Note that when the block is extracted from successive discrete values, and when downsampling is performed so that the input addresses are stretched by a factor of Q. If Q is an integer, this operation is generally simple to perform, while interpolation may be required for non-integer values of Q. This statement is also true for non-integer values of the increment p , i.e. block input step. In one embodiment of the invention, short interpolating filters, such as filters comprising two filter taps, may be applied to the complex-valued subband signal. For example, if a discrete value with a fractional time index k +0.5 is required, interpolation with two taps of the form can lead to sufficient quality .

An interesting special case of formula (4) is the case R =0, where the extracted block consists of a single discrete value, i.e. block length L =1.

In polar representation of a complex number , Where is the amplitude of a complex number, and - complex number phase, nonlinear processing unit 202 generating the output frame from input frame , is predominantly determined with the phase modification coefficient through

, , (5)

Where - geometric amplitude weighting parameter. Happening corresponds to the pure phase modification of the extracted block. Parameter phase correction depends on the details of the filter bank and the indices of the source and target subbands. In one embodiment of the invention, the parameter phase correction can be determined experimentally by sweeping a set of input sinusoids. In addition, the parameter phase correction can be derived by studying the phase difference of adjacent complex sinusoids of the target subbands or by optimizing the efficiency for an input signal of the Dirac pulse type. The phase modification coefficient T must be an integer so that the coefficients T -1 and 1 in the linear combination of phases in the first line of formula (5) are integers. With this assumption, i.e. under the assumption that the phase modification coefficient T is an integer, the result of the nonlinear modification is well determined even when the phase codes are ambiguous due to the addition of arbitrary integer factors .

Thus, formula (5) determines that the phase of a discrete value of the output frame is determined by shifting the phase of the corresponding discrete value of the input frame by a constant shift amount. This constant amount of shift may depend on the modification factor T , which in turn depends on the subband stretch factor and/or the subband conversion factor.

In addition, the constant amount of shift may depend on the phase of a particular input frame sample from the input frame. This particular input frame sample is held fixed to determine the phase of all output frame samples of this block. In the case of formula (5), the phase of the central discrete value of the input frame is used as the phase of the special discrete value of the input frame. In addition, the constant magnitude of the shift may depend on the parameter phase correction, which can be determined experimentally.

The second line of formula (5) specifies that the amplitude of the output frame sample can depend on the amplitude of the corresponding input frame sample value. In addition, the amplitude of the output frame sample may depend on the amplitude of a particular input frame sample. This particular input frame sample can be used to determine the amplitude for all output frame samples. In the case of formula (5), the central discrete value of the input frame is used as a special discrete value of the input frame. In one embodiment of the invention, the amplitude of an output frame sample may correspond to the geometric average of the amplitudes of a corresponding input frame sample and a particular input frame sample.

In window processing module 203, a window of length L is applied to the output frame, resulting in a windowed output frame:

. (6)

Finally, it is assumed that the frames are continued with zeros, and the operation of overlay and addition is given by the equation:

, (7)

where it should be noted that the overlay and addition unit 204 applies the block step Sp , i.e. time step, which is S times larger than the input step of the block p . Due to the specified difference in time steps in formulas (4) and (7), the duration of the output signal S times the duration of the input signal , i.e. the signal of the synthesized subband is stretched with the subband stretching factor S compared to the signal of the analyzed subband. It should be noted that this observation generally assumes that the window length L is negligible compared to the signal duration.

In the case when a complex sinusoid is used as an input signal to the processing of 102 subbands, i.e. the signal of the analyzed subband corresponds to a complex sinusoid

, (8)

by applying formulas (4) - (7) it can be determined that the output signal for processing 102 subbands, i.e. the corresponding signal of the synthesized subband has the form:

. (9)

Thus, a complex sinusoid with a discrete time-dependent frequency will transform into a complex sinusoid with a frequency dependent on discrete time provided that the window shifts with step Sp are summed up into the same constant value K for all k :

(10)

To illustrate, we can consider the special case of a pure transformation, where And . If the block input step is p =1 and R =0, all of the above, i.e., in particular, formula (5), is reduced to a point phase modification rule, or a discrete value-based phase modification rule:

. (eleven)

Advantage of using block size becomes obvious when within the signal of the analyzed subrange, the sum of sinusoids is considered. Difficulty associated with the point rule (11) for the sum of sinusoids with frequencies is that in the output signal of processing 102, i.e. in the signal synthesized sub-range, not only the desired frequencies will be present , but also the frequencies of combinational components in the form . Using a block and satisfying the window of formula (10), as a rule, leads to the suppression of these combinational components. On the other hand, a long block will result in a greater degree of unwanted time smearing for short duration non-periodic signals. Additionally, for pulse train signals, such as the human voice in the case of vowels or a single-tone instrument with a relatively low fundamental tone, combinational components may be desirable, as described in WO 2002/052545. This document is incorporated by reference into this specification.

To solve the problem of the relatively low efficiency of processing 102 based on a block of subbands for short-term non-periodic signals, it is proposed to use a non-zero value of the geometric amplitude weighting parameter in formula (5) . It has been observed (see, for example, Fig. 7) that the choice of the geometric amplitude weighting parameter improves the transient response of subband block-based processing 102 compared to using pure phase modification with , at the same time, while maintaining sufficient intermodulation distortion suppression power for stationary signals. The most attractive value for the amplitude weighting parameter is , for which formula (5) of nonlinear processing is reduced to the following calculation steps:

. (12)

These computational steps represent an amount of computational complexity equivalent to that of pure phase modulation in the case of in formula (5). In other words, determining the amplitude of discrete values of the output frame based on formula (5) of geometric means using the amplitude weighting parameter can be implemented without any additional cost in terms of computational complexity. At the same time, the efficiency of the harmonic converter for short-term non-periodic signals increases while maintaining the efficiency for stationary signals.

As has been described in the context of FIG. 1, 2, and 3, subband processing 102 may be further enhanced by the use of control data 104. In one embodiment of the invention, two configurations of subband processing 102 that share the same value may be used to implement signal-adapted subband processing 102. K in formula (11) and can involve different block lengths. A conceptual starting point in designing a subband processing module with signal-adapted configuration switching would be to think of two configurations running in parallel, with a selector switch at their outputs, where the position of the selector switch depends on the control data 104. Sharing the value of K will ensure that switching will occur without sharp transitions in the case of a single output complex sinusoid. For generalized signals, hard switching at the subband signal level will be automatically windowed by the surrounding filter bank structure 101, 103 so that no switching artifacts are introduced into the final output signals. It can be shown that when the block sizes vary significantly and the update rate of the control data is not very high, the superposition and addition process of formula (7) can produce an output signal similar to that of the above-described conceptual switching system, at the computational cost of a system having the configuration with the longest block. Thus, there is no loss in computational complexity associated with signal adaptation work. According to the above discussion, the shorter block length configuration is more suitable for short duration non-periodic signals and low pitch periodic signals, while the longer block length configuration is more suitable for stationary signals. Thus, a signal classifier may be used to classify audio signal passages into a transient non-periodic signal class and a non-transient signal class and pass this classification information as control data 104 to subband processing module 102 with signal-adapted configuration switching. Subband processing module 102 may use control data 104 to set certain processing parameters, such as block length in block extractors.

Below, the description of subband processing will be extended to the case of FIG. 3 with two subband input signals. Only changes made to the case of a single input signal will be described. In other words, reference is made to the information provided above. Let is the subband input signal to the first block extractor 301-1, and let - sub-band input signal to the second block extractor 301-2. The block extracted by the block extractor 301-1 is determined by formula (4), and the block extracted by the extractor 301-2 consists of a single discrete value:

. (13)

Those. in the described embodiment, the first block extractor 301-1 uses block length L , while the second block extractor 301-2 uses block length 1. In this case, nonlinear processing 302 generates an output frame , which can be determined by the formula

, (14)

and the remaining processing in modules 203 and 204 is similar to the processing described in the context of the single input signal case. In other words, it is proposed to replace the special discrete value of the frame in formula (5) with a single discrete subband value extracted from the corresponding second signal of the analyzed subband.

In one embodiment of the invention, where the frequency spacing ratio block 103 synthesis filters to frequency spacing block 101 of analyzing filters differs from the required coefficient physical transformation, it may be useful to determine the discrete values of the synthesized subrange with index m from the two analyzed subranges, respectively, with indices n and n +1. Given an index m, the corresponding index n can be given by an integer value obtained by truncating the index of the analyzed subrange given by formula (3). One of the analysis subband signals, for example, the analysis subband signal corresponding to index n , is supplied to the first block extractor 301-1, and the second analysis subband signal, for example, the signal corresponding to index n +1, is supplied to the second block extractor 301-2. Based on these two analyzed subband signals, the synthesis subband signal corresponding to the index m is determined in accordance with the processing described above. The routing of adjacent signals of the analyzed subbands to two block extractors 301-1 and 302-1 may be based on the remainder obtained by truncating the index value according to formula (3), i.e. on the difference between the exact value of the index given by formula (3) and the truncated integer value n obtained from formula (3). If the remainder is greater than 0.5, then the analysis subband signal corresponding to index n may be sent to the second block extractor 301-2, otherwise the analysis subband signal may be sent to the first block extractor 301-1.

Fig. 4 illustrates an example scenario for applying subband block based transform using multiple transform orders in an HFR enhanced audio codec. The transmitted bitstream is received by the base decoder 401, which produces a low-bandwidth decoded base signal at the sampling rate . This low-bandwidth base signal may also be referred to as the low-frequency component of the audio signal. Low sample rate signal can be resampled to output sample rate through a 32-band complex modulated QMF filter bank 402, and then a 64-band QMF synthesis filter bank (inverse QMF filters) 405. Both filter banks 402 and 405 have the same physical parameters And , and the HFR processing unit 404 typically passes unmodified lower subbands corresponding to the base low-bandwidth signal. The high-frequency content of the output signal is obtained by providing the higher subbands of the 64-band QMF synthesis filter bank 405 with output bands from the multiple converter module 403 subjected to spectrum shaping and modification performed by the HFR processing module 404. The multiple converter 403 takes the decoded base signal as an input and outputs a plurality of subband signals that represent a 64-band analysis of the superposition, or combination, of several converted signal components. In other words, the signal at the output of the multiple converter 403 must correspond to the converted signals of the synthesized subbands, which can be supplied to the synthesis filter bank 103, which in the case of FIG. 4 is represented by a block 405 of inverse QMF filters.

Possible implementations of multiple converter 403 are described in the context of FIG. 5 and 6. The purpose of the multiple transformer 403 is that if the HFR processing 404 is bypassed, each component corresponds to an integer physical transform without time-stretching the base signal ( , And ). For short-term non-periodic components of the base signal, HFR processing can sometimes compensate for poor transient response of the multiple converter 403, but consistently high quality can generally only be achieved if the transient response of the multiple converter itself is satisfactory. As described herein, the transducer control signal 104 may influence the operation of the multiple transducer 403 and thus provide satisfactory transient response of the multiple transducer 403. Alternatively or in addition, the above geometric weighting circuit (see, for example, Eq. 5) and/or formula (4)) may contribute to improving the transient response of the harmonic converter 403.

Fig. 5 illustrates an example scenario for operating a subband block-based multi-order transform module 403 applying a separate filter block 502-2, 502-3, 502-4 for each transform order. In the illustrated example, in the domain of a 64-band QMF filter bank operating at the output sample rate , three orders of transformation are generated and delivered . Fusion module 504 selects and combines the corresponding subbands from each transform coefficient branch into a single set of QMF subbands supplied to the HFR processing module.

Let's consider the first case . The immediate goal is that the processing sequence of 64-band QMF analysis 502-2, subband processing module 503-2, and 64-band QMF synthesis 405 results in a physical conversion with And (i.e. no stretching). When these three blocks are identified as modules 101, 102 and 103 in FIG. 1, it is found that And Thus, formulas (1) - (3) lead to the following specifications for the subband processing module 503-2. Subband processing module 503-2 must perform subband stretching to , subrange conversion to (i.e. absent) and provide a correspondence between the source subbands with index n and the target subbands with index m, having the form n = m (see formula (3)).

For the occasion An example system includes a sample rate converter 501-3 that downsamples the input sample rate by a factor of 3/2 - from before . The immediate goal is that the processing sequence consisting of 64-band QMF analysis 502-3, subband processing module 503-3, and 64-band QMF synthesis 405 results in a physical conversion with And (i.e. no stretching). By identifying the above three blocks as modules 101, 102 and 103, respectively, in FIG. 1, it is found that due to resampling And Thus, formulas (1) - (3) lead to the following specifications for the subband processing module 503-3. Subband processing module 503-3 must perform subband stretching to , subrange conversion to (i.e., do not execute) and ensure a correspondence between the source subranges with index n and the target subranges with index m, having the form n = m (see formula (3)).

For the occasion An example system includes a sample rate converter 501-4 that downsamples the input sample rate by half - from before . The immediate intent is that the processing sequence consisting of 64-band QMF analysis 502-4, subband processing module 503-4, and 64-band QMF synthesis 405 results in a physical conversion with And (i.e. no stretching). By identifying the above three blocks as modules 101, 102 and 103, respectively, in FIG. 1, it is found that due to resampling And , formulas (1) - (3) lead to the following specifications for the subband processing module 503-4. Subband processing module 503-4 must perform subband stretching to S =4, subband conversion to (i.e., do not execute) and provide a correspondence between the source subranges with index n and the target subranges with index m, which has the form n = m .

As a summary for the example scenario of FIG. 5, all modules 504-2 through 503-4 perform pure stretches of the subband signals and utilize the uniform nonlinear processing of the input subband blocks described in the context of FIG. 2. Control signal 104, if present, can simultaneously influence the operation of all three subband processing modules. In particular, control signal 104 may be used to simultaneously switch between long block processing and short block processing depending on the type (short-term non-periodic or non-transient) of the input signal passage. Alternatively or in addition, if three modules 504-2-504-4 use a non-zero geometric amplitude weighting parameter , then the transient response of the multiple converter will be improved compared to the case .

Fig. 6 illustrates an example scenario for efficiently operating a multi-order transform based on a subband block using a single 64-band QMF analysis filter bank. In fact, using three separate QMF analysis units and two sample rate converters in FIG. 5 leads to a fairly high computational complexity, as well as some inconveniences in implementing frame-based processing due to the sampling rate conversion in module 501-3, i.e. due to fractional sampling rate conversion. Therefore, it is proposed to replace the two conversion branches, including modules 501-3→502-3→503-3 and 501-4→502-4→503-4, with modules 603-3 and 603-4, respectively, while branch 502-2→ 503-2 compared to FIG. 5 remains unchanged. All three orders of transformation are performed in the filter block region with reference to FIG. 1, where And . In other words, only one analysis filter bank 502-2 and one synthesis filter bank 405 are used, thereby reducing the overall computational complexity of the multiple converter.

When The specifications for the subband processing unit 603-3, having the form of formulas (1) to (3), are that the subband processing unit 603-3 must perform subband stretching with S = 2 and subband conversion with Q = 3/2, so that the correspondence between the source subbands with index n and the target subbands with index m has the form . For the occasion The specifications for the subband processing unit 603-4, having the form of formulas (1) to (3), are that the subband processing unit 603-4 must perform subband stretching with S = 2 and subband transformation with Q = 2, so that so that the correspondence between the source ranges with index n and the target subranges with index m has the form .

As can be seen, formula (3) does not necessarily provide an integer index n for the target subrange with index m . Therefore, for convenience, when determining the target subband, two adjacent source subbands can be considered as described above (using formula (14)). In particular, this may be convenient for target subranges with index m , for which formula (3) provides a non-integer value of n . On the other hand, target subbands with index m , for which formula (3) provides an integer value of index n , can be determined from a single source subband with index n (using formula (5)). In other words, it is proposed that sufficiently high quality harmonic conversion can be achieved by using subband processing modules 603-3 and 603-4, which both utilize non-linear subband block processing with two input subbands, as described in the context of FIG. 3. In addition, control signal 104, if present, can simultaneously influence the operation of all three subband processing modules. Alternatively or in addition, if all three modules 503-2, 603-3, 603-4 use a non-zero geometric amplitude weighting parameter , then the transient response of the multiple converter can be improved compared to the case .

Fig. 7 illustrates an example of a transient response for time stretching by a factor of two based on a block of subbands. The top panel depicts the input signal, which is an attack of castanets sampled at 16 kHz. The system based on the structure of FIG. 1 is designed with a 64-band QMF analysis filter bank 101 and a 64-band QMF synthesis filter bank 103. The subband processing module 102 is configured to implement subband stretching with a factor of S = 2, without subband conversion ( Q = 1), and with a direct one-to-one mapping of source subbands to target subbands. The block analysis step is p =1, and the radius of the block size is R =7, that is, the block length L =15 discrete values of subranges, which corresponds to 15⋅64=960 discrete values in the signal domain (time domain). The window w is a power cosine window, i.e. cosine raised to the second power. Middle panel of Fig. 7 depicts the time stretch output when pure phase modification is applied by module 102, i.e. for nonlinear processing of blocks according to formula (5), the weighing parameter is used . The bottom panel shows the time stretching output when the geometric amplitude weighting parameter is used to nonlinearly process blocks using formula (5). . As can be seen, the transient response in the latter case is significantly improved. In particular, it can be seen that processing subbands using the weighting parameter leads to artifacts 701, which are significantly reduced (see reference numeral 702) when processing subbands using the weighting parameter .

Described herein is a method and system for HFR based on harmonic transformation and/or time stretching. The method and system can be implemented with significantly reduced computational complexity compared to traditional harmonic transform-based HFR, while still providing high-quality harmonic transform for both stationary and transient non-periodic signals. The harmonic transform-based HFR described uses nonlinear subband processing on a block basis. To adapt nonlinear subband processing to the signal type, such as a short-term non-periodic or non-transient signal, the use of signal-dependent control data is proposed. In addition, in order to improve the transient response of the harmonic transform using nonlinear block-based subband processing, the use of a geometric weighting parameter is proposed. Finally, for harmonic transform-based HFR, a low complexity method and system is described that uses a single pair of analysis/synthesis filter banks for harmonic transform and HFR processing. The described methods and systems can be used in various decoding devices, such as multimedia receivers, external video/audio devices, mobile devices, audio players, video players, etc.

The methods and systems for conversion and/or high frequency reconstruction and/or time stretching described herein may be implemented as software, firmware and/or hardware. Some components may be implemented, for example, as software running on a digital signal processor or microprocessor. Other components may be implemented, for example, as hardware or as special-purpose integrated circuits. Signals encountered in the described methods and systems may be stored on media such as random access memory or optical storage media. They can be transmitted over networks such as radio networks, satellite networks, wireless networks or wired networks such as the Internet. Typical devices using the methods and systems described herein are portable electronic devices or other consumer equipment that are used to store and/or reproduce audio signals. The methods and system can also be used in computer systems, such as Internet web servers, that store and make available audio signals, such as music signals, for downloading.

Claims (40)

1. An audio processing device, comprising a subband processing module configured to determine a synthesized subband signal from an analyzed subband signal; in this case, the signal of the analyzed sub-range includes a number of complex-valued analyzed discrete values at different times, each of which has a phase and amplitude; in this case, the signal of the analyzed subband is associated with the frequency band of the input audio signal; in this case, the subband processing module contains block extractor configured to re- frame extraction input discrete values from a number of complex-valued analyzed discrete values; and the frame length greater than one; And applying the input block step to a series of complex-valued sampled values before retrieving the next frame input discrete values; thus generating a set of frames input discrete values; a nonlinear frame processing module configured to determine a frame of discrete values to be processed, from a frame of input discrete values, by determining for each discrete value of the frame to be processed: the phase of the processed discrete value based on the phase of the corresponding input discrete value; and the phase of a predetermined input discrete value; And the amplitude of the processed discrete value based on the amplitude of the corresponding input discrete value; And an overlay and addition module configured to determine a synthesized subband signal by overlaying and adding discrete values from a set of frames of processed discrete values; in this case, the input step of the block is equal to one discrete value, and in this case, the signal of the synthesized subband is associated with the frequency band of the signal, stretched in time and/or converted in frequency relative to the input audio signal. 2. The audio processing device according to claim 1, characterized in that the block extractor is configured to downsample a number of complex-valued analyzed discrete values by a factor subrange conversions. 3. The audio processing apparatus of claim 1, wherein the block extractor is configured to interpolate two or more complex-valued sample values to extract the input sample value. 4. The audio processing device according to claim 1, characterized in that the nonlinear frame processing module is configured to determine the amplitude of the processed discrete value as the average value of the amplitude of the corresponding input discrete value and the amplitude of the predetermined input discrete value. 5. The audio processing device according to claim 4, characterized in that the nonlinear frame processing module is configured to determine the amplitude of the processed sample value as the geometric average of the amplitude of the corresponding input sample value and the amplitude of the predetermined input sample value. 6. An audio processing device according to claim 5, characterized in that the geometric mean value is defined as the amplitude of the corresponding input discrete value raised to a power , multiplied by the amplitude of a predetermined input discrete value raised to a power , while the geometric amplitude weighting parameter is . 7. The sound processing device according to claim 6, characterized in that the geometric amplitude weighing parameter is a function of the coefficient subrange and coefficient conversions sub-range stretching. 8. The sound processing device according to claim 7, characterized in that the geometric amplitude weighing parameter . 9. The audio processing device according to claim 1, characterized in that the nonlinear frame processing module (202) is configured to determine the phase of the processed sample value by shifting the phase of the corresponding input sample value by a phase shift amount that is based on a predetermined input sample value from frame of input discrete values, coefficient conversion and coefficient sub-range stretching. 10. The audio processing device of claim 9, wherein the phase offset value is based on a predetermined input sample value multiplied by . 11. The audio processing device according to claim 10, characterized in that the phase shift value is set by a predetermined input discrete value multiplied by , plus parameter phase correction. 12. Sound processing device according to claim 11, characterized in that the parameter phase correction has been determined experimentally for a number of input signals having certain acoustic properties. 13. The audio processing device according to claim 1, characterized in that the predetermined input sample value is the same for each processed sample value of the frame. 14. The audio processing device according to claim 1, characterized in that the predetermined input sample value is the central sample value of the frame of input samples. 15. An audio processing device according to claim 1, characterized in that the superposition and addition module applies a block step to the next frames of processed discrete values, and the block step is equal to the input block step multiplied by a coefficient sub-range stretching. 16. The audio processing apparatus of claim 1, wherein the subband processing module further comprises a windowing module that is located in front of the overlay and addition module, and is configured to apply a windowing function to the frame of samples being processed. 17. Sound processing device according to claim 1, characterized in that the subband processing module is configured to determine a number of synthesized subband signals from the number of analyzed subband signals; a number of signals of the analyzed subband are associated with a number of frequency bands of the input audio signal; And a number of synthesized subband signals are associated with a number of frequency bands of a signal stretched in time and/or frequency converted relative to the input audio signal. 18. A method performed by an audio processing device for generating a synthesis subband signal associated with a frequency band of a time-stretched and/or frequency-converted signal relative to an input audio signal, the method comprising: creating a signal of the analyzed subband associated with the frequency band of the input audio signal; in this case, the signal of the analyzed sub-range includes a number of complex-valued analyzed discrete values at different times, each of which has a phase and amplitude; frame extraction input discrete values from a number of complex-valued analyzed discrete values; and the frame length greater than one; applying the input block step to a series of complex-valued sampled values before retrieving the next frame input discrete values; thus generating a set of frames of input samples; determining the frame of processed discrete values from the frame of input discrete values by defining for each processed discrete value the frame: the phase of the processed discrete value based on the phase of the corresponding input discrete value; and the phase of a predetermined input discrete value; And the amplitude of the processed discrete value based on the amplitude of the corresponding input discrete value; And determining the signal of the synthesized subband by superimposing and adding discrete values of a set of frames of processed discrete values; in this case, the input step of the block is equal to one discrete value. 19. A non-volatile storage medium containing a program implemented in software, adapted for execution on a processor and for performing the steps of the method according to claim 18 when implemented on an audio processing device.