JP5894347B2 - System and method for reducing latency in a virtual base system based on a transformer - Google Patents

Description

<Cross-reference to related applications>
This application claims priority to US Provisional Patent Application No. 13 / 652,023, filed October 15, 2012, which is hereby incorporated by reference in its entirety.

<Technical field>
One or more embodiments relate to audio signal processing based on a transposer, and more particularly to reducing latency in a virtual-based synthesis system based on a transposer.

  Bass synthesis refers to a method of adding components to the low frequency range of a signal to improve the perceived base. Of these methods, sub-bass synthesis techniques are low frequency components below the existing portion of the signal to extend and improve the lowest frequency range present in the audio content of interest. To produce. Another method is to generate audible overtones from an inaudible bass range (for example, a low-pitch bass played through a small speaker), thereby improving the bass response by making the overtones and eventually the pitch audible. Use a virtual pitch algorithm.

  Virtual base synthesis is a virtual pitch method that increases the perceived level of base content in audio when played on a small speaker that cannot physically reproduce the lower end base frequency. This method is a psychoacoustic observation that the human auditory system can estimate a low pitch from the upper harmonics even when the fundamental and the first harmonic itself are missing. based on. The basic method of function is to analyze the base frequencies present in the audio and generate audible overtones that help perceive the missing low frequencies. The main feature of the virtual base is to improve the perceived base response in devices with small speakers by synthesizing overtones for frequencies below the device's low frequency roll-off (eg below 150Hz). It is. Inaudible signal components are transferred to higher audible frequencies (overtones) using multiple transfer factors, followed by energy adjustment. Virtual base synthesis may also increase the perceived base for headphone playback or playback on a full range speaker. FIG. 1A shows a frequency-amplitude spectrum of an audio signal having a non-audible range 10 of frequency components and an audible range of frequency components above the non-audible range. The harmonic transposition of the frequency component in the non-audible range 10 can produce a frequency component that is transferred in the portion 11 of the audible range, which reduces the perceived level of the base content of the audio signal being played back. Can be improved. Such harmonic transitions may include applying a plurality of transition factors to each significant frequency component of the input audio signal to generate a plurality of harmonics of the component.

In certain audio processing systems that utilize legacy virtual base systems, delays or latencies associated with frequency transfer functions may be excessive for certain applications. For example, a digital audio processing system with a 1025 sample latency may use a legacy virtual base system that adds an additional 3200 sample delay. This can cause a total delay in excess of 88 milliseconds given a sampling frequency (f _s ) of 48 kHz. This amount of latency is generally a problem and is even prohibitive for gaming and telecommunications applications. In such applications, a latency of about 100 milliseconds begins to become noticeable in terms of audible signal delay.

Traditional transformer systems used in legacy virtual base systems use symmetric time-domain windows for the time-to-frequency conversion and frequency-to-time conversion decomposition and synthesis stages, respectively. FIG. 1B graphically illustrates the delay imposed by a second order transformer, i.e., a second order harmonic generator. As shown in time plot 100, one center of the stylistic symmetric decomposition window is chosen as the time zero reference, and a new input sample 104 is generated from time t ₀ in decomposition phase 102 from the time stride of the decomposition window. Assuming S _A can be added. The time plot 110 shows the time stretch duality of the transformer. Here, t ₀ is stretched to 2t ₀ in the synthesis phase 112.

The total decomposition / synthesis chain delay D _ts for the exemplary process shown in FIG. 1B is given by Equation (1) below, where L is the transformer window size and S _A is the decomposition time stride or hop size: It can be expressed as

D _ts = L / 2 + 2 (L / 2−S _A ) = 3L / 2−2S _A (1)
In an audio processing system based on an HQMF (Hybrid Quadrature Mirror Filter) bank, the input signal to the CQMF (Complex Quadrature Mirror Filter) decomposition stage and the output signal from the CQMF synthesis stage Generally have the same sampling frequency f _s . Here, f _s is typically set to 44.1 or 48 kHz. The input signal sampling rate to the virtual base process may be f _s / 64. This is because the system typically processes the first CQMF signal only from a 64 channel CQMF bank. Note that CQMF sizes other than 64 channels can also be used. The transferred output from the legacy virtual base processing system has a sampling frequency of 2f _s / 64 due to the combined transfer function using a factor 2 base transposition factor. The result is a factor 2 bandwidth extension. In the combined transfer device, the basic transfer factor is a factor in which the source conversion bin (or frequency band) is mapped to the target conversion bin (or frequency band) in a one-to-one relationship. That is, no interpolation or decimation is involved in mapping bins from source to target. The fundamental transfer factor also dominates the relationship between the time stride of the decomposition window and the composition window. More specifically, the synthesis time stride is equal to the decomposition time stride multiplied by the basic transfer factor. The delay in the output samples from a 64 channel CQMF based system is L = 64 and S _A = 4
D _ts = {3L / 2−2S _A ) ・ 64/2 = 2816 samples (2)
It becomes.

  In addition to this delay, a delay from the Nyquist filter bank decomposition stage processing of the two virtual base output CQMF subband signals is added. This delay may be on the order of 384 samples, thus giving a total delay of 2816 + 384 = 3200 samples for this exemplary prior art legacy virtual-based processing system.

  One solution to the latency imposed by legacy virtual base systems is to change the actual processing circuit, such as a harmonic generator, for example by replacing the harmonic transformer with an alternative component . However, this can add significant cost and complexity to the system and can negatively impact audio quality.

  The subject matter discussed in the background section should not be assumed to be prior art, merely as a result of that reference in the background section. Similarly, problems mentioned in the background section or related to the subject matter of the background section should not be assumed to have been previously recognized in the prior art. The subject matter of the background section is merely representative of various approaches, and may itself be an invention.

  Embodiments include a latency reduction system in a virtual base processing system that performs harmonic transitions on low frequency components of an audio signal to produce transferred data indicative of harmonics. The harmonic transition process uses a fundamental transition factor greater than 2 and generates the harmonics in response to frequency domain values determined by transformation and inverse transformation stages using asymmetric decomposition and synthesis windows. An improved audio signal is generated by combining the virtual base signal with the delayed audio signal through the use of a Nyquist decomposition filter bank with a truncated prototype filter. In order to further reduce the latency caused by the harmonic transition process, a virtual base signal may be allowed to delay by a defined time period when combining the delayed audio signal with the audio signal. .

  Embodiments include a method for reducing latency in a virtual base processing system that performs harmonic transitions on low frequency components of an input audio signal to generate transferred data indicative of harmonics. Harmonic transitions use fundamental transition factors with integer values greater than 2. The process consists of a time-to-frequency domain transformation stage and subsequent inverse frequencies through the use of asymmetric decomposition and synthesis windows for time-to-frequency domain transformation and inverse frequency-to-time domain transformation. The harmonics are generated in response to frequency domain values determined by the time domain conversion stage. The input audio signal is a subbanded CQMF (complex-valued quadrature mirror filter) signal, and samples of the input audio signal may be preprocessed to produce critically sampled audio that exhibits low frequency components. Good.

  In one embodiment, the method processes the input audio signal through a decomposition filter bank or transform to provide a set of decomposition subband signals or frequency bins from the low frequency components, and uses a basic transfer factor B and a transfer factor T. A set of synthesized subband signals or frequency bins and processing the decomposed subband signals or frequency bins through a synthesis filter bank or transform to generate high frequency components from the set of synthesized subband signals. This represents a standard way of performing the transition. That is, forward FFT transformation is performed, followed by non-linear processing including transformation bin mapping, and then inverse FFT transformation. The method further generates a virtual base signal in response to the transferred data and applies the virtual base signal to the input audio signal by applying one or more decomposition filter banks to the virtual base audio output signal. Combining may include generating an enhanced audio signal. Here, the decomposition filter bank includes a truncated prototype filter from which a defined number of filter coefficients have been removed. The method further includes combining the input audio signal with the input audio signal by delaying the input audio signal by a predefined time period shorter than what the processing delay of the virtual base system would imply. Improved including a time-delayed virtual base processed subband sample combined with a delayed input subband sample, delaying the virtual base signal by a predefined time period It may include generating an audio signal.

  A basic transfer factor under some embodiments extends the input audio signal in the frequency domain by a degree proportional to the value of the basic transfer factor to produce a transferred audio signal, which is the basic transfer factor. The factor may be an even integer value between 4 and 16. In one embodiment, the decomposition filter bank acting on the transformer CQMF output subband comprises an 8-channel Nyquist filter bank and a 4-channel Nyquist filter bank, with a defined number of rejects. The prototype filter coefficients are 6 coefficients. In certain further embodiments, the input CQMF signal is routed directly from the preceding CQMF decomposition bank channel 0 output, thus bypassing subsequent Nyquist filter bank stages and thus avoiding the associated delay.

  Embodiments of the method are further oversampled in the frequency domain relative to the input audio signal by generating samples that are windowed and zero-padded at a defined sample frequency (using a decomposition time stride). Generating a low frequency component by performing the conversion. The predefined time period when combining the virtual base signal with the delayed input audio signal may be a value selected from the range of 0 samples to 1000 samples. This is because the virtual base signal can be allowed to delay the wideband input audio signal to 20 ms without perceptible degradation of the improved audio signal. In some embodiments, the asymmetric decomposition and synthesis window is configured such that a longer portion of the decomposition window is stretched toward past input samples and a longer portion of the synthesis window is stretched toward future output samples. The

In the drawings, like reference numerals are used to refer to like elements. The following drawings depict various examples, but one or more implementations are not limited to the examples depicted in the drawings.
FIG. 6 is a diagram illustrating frequency component transition from a non-audible frequency range to an audible frequency range in a known virtual base processing system. FIG. 3 illustrates the delay associated with a symmetric window used in a legacy virtual base system known in the prior art. 1 is a generalized block diagram of a virtual based processing system that implements a latency reduction process under an embodiment. FIG. FIG. 3 illustrates a pre-processing hybrid filter bank stage in a system based on HQMF under an embodiment. FIG. 6 illustrates a preceding Nyquist synthesis filter bank stage of a virtual base processing system under an embodiment. FIG. 3 is a more detailed view of the virtual base processing system shown in FIG. 2 under an embodiment. FIG. 3 is a block diagram of the main functional components utilized by the virtual base latency reduction process and system under an embodiment. FIG. 7 is a table illustrating first hop size related delays for a virtual base latency reduction system using various orders of basic transposable elements, under an embodiment. FIG. 7 is a table illustrating second hop size related delays for a virtual base latency reduction system using various orders of basic transposable elements, under an embodiment. FIG. 5 is an exemplary plot of the time response of an asymmetric window compared to some legacy symmetric window. FIG. 4 is an exemplary plot of the frequency response of an asymmetric window compared to some legacy symmetric window. FIG. 6 illustrates the use of asymmetric windows and associated delays imposed by a Bth order fundamental transition, under an embodiment. A is the total latency value for the first hop size for a virtual base latency reduction system using an asymmetric transformation window and various orders of the fundamental transfer factor, under an embodiment It is a table | surface which shows. B is the total latency value for the second hop size for a virtual base latency reduction system using an asymmetric transformation window and various orders of the fundamental transfer factor under certain embodiments It is a table | surface which shows. 1 is a block diagram illustrating an audio processing system that includes a virtual base generation system and a latency reduction system, under an embodiment. FIG.

  Embodiments of systems and methods are described for reducing latency and algorithmic delays in a virtual-based system based on a transformer. Such systems and methods are time delayed with respect to higher order fundamental transfer factors, low latency asymmetric transformation windows, truncated Nyquist prototype filters, and original audio signals. ) Utilize the virtual base signal and the bypassed Nyquist decomposition filter bank in the preceding hybrid filter bank stage.

  Throughout this disclosure, including the claims, the expression performing an operation on a signal or data (eg, filtering, scaling, transforming or applying gain to the signal or data) is broadly defined as: Directly on the signal or data or on a processed version of the signal or data (eg, on a version of the signal that has undergone preliminary filtering or preprocessing prior to performing the operation) Used to represent performing the operation. The term “transposer” broadly refers to an algorithm unit or device that performs pitch shifting or time stretching of a real or complex valued input signal over part or all of the available input signal spectrum. Used to represent. The expressions "translator", "harmonic transformer", "phase vocoder", "high frequency generator" or "harmonic generator" may be used interchangeably. The expression “system” is used in a broad sense to denote a device, system, or subsystem. For example, a subsystem that implements a decoder may be referred to as a decoder system and includes such a subsystem (eg, a system that generates X output signals in response to multiple inputs, where: The subsystem generates M of the inputs and the remaining X-M inputs are received from an external source) may also be referred to as a decoder system. The term “processor” is broadly programmable or otherwise configurable (eg, using software or firmware) to perform operations on data (eg, audio or video or other image data). Used to represent a system or device. An example processor is programmed to perform pipelined processing on a field programmable gate array (or other configurable integrated circuit or chipset), audio or other sound data. And / or other configured digital signal processors, programmable general purpose processors or computers and programmable microprocessor chips or chipsets. The expressions “audio processor” and “audio processing unit” are used interchangeably and in a broad sense to refer to a system configured to process audio data. Examples of audio processing units include, but are not limited to, encoders (eg, transcoders), decoders, vocoders, codecs, pre-processing systems, post-processing systems, and bitstream processing systems (sometimes referred to as bitstream processing tools). Absent.

  Embodiments are directed to systems and methods that reduce virtual base delay without requiring substantial changes to existing virtual base processing components, such as harmonic transformers used in virtual base processing systems. Aspects of the virtual base latency reduction system and method may be used in the context of a harmonic generator (transformer) in an audio codec (eg, in a decoder). Aspects of the virtual base latency reduction system and method are in the context of other transformers or phase vocoder systems, such as traditional phase vocoders used for general time stretching or pitch shifting of audio signals. May be used.

  As shown generally in FIG. 1A, the virtual base generation method using harmonic transitions is based on the bass content in limited playback facilities, such as through small speakers that cannot physically reproduce the missing low frequencies. In order to improve reproduction, it involves the transfer of frequency components from the non-audible frequency range to the audible frequency range. Embodiments of a virtual base latency reduction system and method perform harmonic transitions on low frequency components of an audio signal to produce transferred data that indicates harmonics that are expected to be audible during playback An improved audio signal that improves upon the virtual base generation method, generates a virtual base signal in response to the transferred data, and combines the virtual base signal with the (delayed) input audio signal. Is generated. Typically, the enhanced audio signal is perceived by the one or more speakers that cannot physically reproduce the low frequency components during the playback of the enhanced audio signal. Give a level.

  The harmonic transitions performed by the virtual base generation method include a second order transformer and at least one higher order transformer (typically third and fourth order, optionally at least one) for each low frequency component. Combined transitions are used to generate harmonics using two additional higher order transformers, so that all of the harmonics are common, time to frequency domain conversion stages (eg, , By performing phase multiplication or other phase operations on the frequency coefficients resulting from a single time-to-frequency domain transformation, followed by a common frequency-to-time domain transformation (in practice, The common frequency to time domain transform is divided into two smaller transforms to accommodate the subband bandwidth and sampling frequency of the CQMF framework). Generated in response to a defined frequency domain value.

  FIG. 2 is a block diagram of a virtual-based processing system that implements or is used in connection with certain latency reduction processes under certain embodiments. In one embodiment, the virtual base processing system 200 takes as input 201 (input A) a plurality of complex-valued subband samples (HQMF samples) from a so-called hybrid filter bank. In one embodiment, a hybrid filter bank preceding the virtual base process splits the original time domain audio input signal into such multiple hybrid subbands 201 (which will be described in more detail below). Those subbands may be buffered by the input buffer 206. The buffered input is then synthesized to reconstruct a single complex-valued QMF (CQMF) domain signal 202 (signal C) that represents low frequency audio content (eg, between 0 and 375 Hz). Processed by the Nyquist synthesis filter bank 208 that performs the function. In another embodiment, the virtual base system includes a latency saving mechanism by bypassing the Nyquist decomposition filter bank stage in the preceding hybrid filter bank. This allows the system to save the delay (eg, 384 samples) associated with the Nyquist decomposition bank by feeding the CQMF channel 0 signal directly to the virtual base module as input 203 (input B). As shown in FIG. 2, one of the two inputs 202 or 203 is selected by a switch, such as a selector 204, and the selected signal is a virtual base input signal 205 (signal D) that is further processed by a transition 209. )including.

  A transition (or phase vocoder) generally consists of a time-to-frequency conversion or filter bank followed by a non-linear stage (performing phase multiplication or phase shifting) followed by a frequency-to-time conversion or filter bank. It is a combination. Thus, as shown in FIG. 2, the transition 209 includes a time to frequency conversion component 210, a non-linear stage 212 and a frequency to time conversion 214. Non-linear stage 212 in transition 209 is a processing block that modifies the phase and applies some gain (amplitude) control signal to the subbands or transform components of the signal. The transferred signal is then buffered by output buffer 216 and then processed by Nyquist decomposition filter bank 218. The Nyquist decomposition filter bank 218 performs a decomposition function that decomposes the virtual base output CQMF signal into subbands corresponding to the hybrid subband samples (HQMF) of the input signal 201. A delayed, unprocessed version of input A signal 220 is mixed with the Nyquist filter bank 218 output to produce an enhanced audio output signal 222 that includes the virtual base output signal plus the delayed input signal. To do.

  Embodiments may be directed to the use of Nyquist filter banks for certain functions, such as synthesis 208 and decomposition 218 stage processing, although other types of filter banks or frequency split or partition circuits and techniques may be used. May be used. In other embodiments, the filter bank or frequency split or partition circuit and technique described above may not be present at all.

  3A-3C are more detailed views of the virtual base processing system shown in FIG. FIG. 3A shows a pre-processing hybrid filter bank stage 300, i.e., a stage preceding it, typically not part of a virtual base system. A hybrid filter bank is a combination of CQMF banks where a certain number of the lowest CQMF bands are processed by various Nyquist filter banks of a predetermined size to increase the frequency resolution in the low frequency range. Good. The combination of low frequency subband samples from the Nyquist decomposition stages and the remaining CQMF channels is referred to as a hybrid subband sample or HQMF (hybrid QMF) signal. As shown in FIG. 3A, a time domain input signal 302 is input to a 64-channel CQMF decomposition filter bank 304. In one embodiment, one output of this filter bank, CQMF channel 0 (denoted as signal B) 306 is fed directly to the virtual base module 330 of FIG. 3C (this signal is input to input B 203 of FIG. 2). Corresponding). It should be noted that signal B 306 bypasses Nyquist decomposition filter bank 307, thus avoiding the associated delay. Also, CQMF channels 0, 1, 2 are input to several Nyquist decomposition filter banks 307-309. The output from the Nyquist decomposition filter bank and the remaining CQMF subbands (3 to 63) produce hybrid subband samples 0-76 (denoted as signal A) 310.

  A plurality of complex-valued hybrid subband samples (sample A) 322 are input to Nyquist synthesis filter bank stage 324, as shown in system 320 of FIG. 3B. The virtual base module 330 of FIG. 3C is assumed to be one of the other modules in the system that operates on hybrid subband samples (HQMF samples). Thus, signal A 310 in FIG. 3A may be processed by other modules after pre-processing filter bank stage 300 before becoming input A 322 in FIG. 3B. In one exemplary embodiment, the first eight hybrid subbands, ie, subbands from the low frequency 8-channel Nyquist filter bank 307 (which depends on the sampling rate, Which produces a signal bandwidth of approximately 344-375 Hz). Since the Nyquist filter bank is not downsampled in contrast to the CQMF bank, the Nyquist filter bank synthesis step is particularly straightforward because it is simply the sum of the subband samples for each CQMF (or HQMF) time slot. After summing the eight lowest hybrid subband samples in stage 324, the system has reconstructed the CQMF channel 0 signal C 326, which becomes the input 332 to the virtual base module 330 of FIG. 3C. .

  FIG. 3C illustrates a virtual base system that implements or is used in connection with some kind of latency reduction process under an embodiment. The virtual base module 330 of FIG. 3C has the signal D 332 as an input. In certain embodiments where the preceding Nyquist decomposition filter bank 307 is bypassed, signal D 332 may be routed from signal B 306 of FIG. 3A. In another embodiment, signal D 332 may be fed from signal C 326 of Nyquist synthesis stage 320 of FIG. 3B. In either embodiment, signal D 332, ie, the input signal to the virtual base module, is a single complex-valued CQMF signal (eg, the first channel (channel 0) from a set of CQMF subband signals). It is.

  For virtual base applications, an optional dynamics processing function may be performed by the dynamics processor 336 to change the dynamics of the virtual base input signal. The processor 336 may be used to reduce the level of the weak base and maintain or increase the strong base. That is, it may be used as an expander. This scheme matches the shape of an Equal Loudness Contour (ELC) at the base range. The loudness curve is flatter in frequency for signals with greater loudness and steeper for signals with weaker loudness. Thus, when generating harmonics, a weaker base can be attenuated more than a stronger base in order to maintain the relative loudness between the fundamental component and the generated harmonics. The gain of the dynamics processor 336 may be controlled by a moving average energy signal, for example a downmixed (mono) version of the moving average energy of the first CQMF band signal 332.

  For an embodiment of system 330, a first windowing function 338, forward FFT 340, and modulation function 342 using window size L (including zero padding to length N) is prior to input to nonlinear processing block 344 ( Performed on CQMF signal (possibly dynamics). In some embodiments of the invention, the window shape is asymmetric. In another embodiment, the transformer (having components 338-356) represents an improved phase vocoder that uses an interpolation technique referred to as "combined transposition". This produces second, third, fourth and possibly higher harmonics (transition factors) using the same FFT decomposition / synthesis chain as for the basic transformer. In general, such a combined transition saves computation, although the quality of harmonics other than the fundamental order harmonics may be somewhat impaired. Without combined transfer, at least either forward or reverse conversion needs to be distinct for different transfer factors. Non-linear processing block 344 uses integer transfer factors. This makes some phase estimation, phase restoration or phase locking techniques extraneous and inaccurate, which is generally unstable and inaccurate as used in many standard phase vocoders. In some embodiments, phase multiplier 344 uses a base transfer factor B higher than 2, such as 8 or any other suitable value.

  Transformers 338-356 use oversampling in the frequency domain (ie, zero padded decomposition and synthesis windows in blocks 338 and 356) to improve impulse (impact) sound. This is outstanding when used in the base frequency range. Without such oversampling, impulsive drum sounds are likely to generate at least some pre- and post-echo artifacts, making the base blurry and obscure. In some embodiments, the oversampling factor F is selected to be at least the factor F = (B + 1) / 2. Here, B is a basic transposable factor (for example, B = 8). This helps to ensure that the front and back echoes are suppressed for isolated transients.

  As shown in FIG. 3C, the transition includes a phase multiplier circuit (non-linear processing block 344) followed by gain and slope compensation per FFT bin applied by amplifier 346. This allows the overall gain for the various transposable factors to be set independently. For example, gains can be set to approximate a certain equal loudness curve (ELC). As an approximation, the ELC can be well modeled by a straight line on a logarithmic scale for frequencies below 400 Hz. In this case, the odd order harmonics can be attenuated to a greater degree. Odd-order harmonics (eg, third-order, fifth-order, etc.) are important for the resulting virtual base effect, but can sometimes feel more harsh than even-order harmonics. Each transferred signal may further have a slope gain, ie a roll-off decay factor. This is measured, for example, in dB per octave. This attenuation is also applied by the amplifier 346 for each bin in the transform domain.

In a non-hybrid filter bank based system, such as a time domain system, taking the signal 302 of FIG. 3A as an input, the transformers 338-356 convert the time domain signal to a full sampling rate (eg, 44.1 or 48 kHz). An FFT size of approximately 4096 lines would then be used to work directly against and then give sufficient resolution in the low frequency (base) range. In some embodiments, however, all processing is performed on CQMF channel 0 subband samples (signal D 332 of system 330). This is a normal process, such as saving computational complexity by processing only the signal of interest at the transformer, that is, by processing the critically sampled (or maximally decimated) low-pass signal. Offers certain advantages over cases. For example, by using a fourth order fundamental transition, the virtual base system expands the bandwidth of the input signal by a factor of four. In general, virtual base systems are not required to output signals with bandwidths above approximately 500 Hz. This is because the first CQMF channel (channel 0) with a bandwidth of 375 Hz (for f _s = 48 kHz) is more than enough for the virtual base input, and the first two CQMF channels (channels 0 and 1) It means having sufficient bandwidth for the virtual base output (750 Hz at f _s = 48 kHz). Having CQMF channel 0 as input allows the system to process complex-valued samples using an FFT transform of size 64 (4096/64) instead of 4096. Here, the 64-fold reduction comes from the CQMF bank downsampling factor, which is also equal to the reduced bandwidth of the first CQMF subband signal compared to the time domain input signal. Because of the inherent bandwidth expansion, the output from the transformer needs to be converted to CQMF bands 0 and 1. This divides a 64-line FFT into four 16-line FFTs and performs CQMF prototype filter response compensation in the transform domain before the inverse FFT of the two 16-line FFTs that make up CQMF bands 0 and 1 is calculated. It may be performed approximately by using. Note that in the above example, frequency domain oversampling is not considered. This is because the size of the forward and inverse transforms is increased by the excessive sampling factor described above. In some applications, the FFT spectrum may be split in module 348 of virtual base module 330 and CQMF filter response compensation may be performed by multiplier 350. In other embodiments, CQMF filter response compensation may be performed on the full (eg, 64 lines in the above example) FFT spectrum before the FFT splitting module 348.

  As further shown in FIG. 3C, the output from the CQMF filter response compensation block 350 is input to a modulation step 352, which uses an inverse FFT circuit 354 that uses the N / B point transform size and subsequent window length L / B. A windowing and duplication / addition step 356 follows. In some embodiments of the invention, the window shape is asymmetric. Modulation step 352 may be applied before FFT division 348 and CQMF filter response compensation 350 block. The output signal from the windowing and overlap / add circuit 356 is two CQMF signals including a virtual base signal to be mixed with the delayed HQMF signal A 364. However, any signal must first be filtered through the 8- and 4-channel Nyquist decomposition filter bank 360, respectively, in order to fit in the hybrid domain. In one embodiment of the invention, Nyquist decomposition filter bank 360 uses a truncated prototype filter. The HQMF output from filter bank 360 may be bandpass filtered and mixed with delayed input component A 364 at module 362 to produce an enhanced audio output HQMF signal 366. In one embodiment, the delay of input A 364 to hybrid band mixing block 362 is the virtual base system delay (if signal B 306 is used as an input) to form a temporally delayed virtual base signal. Less the Nyquist decomposition delay).

  The phase relationship between the subband signals from the CQMF decomposition bank is not maintained when performing the FFT split as outlined above. To alleviate this, in one embodiment, system 330 uses phase compensation by exp (−jπ / 2) multiplication 358 for CQMF channel 1 prior to Nyquist decomposition block 360. The specific argument to the phase compensation function 358 depends on the modulation scheme used by the preceding CQMF bank 304 of FIG. 3A and may vary between embodiments. Also, the compensation factor 358 may be transferred to another processing block and absorbed.

<Virtual base latency reduction>
As mentioned in the background section, the virtual base processing system introduces some delay when processing the input signal. Referring to Figure 1B, the delay of the legacy transition unit (measured on metastatic output sampling frequency) can be expressed as D = 3L / 2-2S _A. Where L is the transformer window size and S _A is the decomposition stride or hop size. In a system where L = 64 and S _A = 4, the total delay of the transformer and Nyquist filterbank decomposition stage can be on the order of 3200 samples as described above.

  In some embodiments, the virtual base processing system includes components that perform certain steps to reduce latency associated with virtual base processed content. FIG. 4 is a block diagram of the major functional components utilized by the virtual base latency mitigation process and system, under an embodiment. As shown in the drawing 400 of FIG. 4, the latency mitigation process is delayed in time by a higher order fundamental transfer factor 402, a low latency asymmetric transformation window 404, a truncated Nyquist prototype filter 406, and time. Use of the virtual base signal 408. Each of the functional components of drawing 400 may be used alone or in conjunction with one or more of the other components to help reduce the latency of virtual-based content. It may be broken. Drawing 400 may represent a system when each of components 402-408 is embodied as a hardware component such as a circuit, processor, or the like. The drawing also represents a process such as when each of components 402-408 is implemented as a step performed by a functional component, such as a computer-implemented process executed by one or more processors. May be. Alternatively, drawing 400 may represent a hybrid system and method in which certain components may be implemented in hardware circuitry and as method steps in which other components may be implemented. Components 402-408 may be implemented as separate stand-alone components or may be combined in one or more integrated latency reduction functions. Details of the composition and operation of each component of the system 400 are described below.

<Higher order basic transfer factor>
For the higher order fundamental transposable element 402 of FIG. 4, the legacy transposer delay equation D _ts = {3L / 2−2S _A } · 64/2 (equation (2)) is as shown in equation (3): Can be derived.

D _ts = {(B + 1) L / 2−B ・ S _A } ・ 64 / B (3)
In equation (3), the base transition factor 2 of the legacy system is replaced by an arbitrary number of base transition factors B. Note that equation (3) refers to the delay in the output samples of the CQMF based framework with 64 channels. For certain L and S _A, B increases the delay can verify that the decrease. FIG. 5A is a table illustrating the delay associated with the first hop size, and FIG. 5B illustrates the delay associated with the second hop size for the virtual base latency mitigation system under an embodiment. It is a table | surface which shows. Table 1 of FIG. 5A shows the latency for a hop size of S _A = 4 for various window sizes (L = 16 to 128) and basic transition factors (B = 2 to 16). In contrast, Table 2 of FIG. 5B shows the latency for a hop size of S _A = 2 for the same various window sizes (L = 16 to 128) and basic transition factors (B = 2 to 16). Show. As can be seen in FIGS. 5A and 5B, significant latency mitigation can be achieved by increasing the base transposition factor, eg, from 2 to 8, (eg, from 2816 to 2048 samples for the nominal case of L = 64 and S _A = 4). .

  Referring to FIG. 3C, in the combined transformers 338-356, when T is greater than B (T> B), when generating a higher order transposable factor T, the transformer source range in the decomposition transform spectrum Is less than the target range of the transformer. The target bin results from interpolation of the source bin. When using a higher order fundamental transformer to produce a lower order transposable factor, ie when T is less than B (T <B), the source range will be greater than the target range and the target bin will be decimation As a result. However, even for T <B, when T is odd, the source bin index derived as k = nB / T is generally not an integer, assuming that n is the target bin index, Thus, the target bin is derived from the interpolation of two consecutive source bins.

The increased order of the basic transposable element has some implications for the virtual base process. First, control needs to be established to force the transformer source range to remain within the resolved conversion range (ie, within the range of 0 to N−1). Secondly, compared to the system using basic transposable factor 2, the two composite transformations 354 are now N / B in size rather than N / 2. Here, N is a decomposition transformation size. This means that the synthesis window is thinned by B times instead of by 2, and the spectral split 348 is correspondingly downscaled with the gain vector for the filter response compensation 350. This is a result of increased bandwidth expansion for larger values of B; the transformer output inherently covers the frequency range of B CQMF bands (assuming one CQMF band input). . Here, only the first two are actually combined, thereby saving complexity. The basic transposable element B = 8 and the frequency domain oversampling factor F = 4, two synthetic transformation size is _{N S = F · L / B} = 4 · 64/8 = 32, the synthetic transformation window 356 only L / B = 64/8 = 32 taps.

The quality of the transferred signal is governed by the fundamental transfer factor and is reduced for higher order transfer orders, but can be improved by using a reduced resolution hop size (increased oversampling in the time domain). . In addition, the order of frequency domain oversampling needs to be increased for higher fundamental transfer factors in order to maintain quality for impact sounds (transient sounds). However, increased oversampling in both time and frequency can add to the computational complexity of the transformer. In some embodiments, the decomposition hop size is reduced by a factor 2 compared to the legacy system. A basic transformer with a factor B = 8 requires a frequency domain oversampling factor of at least F = (B + 1) /2=4.5. In one embodiment, the system uses oversampling of factor 4 (F = 4), and a missing value of 0.5 is generally negligible in practice. This is because the conversion window is gradually reduced at the end. Thus, in this embodiment, the computational complexity is increased by a factor 2 in total, resulting from increased oversampling in time. Increased time oversampling also comes at the cost of slightly increased delay, resulting in a total latency of 2176 for L = 64, B = 8 and S _A = 2 as shown in Table 2 of FIG. 5B. Should be noted.

<Asymmetric conversion window>
Given what is shown in Tables 1 and 2 of FIGS. 5A and 5B, an obvious way to reduce the transformer delay is to use a shorter transform window, and thus a smaller decomposition and synthesis transform size. You might think. However, this generally comes at the cost of reduced quality for dense tone-like signals. This is due to the reduced frequency resolution resulting from the shorter conversion window. It has been found that by using asymmetric decomposition and synthesis windows in the forward and inverse transform stages, a more robust reduction in the algorithm delay of the transferer can be achieved. Thus, with respect to the low latency asymmetric transformation 404 of FIG. 4, in one embodiment, the latency reduction system uses asymmetric decomposition and synthesis windows in the forward and inverse transformation stages (eg, windowing stages 338 and 356, respectively, FIG. 3C). . This inherently improves the frequency response of a limited length symmetric window by extending the “tail” of the window toward the historical samples that do not contribute to the conversion delay. In a more general embodiment, the length of the decomposition window and the size of the forward transform may be different from those of the synthesis window and the inverse transform.

  FIG. 5C is an exemplary plot of the time response of an asymmetric window compared to a legacy symmetric Hanning window. FIG. 5C shows the time response as a function of sample (x axis) versus signal amplitude (eg, in volts) for a length 64 Hanning window shown as plot 514 and a length 41 Hanning window shown as plot 516. A time response plot 512 for an asymmetric window of length 64 and delay 40 (a delay equal to a Hanning window of length 41) is shown. FIG. 5D is an exemplary plot of the frequency response of an asymmetric window compared to a legacy symmetric Hanning window. FIG. 5D shows the normalized frequency (x-axis) for signal amplitude (eg, in dB) on a logarithmic scale for a length 64 Hanning window shown as plot 524 and a length 41 Hanning window shown as plot 526. ) As a function of frequency response plot 522 for a length 64 and delay 40 asymmetric window (equivalent to a length 41 Hanning window). As can be seen in FIG. 5D, the main lobe of the asymmetric window has a width intermediate that of the symmetric Hanning window. This indicates an intermediate frequency resolution or selectivity between the two Hanning windows.

In order to accept the asymmetric window transformation process, the transferor algorithm needs to be partially modified compared to the legacy implementation, taking into account the reduced transformation delay D of the decomposition / synthesis chain. Instead of frequency modulation with e- ^iπk after forward conversion of legacy systems and before inverse conversion, asymmetric systems require frequency modulation 342 after decomposition conversion:
M _A (k) ＝ e ^{-i (2π / N) (D / 2-L + 1) k} 0 ≦ k <N (4)
The system also requires prior modulation of the composite FFT spectrum:
M _S (k) ＝ e ^{-i (π / N ・ D ・ n)} 0 ≦ n <N (5)
In equations (4) and (5) above, k and n are the transform frequency coefficient indices, respectively, N is the decomposition transform size, i.e. N = FL, where F is the frequency domain oversampling factor, L is the resolution window size and D is the conversion delay. As shown in FIG. 3C, the modulation of equation (5) may also be applied in the modulation stage 352 after the FFT division module 348 and the response compensation step 350.

FIG. 6 illustrates in style the use of an asymmetric window and the associated delay imposed by a B order fundamental transition, under an embodiment. In legacy virtual base systems, B is typically set to 2, but if the asymmetric window process 404 is used in the context of a higher order fundamental transfer factor process 402, B will be an integer value greater than 2. (For example, B = 4, 8, or 16). Time plot 600 shows a time zero reference as a group delay (approximately D / 2) of the decomposition window. In the decomposition phase 602, new samples 604 is applied from time t _0. Time plot 610 shows that the time-stretching duplex of the transformer shifts t ₀ to B · t ₀ for the new time-stretched sample 614 in synthesis phase 612. When an asymmetric window such as that shown in FIG. 5 (512) or FIG. 6 is used, the total resolution / synthesis chain delay is approximately D / 2 + B (D / 2−S _A ).

  For the case of a symmetric window where frequency domain modulation can be implemented by a cyclic time shift with N / 2 samples, the calculations in equations (4) and (5) above are similarly N- (D / 2-(L-1)) (mod N) samples and may be implemented by a cyclic time shift of ND / 2 samples after a (single) composite transformation. However, when combining an asymmetric window with a higher order fundamental transition factor, eg, B = 8 and FFT partition stage 348, the time shift after the combined transform is (N−D / 2) / B samples. This may not be an integer value. In this case, a rounded value may be used as an approximation. Furthermore, in order to save computational complexity, the decomposed modulation may be combined with the combined modulation as a combined combined modulation as given by equation (6).

M _ASC (k) ＝ e ^{-i (2π / N) (D / 2 ・ (B + 1) -L + 1) ・ B) ・ k} 0 ≦ k <N (6)
The combined modulation of equation (6) is exact only when the transfer factor T is equal to B. For other transfer factors, equation (6) is also an approximation.

  Alternatively, the modulation of equation (6) may be implemented as a combined cyclic time shift after composite transformation, as shown in equation (7).

に等しい。 f _x (m) = g _x (S + m) 0 ≦ m <N / B−S
f _x (N / B−S + m) = g _x (m) 0 ≦ m <S (7)
In equation (7) above, g _x (m) is the time domain output from one of the composite inverses, f _x (m) is the shifted time sequence, and S is:

be equivalent to.

  Again, equation (7) may be an approximation of the frequency modulation implemented by equation (6) itself, if the ceiling function (round to the nearest integer) argument is not a strict integer. It only gives an approximation of It should also be noted that the above equation (5) or (6) is preferably applied only to a limited part of the coefficients included in the two inverse Fourier transforms.

  Referring to FIG. 6, the exact expression for the total system delay of the asymmetric window transition framework is as shown in equation (8).

D _ta = {(B + 1) ・ D / 2−B (S _A −1)} ・ 64 / B (8)
Again, equation (8) refers to the delay in the output samples using a 64 channel CQMF based framework.

For a virtual base latency reduction system using an asymmetric transformation window under an embodiment, FIG. 7A is a table showing the total latency values for the first hop size, FIG. B is a table showing total latency values for the second hop size. Table 3 of FIG. 7A shows the latency for hop size S _A = 4 for various conversion delay values (D = 15 to 127) and basic transfer factors (B = 2 to 16). In comparison, Table 4 of FIG. 7B shows the latency for the hop size S _A = 2 for the same various conversion delay values (D = 15 to 127) and basic transition factors (B = 2 to 16). Show. As can be seen in Table 4, the latency reduction in transitioning from a symmetric 64-tap window (D = 63) to an asymmetric window is 828 samples (for the nominal case where S _A = 2 and B = 8) , 2204-1376 = 828).

Comparing Eq. (3) and Eq. (8), D _ts = D _ta
D = L- (2B / (B + 1)) (9)
Can be verified. Equation (9) above represents the expected conversion delay of D = L−1 for the symmetric window when B = 1.

となる。これは、非対称フィルタの設計のための最適化フェーズの間に、群遅延についての制約条件を含めることによって達成されてもよい。 The amount of asymmetry in the transition window can vary depending on system constraints and requirements. In certain embodiments and individual implementations, the group delay of the asymmetric window is selected to be close to half the conversion delay in order to maintain sufficient transition quality. Thus, in this case,

It becomes. This may be achieved by including a constraint on group delay during the optimization phase for the design of the asymmetric filter.

<Canceled Nyquist prototype filter>
Referring to FIG. 4, a third latency mitigation element involves using a truncated Nyquist prototype filter 406. As shown in FIG. 3C, an 8-channel and 4-channel Nyquist decomposition filter bank 360 is applied to the virtual base output CQMF channel in order to be able to mix the virtual base signal in the hybrid domain (these filter banks are shown in FIG. 3A). Corresponding to Nyquist filter banks 307 and 308). In one embodiment, the Nyquist decomposition filter bank 360 uses a symmetric 13-tap prototype filter. This can result in a delay of 6 CQMF samples (eg, 6 · 64 = 384 output samples in this case). By removing the six coefficients of the prototype filter that affect future samples, this entire delay (eg, 384 samples) can be eliminated. In general, the Nyquist teardown / synthesis chain still provides perfect reconstruction. However, the frequency response of Nyquist filter banks that use truncated filters can vary. Optimization of the remaining filter coefficients may improve the potentially poorer frequency response of the Nyquist filter bank that uses truncated filters.

<Virtual base signal delayed in time>
Referring to FIG. 4, the fourth latency mitigation element includes lagging the virtual base signal from the original signal 408. In this case, the wideband signal (ie, hybrid signal A 364 in FIG. 3C) is delayed for a shorter period of time than the virtual base system delay actually implies, so the overall system latency. Can be shortened. A short listening test shows that a lag of less than 20 ms does not interfere with the virtual base effect. This delay corresponds to 960 samples for a 49 kHz audio signal.

  In one particular implementation of an embodiment, the virtual base signal is allowed to delay the wideband signal by a total of 352 samples (7.33 ms at 48 kHz). Of these 352 samples, 32 samples are derived from asymmetric conversion windows. This is because 1376 is not divisible by 64 of the CQMF filter bank size. Thus, the delay from the asymmetric window transformation can be divided into 1344 wideband latency +32 sample delay. The additional delay added to these 32 samples is 320 samples (5 CQMF samples, corresponding to 6.67 ms at 48 kHz sampling frequency).

The various latency mitigation elements 402-408 of FIG. 4 can be used in any practical number combination to achieve virtual base system latency mitigation. Further, the appropriate variable for each latency mitigation method may be modified to increase latency in connection with some perceived decrease in virtual base signal quality. In one embodiment, the four latency mitigation elements were implemented using the following values: basic transfer factor B = 8, hop size S _A = 2, conversion delay D = 40, truncated Nyquist filter, and Additional virtual base delay of 320 samples. In this exemplary case, the resulting virtual base system delay in the output samples was:

D _VB = {(B + 1) ・ D / 2−B ・ (S _A −1)} ・ 64 / B−32 + 0−320 = 1376−352 = 1024
Avoiding the Nyquist decomposition filter bank in the preprocessing stage as described above (eg, using input B 203 in FIG. 2 and signal B 306 in FIG. 3A as input D 332 of virtual base module 330 in FIG. 3C) An additional 384 samples of delay can be saved. The result is a virtual base system delay of 1024−384 = 640 samples (corresponding to 13 ms at a sampling frequency of 48 kHz).

The delay of 640 samples in this exemplary case is significantly less than the nominal delay of 3200 samples in the legacy virtual base system described above. This delay is designed by adding a further virtual base delay, increasing the hop size S _A to 4 instead of 2, or designing an asymmetric transformation window with a resulting decomposition / synthesis delay shorter than 40 This can be further shortened. However, any such value change can lead to slightly poorer virtual base quality, although the latency can be further reduced.

  The virtual base latency mitigation embodiments described herein may be used in connection with any suitable virtual base generation system as shown in FIGS. FIG. 8 is a block diagram illustrating an audio processing system including a virtual base generation system and a latency mitigation system, under an embodiment. As shown in FIG. 8, the system 800 has a virtual base system 330 as shown in FIG. 3C. Virtual base system 330 receives an input audio signal 801 and performs some kind of frequency transition function to enhance audio for playback through speaker 806, which may be of a limited frequency response function.・ Generate content. Certain latencies may be associated with the transfer function performed by the virtual base system 330. In some embodiments, the virtual base latency mitigation system 400 (as shown in FIG. 4) is provided as a post-process to the virtual base system 330 to mitigate latency associated with virtual base processing. The reduced latency audio signals from the virtual base systems 330 and 400 are then sent to the rendering subsystem 802. The rendering subsystem 802 is configured to generate a speaker feed that can be fed through an amplifier 804 for left and right (or multi-channel) speakers 806.

  Although the virtual base latency mitigation system 400 is shown to be a separate post-process element in the system 800, such a latency mitigation system is part of the virtual base system 330 (as previously indicated). It should be noted that it may be implemented as, or as part of any other suitable element of system 800, such as a functional component within rendering subsystem 802. Similarly, virtual base system 330 may be the legacy virtual base generation system outlined in the background, or input audio to increase the perceived level of base content for playback through speaker 806. There may be any other virtual base generation and processing system that uses harmonic transitions to improve signal 801.

  Embodiments of the virtual base latency mitigation system can be used in any audio processing system that renders and plays digital audio through a variety of different playback devices and audio speakers (transducers). These speakers may be embodied in any of a variety of different listening devices or playback equipment items, such as computers, televisions, stereo systems (home or movie theaters), cell phones, tablets and other portable playback devices. Good. The speakers may be of any suitable size and power rating, and may be provided in the form of free-standing drivers, speaker enclosures, surround sound systems, sound bars, headphones, earphones, and so forth. The speakers may be configured in any suitable array and may include monophonic drivers, binaural speakers, surround sound speaker arrays, or any other suitable array of audio drivers.

  Aspects of one or more embodiments described herein are implemented in an audio system that processes audio signals for transmission over a network that includes one or more computers or processing devices that execute software instructions. May be. Any of the described embodiments may be used with each other alone or in any combination. While the various embodiments have been motivated by various shortcomings of the prior art that may be discussed or implied in one or more places in the specification, The form does not necessarily address any of these drawbacks. In other words, the various embodiments may address various drawbacks that may be discussed in the specification. Some embodiments may only partially address some or only one drawback that may be discussed in the specification, and some embodiments may not address any of these disadvantages. May not be addressed.

  The system aspects described herein may be implemented in a suitable computer-based sound processing network environment for processing digital or digitized audio files. The parts of the adaptive audio system can include any desired number of individual machines, including one or more routers (not shown) that serve to buffer and route data transmitted between computers. One or more networks may be included. Such a network may be built on a variety of different network protocols and may be the Internet, a wide area network (WAN), a local area network (LAN), or any combination thereof.

  One or more of the above components, blocks, processes or other functional components may be implemented through a computer program that controls the execution of the processor-based computing device of the system. The various functions disclosed in this article are behavioral, register transfers using any combination of hardware, firmware and / or as data and / or instructions embodied in various machine-readable or computer-readable media. It should be noted that logic components and / or other characteristics may be described. Computer readable media on which such formatted data and / or instructions can be implemented include various forms of physical (non-transitory), non-volatile storage media such as optical, magnetic or semiconductor storage media. Is not limited to this.

  Unless the context clearly requires otherwise, the words “comprising”, “including”, and the like are to be interpreted in an inclusive rather than an exclusive or exhaustive sense throughout the description and claims. To do. In other words, it means “including but not limited to”. Words using the singular or plural number also include the plural or singular number respectively. Further, the words “in this article”, “below”, “above”, “below” and similar meanings refer to the present application as a whole, and not to any particular part of the present application. When the word “or” is used with reference to a list of two or more items, the word covers all of the following interpretations of the word: any of the items in the list, all of the items in the list And any combination of items in the list.

  Although one or more implementations are described by way of example with particular embodiments, it is to be understood that one or more implementations are not limited to the disclosed embodiments. On the contrary, it is intended to cover various modifications and similar arrangements that will be apparent to those skilled in the art. Accordingly, the scope of the appended claims should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements.

Claims (22)

A method for generating a low latency virtual base comprising:
Receiving an input audio signal;
Performing a harmonic transition on the low frequency components of the input audio signal to generate the transferred data indicative of the harmonics of the input audio signal;
Generating a virtual base signal in response to the transferred data;
Combining the virtual base signal with a delayed version of the input audio signal to generate an enhanced audio signal;
The harmonic transition uses a combined transition that uses a fundamental transition order B greater than 2, whereby the harmonics are each second harmonic and at least one higher harmonic of the low frequency component. And all of the harmonics are frequency domain values determined by a common time to frequency domain conversion stage using an asymmetric decomposition window and a common frequency to time domain using an asymmetric synthesis window. Generated in response to a subsequent inverse transformation determined by the transformation stage of
Method. The method according to claim 1, wherein the basic transition order B is an integer value selected from the group consisting of 4, 8, 16 or 32.   The input audio signal is a subband complex valued quadrature mirror filter (CQMF) signal indicative of critically sampled or near critically sampled low frequency audio from a set of CQMF subband signals. The method described.   4. The method of claim 3, wherein the critically sampled or near critically sampled low frequency input audio is a CQMF channel 0 signal indicating a lowest frequency band from a set of CQMF subband signals. Generating data transferred from low frequency components produces asymmetrically windowed, zero-padded samples, and asymmetrically windowed, zero-padded samples from time to frequency domain Performing a frequency domain oversampled transform on the input audio signal by performing a transform, and then performing a non-linear operation on the output from the time to frequency domain transform to the low frequency Performing by generating said transferred data from components; and
From frequency components that are processed by said non-linear operation, by dividing the second set of the first set and a second frequency component in the frequency band of the frequency components within the first frequency band, two sets of Generating a frequency component;
In addition, a first frequency to time domain transformation is performed on the first set of frequency components, and a second frequency to time domain transformation is performed on the second set of frequency components. Each of the first frequency to time domain transform and the second frequency to time domain transform has a transform size that is B times smaller than the time to frequency domain transform. Stages;
And applying an asymmetric zero-padded window to the samples from the frequency to time domain transform, wherein the asymmetric zero-padded window is generated from the input audio signal. Comprising B times shorter than the asymmetrically windowed and zero-padded sample, thereby forming two sets of transferred data;
The method of claim 4.   6. The method of claim 5, wherein the first frequency band is a CQMF channel 0 frequency band and the second frequency band is a CQMF channel 1 frequency band from a set of CQMF subband signals.   Generating a virtual base signal in response to the transferred data includes a decomposition filter bank applied to one or both of the two sets of transferred data, the decomposition filter bank being a symmetric filter The method of claim 6, comprising a truncated version.   8. The method of claim 7, wherein the decomposition filter bank is a Nyquist filter bank and the truncated version of a symmetric filter is a filter with one of the symmetric halves of the filter removed.   9. The decomposition filter bank comprises one of an eight channel Nyquist filter bank or a four channel Nyquist filter bank, and one of the symmetrical halves of the filter to be removed comprises six coefficients. Method.   The delayed version of the input audio signal is delayed by a pre-defined time period that is shorter than the latency of the virtual base signal, and the enhanced audio signal is a time delayed virtual base signal. The method of claim 1, wherein:   The method of claim 10, wherein the predefined time period is a value selected from a range of 0 to 1000 samples. Wherein the input audio CQMF channel 0, is received directly from the decomposition CQMF bank output of the preprocessing hybrid filter bank stage, the Nyquist filter bank of the pretreatment hybrid filter bank stage is bypassed method of claim 8, . A device that generates a low-latency virtual base:
A first component that receives an input audio signal and performs a harmonic transition on a low frequency component of the input audio signal to generate transferred data indicative of harmonics of the input audio signal;
A second component for generating a virtual base signal in response to the transferred data and combining the virtual base signal with a delayed version of the input audio signal to generate an enhanced audio signal, Harmonic transitions use a combined transition that uses a fundamental transition order B greater than 2, so that the harmonics are each second harmonic and at least one higher harmonic of the low frequency component. All of the harmonics are frequency domain values determined by a common time to frequency domain conversion stage using an asymmetric decomposition window and a common frequency to time domain using an asymmetric synthesis window. Generated in response to a subsequent inverse transformation determined by the transformation stage,
apparatus. The apparatus of claim 13, wherein the basic transition order B is an integer value selected from the group consisting of 4, 8, 16 or 32.   14. The input audio signal is a subband complex valued quadrature mirror filter (CQMF) signal indicative of critically sampled or near critically sampled low frequency audio from a set of CQMF subband signals. The device described.   16. The apparatus of claim 15, wherein the critically sampled or near critically sampled low frequency audio is a CQMF channel 0 signal indicating a lowest frequency band from a set of CQMF subband signals. Generating data transferred from low frequency components produces asymmetrically windowed, zero-padded samples, and asymmetrically windowed, zero-padded samples from time to frequency domain Performing a frequency domain oversampled transform on the input audio signal by performing a transform, and then performing a non-linear operation on the output from the time to frequency domain transform to the low frequency A third component, by generating the transferred data from the component;
From frequency components that are processed by said non-linear operation, by dividing the second set of the first set and a second frequency component in the frequency band of the frequency components within the first frequency band, two sets of A fourth component that generates frequency components;
In addition, a first frequency to time domain transformation is performed on the first set of frequency components, and a second frequency to time domain transformation is performed on the second set of frequency components. Each of the first frequency to time domain transform and the second frequency to time domain transform has a transform size that is B times smaller than the time to frequency domain transform. And a fifth component;
A sixth component that applies an asymmetric zero-padded window to samples from the frequency to time domain transform, wherein the asymmetric zero-padded window is generated from the input audio signal; A sixth component that is B times shorter than the asymmetrically windowed and zero-padded sample, thereby forming two sets of transferred data;
The apparatus of claim 16. 18. The apparatus of claim 17, wherein the first frequency band is a frequency band of CQMF channel 0 and the second frequency band is a frequency band of CQMF channel 1 from a set of CQMF subband signals.
Generating a virtual base signal in response to the transferred data includes a decomposition filter bank applied to one or both of the two sets of transferred data, the decomposition filter bank being a symmetrical filter bank. A device that contains a truncated version.   19. The apparatus of claim 18, wherein the decomposition filter bank is a Nyquist filter bank and the truncated version of a symmetric filter is a filter with one of the symmetric halves of the filter removed.   20. The decomposition filter bank comprises one of an eight channel Nyquist filter bank or a four channel Nyquist filter bank, and one of the symmetrical halves removed of the filter comprises six coefficients. apparatus. A timing component that generates a version of the input audio signal delayed by a predefined time period shorter than the latency of the virtual base signal;
A mixing component that combines the virtual base signal with the delayed input audio signal to generate an enhanced audio signal indicative of the temporally delayed virtual base signal;
The apparatus of claim 13. The CQMF channel 0, received directly from the decomposition CQMF bank output of the preprocessing hybrid filter bank stage, further comprising an interface component that bypasses the Nyquist filter bank of the pretreatment hybrid filter bank stage, claim 19 The device described.

Images