EP2720477B1 - Virtuelle Basssynthese mit harmonischer Transposition - Google Patents

Virtuelle Basssynthese mit harmonischer Transposition Download PDF

Info

Publication number
EP2720477B1
EP2720477B1 EP13188415.7A EP13188415A EP2720477B1 EP 2720477 B1 EP2720477 B1 EP 2720477B1 EP 13188415 A EP13188415 A EP 13188415A EP 2720477 B1 EP2720477 B1 EP 2720477B1
Authority
EP
European Patent Office
Prior art keywords
frequency
signal
harmonics
audio signal
stage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP13188415.7A
Other languages
English (en)
French (fr)
Other versions
EP2720477A1 (de
Inventor
Per Ekstrand
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Dolby International AB
Original Assignee
Dolby International AB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US13/652,023 external-priority patent/US8971551B2/en
Application filed by Dolby International AB filed Critical Dolby International AB
Publication of EP2720477A1 publication Critical patent/EP2720477A1/de
Application granted granted Critical
Publication of EP2720477B1 publication Critical patent/EP2720477B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/04—Circuits for transducers, loudspeakers or microphones for correcting frequency response
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/038—Speech enhancement, e.g. noise reduction or echo cancellation using band spreading techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2430/00—Signal processing covered by H04R, not provided for in its groups
    • H04R2430/03—Synergistic effects of band splitting and sub-band processing

Definitions

  • the invention relates to methods and systems for virtual bass synthesis.
  • Typical embodiments employ harmonic transposition to generate an enhancement signal which is combined with an audio signal to generate an enhanced audio signal, such that the enhanced audio signal provides an increased perceived level of bass content during playback by one or more loudspeakers that cannot physically reproduce bass frequencies of the audio signal or the enhanced audio signal.
  • Bass synthesis is the collective name for a class of techniques that add in components to the low frequency range of an audio signal in order to enhance the bass that is perceived during playback of the enhanced signal.
  • Some such techniques (sometimes referred to as sub bass synthesis methods) create low frequency components below the signal's existing frequency components in order to extend and improve the lowest frequency range.
  • Other techniques in the class known as "virtual pitch" algorithms, generate audible harmonics from an inaudible bass range (e.g., a bass range that is inaudible when the signal is rendered by small loudspeakers), so that the generated harmonics improve the perceived bass response.
  • Some virtual pitch methods are designed to increase the perceived level of bass content of an audio signal during playback of the signal by one or more loudspeakers (e.g., small loudspeakers) that cannot physically reproduce bass frequencies of the audio signal.
  • Such methods typically include steps of analyzing the bass frequencies present in input audio and enhancing the input audio by generating (and including in the enhanced audio) audible harmonics that aid the perception of lower frequencies that are missing during playback of the enhanced audio (e.g., playback by small loudspeakers that cannot physically reproduce the missing lower frequencies).
  • Such methods perform harmonic transposition of frequency components of the input audio that are expected to be inaudible during playback of the input audio (i.e., having frequencies too low to be audible during playback on the expected speaker(s)), to generate audible higher frequency components (i.e., having frequencies that are sufficiently high to be audible during playback on the expected speaker(s)).
  • FIG. 1 shows the frequency-amplitude spectrum of an audio signal, having an inaudible range 100 of frequency components, and an audible range of frequency components above the inaudible range.
  • Harmonic transposition of frequency components in the inaudible range 100 can generate transposed frequency components in portion 101 of the audible range, which can enhance the perceived level of bass content of the audio signal during playback.
  • Such harmonic transposition may include application of multiple transposition factors to each relevant frequency component of the input audio, to generate multiple harmonics of the component.
  • United States Patent Application Publication No. 2007/0253576 A1 discloses a method for virtual bass synthesis.
  • a low frequency signal is attained by applying a low pass filter to an original signal.
  • process of down sampling the low frequency signal, moving the low frequency signal to a series of harmonics whose frequencies are integral times as large as the frequency of low frequency signals, and then up sampling them are provided.
  • Typical embodiments of the inventive method are designed to increase the perceived level of bass content of an audio signal during playback of the signal by one or more loudspeakers (e.g., small loudspeakers) that cannot physically reproduce bass frequencies of the audio signal.
  • Typical embodiments include steps of: applying harmonic transposition to bass frequencies present in the input audio signal (but expected to be inaudible during playback of the input audio signal using an expected speaker or speaker set) to generate harmonics that are expected to be audible during playback of the enhanced audio signal using the expected speaker(s), and generating enhanced audio (an enhanced version of the input audio) by including the harmonics in the enhanced audio.
  • the method typically includes steps of performing a time-to-frequency domain transform (e.g., an FFT) on the input audio to generate frequency components indicative of bass content of the input audio, and enhancing the input audio by generating (and including in an enhanced version of the input audio) audible harmonics of these frequency components that aid the perception of lower frequencies that are expected to be missing during playback of the enhanced audio (e.g., by small loudspeakers that cannot physically reproduce the missing lower frequencies).
  • a time-to-frequency domain transform e.g., an FFT
  • the invention is a virtual bass generation method, including steps of: (a) performing harmonic transposition on low frequency components of an input audio signal (typically, bass frequency components expected to be inaudible during playback of the input audio signal using an expected speaker or speaker set) to generate transposed data indicative of harmonics (which are expected to be audible during playback, using the expected speaker(s), of an enhanced version of the input audio which includes the harmonics); (b) generating an enhancement signal in response to the transposed data (e.g., such that the enhancement signal is indicative of the harmonics or amplitude modified (e.g., scaled) versions of the harmonics); and (c) generating an enhanced audio signal by combining (e.g., mixing) the enhancement signal with the input audio signal.
  • an input audio signal typically, bass frequency components expected to be inaudible during playback of the input audio signal using an expected speaker or speaker set
  • transposed data indicative of harmonics (which are expected to be audible during playback, using the expected speaker(s), of an enhanced version of the input
  • the enhanced audio signal provides an increased perceived level of bass content during playback of the enhanced audio signal by one or more loudspeakers that cannot physically reproduce the low frequency components.
  • combining the enhancement signal with the input audio signal aids the perception of low frequencies that are missing during playback of the enhanced audio signal (e.g., playback by small loudspeakers that cannot physically reproduce the missing low frequencies).
  • the harmonic transposition performed in step (a) employs combined transposition to generate harmonics, by means of a second order (“base") transposer and at least one higher order transposer (typically, a third order transposer and a fourth order transposer, and optionally also at least one transposer of order higher than four), of each of the low frequency components, such that all of the harmonics (and typically also the transposed data) are generated in response to frequency-domain values determined by a single, common time-to-frequency domain transform stage (e.g., by performing phase multiplication on frequency coefficients resulting from a single time-to-frequency domain transform), and a single, common frequency-to-time domain transform is subsequently performed.
  • the harmonic transposition is performed using integer transposition factors, which eliminates the need for unstable (or inexact) phase estimation, phase unwrapping and/or phase locking techniques (e.g., as implemented in conventional phase vocoders).
  • step (a) is performed on low frequency components of the input audio signal which have been generated by performing a frequency domain oversampled transform on the input audio signal, by generating windowed, zero-padded samples, and performing a time-to-frequency domain transform on the windowed, zero-padded samples.
  • the frequency domain oversampling typically improves the quality of the virtual bass generation in response to impulse-like (transient) signals.
  • the method includes a preprocessing step on the input audio signal to generate critically sampled audio indicative of the low frequency components, and step (a) is performed on the critically sampled audio.
  • the input audio signal is a sub-banded, complex-valued QMF domain (CQMF) signal
  • the critically sampled audio is indicative of content of a set of low frequency sub-bands of the CQMF signal.
  • CQMF complex-valued QMF domain
  • the input audio signal is indicative of low frequency audio content (in a range from 0 to B Hz, where B is a number less than 500)
  • the critically sampled audio is an at least substantially critically sampled (critically sampled or close to critically sampled) signal indicative of the low frequency audio content, and has sampling frequency Fs/Q, where Fs is the sampling frequency of the input audio signal, and Q is a downsampling factor.
  • Q is the largest factor which makes Fs/Q at least substantially equal to (but not less than) two times the bandwidth B of the input signal (i.e., Q ⁇ Fs/2B).
  • step (a) is performed in a subsampled (downsampled) domain, which is the first (lowest frequency) band (channel 0) of a CQMF bank for the transposer analysis stage (input), and the first two (lowest frequency) bands (channels 0 and 1) of a CQMF bank for the transposer synthesis stage (output).
  • the separation of CQMF channels 0 and 1 is accomplished by a splitting of processed frequency coefficients (i.e., frequency coefficients formerly processed by non-linear processing stages 9-11 and energy adjusting stages 13-15 of Fig.
  • the first set of frequency components and the second set of frequency components are magnitude compensated to account for the CQMF channel 0 and CQMF channel 1 frequency responses.
  • the magnitude compensations are applied to the frequency components indicative of the overlapping regions between CQMF channel 0 and CQMF channel 1 (e.g., for the frequency components of CQMF channel 0 indicative of the middle of the pass band and upwards in frequency, and for the frequency components of CQMF channel 1 indicative of the middle of the pass band and downwards in frequency).
  • the transposed data are energy adjusted (e.g., attenuated).
  • the transposed data may be attenuated in a manner determined by the well-known Equal Loudness Contours (ELCs) or an approximation thereof.
  • ELCs Equal Loudness Contours
  • the transposed data indicative of each generated harmonic overtone spectrum may have an additional attenuation (e.g., a slope gain in dB per octave) applied thereto.
  • the attenuation may depend on a tonality metric (e.g., for the frequency range of the low frequency components of the input audio signal), e.g., so that a strong tonality results in a larger attenuation (in dB per octave) within the spectrum of each generated harmonic overtone.
  • a tonality metric e.g., for the frequency range of the low frequency components of the input audio signal
  • data indicative of the harmonics are energy adjusted (e.g., attenuated) in accordance with a control function which determines a gain to be applied to each hybrid sub-band of the transposed data (where a hybrid sub-band may constitute a frequency band division of the audio data, indicative of a frequency resolution somewhere in-between the resolution provided by the time-to-frequency domain transform of the "base" transposer and the bandwidth of the sub-banded input signal respectively).
  • g b H G ⁇ nrg orig b - nrg vb b / G ⁇ nrg orig b + nrg vb b + B , where H, G and B are constants, and nrg orig ( b
  • Another aspect of the invention is a system (e.g., a device having physically-limited or otherwise limited bass reproduction capabilities, such as, for example, a notebook, tablet, mobile phone, or other device with small speakers) configured to perform any embodiment of the inventive method on an input audio signal.
  • a system e.g., a device having physically-limited or otherwise limited bass reproduction capabilities, such as, for example, a notebook, tablet, mobile phone, or other device with small speakers
  • the invention is an audio playback system which has limited (e.g., physically-limited) bass reproduction capabilities (e.g., a notebook, tablet, mobile phone, or other device with small speakers), and is configured to perform virtual bass generation on audio (in accordance with an embodiment of the inventive method) to generate enhanced audio, and to playback the enhanced audio.
  • the virtual bass generation is performed such that playback of the enhanced audio by the system provides the perception of enhanced bass response (relative to the bass response perceived during playback of the non-enhanced input audio by the device), including by synthesizing audible harmonics of frequencies (of the input audio) which are below the system's low-frequency roll-off (e.g., below approximately 100-300 Hz).
  • the bass perceived during playback of the enhanced audio using headphones or full-range loudspeakers is also increased.
  • the invention is a method for performing harmonic transposition of inaudible signal components of input audio (components having frequencies too low to be audible during playback by an expected speaker or set of speakers), to generate enhanced audio including audible harmonics of the inaudible components (i.e., harmonics having frequencies that are audible during playback on the expected speaker or set of speakers), including by application of plural transposition factors (to produce the audible harmonics) followed by energy adjustment.
  • Other aspects of the invention are systems and devices configured to perform such harmonic transposition.
  • the upper (audible) harmonics thereof that are included in an enhanced audio signal typically must constitute an at least substantially complete (but truncated) harmonic series.
  • typical embodiments of the invention transpose all frequency components in a predetermined source range and these components might themselves be harmonics of unknown order.
  • a missing fundamental itself may not be perceived when the enhanced audio is rendered.
  • the sensation of bass will be typically recognized because a source (e.g., a musical instrument) generating a bass signal will be perceived as being present in the enhanced audio although at a higher pitch (e.g., at the first harmonic of the fundamental).
  • the inventive system comprises a preprocessing stage (e.g., a summation stage) coupled to receive input audio indicative of low frequency audio content (in a range from 0 to B Hz, so that B is the bandwidth of the low frequency audio content) and configured to generate critically sampled audio indicative of the low frequency audio content; a bass enhancement stage (including a harmonic transposer) coupled and configured to generate a bass enhancement signal in response to the critically sampled audio; and a bass enhanced audio generation stage coupled and configured to generate to a bass enhanced audio signal by combining (e.g., mixing) the bass enhancement signal and the input audio.
  • a preprocessing stage e.g., a summation stage
  • a bass enhancement stage including a harmonic transposer
  • a bass enhanced audio generation stage coupled and configured to generate to a bass enhanced audio signal by combining (e.g., mixing) the bass enhancement signal and the input audio.
  • the preprocessing stage is preferably configured to provide an at least substantially critically sampled (critically sampled or close to critically sampled) signal to the bass enhancement stage.
  • the at least substantially critically sampled signal is indicative of the low frequency audio content (in the range from 0 to B Hz), and has sampling frequency Fs/Q, where Fs is the sampling frequency of the input audio, and Q is a downsampling factor.
  • Q is the largest factor which makes Fs/Q at least substantially equal to (but not less than) two times the bandwidth B of the input signal (i.e., Q ⁇ Fs/2B).
  • Transposed frequency components produced in the bass enhancement stage
  • the downsampling factor Q preferably forces the output signal of the summation stage to be critically sampled or close to critically sampled.
  • the inventive system is or includes a general or special purpose processor programmed with software (or firmware) and/or otherwise configured to perform an embodiment of the inventive method.
  • the inventive system is a general purpose processor, coupled to receive input audio data, and programmed (with appropriate software) to generate output audio data by performing an embodiment of the inventive method.
  • the inventive system is a digital signal processor, coupled to receive input audio data, and configured (e.g., programmed) to generate output audio data in response to the input audio data by performing an embodiment of the inventive method.
  • aspects of the invention include a system configured (e.g., programmed) to perform any embodiment of the inventive method, and a computer readable medium (e.g., a disc) which stores code for implementing any embodiment of the inventive method.
  • a computer readable medium e.g., a disc
  • performing an operation "on" a signal or data e.g., filtering, scaling, transforming, or applying gain to, the signal or data
  • a signal or data e.g., filtering, scaling, transforming, or applying gain to, the signal or data
  • performing the operation directly on the signal or data or on a processed version of the signal or data (e.g., on a version of the signal that has undergone preliminary filtering or pre-processing prior to performance of the operation thereon).
  • system is used in a broad sense to denote a device, system, or subsystem.
  • a subsystem that implements a decoder may be referred to as a decoder system, and a system including such a subsystem (e.g., a system that generates X output signals in response to multiple inputs, in which the subsystem generates M of the inputs and the other X - M inputs are received from an external source) may also be referred to as a decoder system.
  • processor is used in a broad sense to denote a system or device programmable or otherwise configurable (e.g., with software or firmware) to perform operations on data (e.g., audio, or video or other image data).
  • data e.g., audio, or video or other image data.
  • processors include a field-programmable gate array (or other configurable integrated circuit or chip set), a digital signal processor programmed and/or otherwise configured to perform pipelined processing on audio or other sound data, a programmable general purpose processor or computer, and a programmable microprocessor chip or chip set.
  • Coupled is used to mean either a direct or indirect connection.
  • that connection may be through a direct connection, or through an indirect connection via other devices and connections.
  • the inventive virtual bass synthesis method implements the following basic features:
  • the input data to be processed in accordance with the invention are sub-banded CQMF (complex-valued quadrature mirror filter) domain audio data.
  • the CQMF data for the low frequency sub-band channels can undergo further frequency band splittings (e.g., in order to increase the frequency resolution for the low frequency range) by means of Nyquist filter banks of different sizes.
  • Nyquist filter banks do not employ downsampling of the sub-band samples.
  • the Nyquist filter banks have a particularly straightforward synthesis step, i.e. pure addition of the sub-band samples.
  • hybrid sub-band samples In such systems, the combination of low frequency sub-band samples from the Nyquist analysis stages and the remaining CQMF channels (i.e., the CQMF channels that were not subjected to Nyquist filtering) are herein referred to as "hybrid" sub-band samples.
  • a number of the lowest hybrid sub-bands can be combined (e.g., added together).
  • the lowest frequency hybrid sub-bands of the data e.g., sub-bands 0-7, as shown in Fig. 2 , where the sub-bands together span the range from 0-375 Hz
  • the sub-bands together span the range from 0-375 Hz
  • the latter signal is a low-pass filtered, complex-valued, time-domain audio signal (preferably, a critically sampled signal) whose pass band is 0 Hz to 375 Hz.
  • the CQMF channel 0 signal undergoes optional compression (e.g., in stage 45 of the Fig. 2 system), windowing and zero-padding (e.g., in stage 3 of the Fig. 2 system), and then time-to-frequency domain transformation (e.g., in transform stage 5 of the Fig. 2 system).
  • optional compression e.g., in stage 45 of the Fig. 2 system
  • windowing and zero-padding e.g., in stage 3 of the Fig. 2 system
  • time-to-frequency domain transformation e.g., in transform stage 5 of the Fig. 2 system.
  • the transform stage typically implements an FFT (Fast Fourier Transform)
  • the transform stage implements a time-to-frequency domain transform of another type (e.g., in variations on the Fig.
  • transform stage 5 implements a Fourier Transform, a Discrete Fourier Transform, or a Wavelet Transform, or another time-to-frequency domain transform or analysis filter bank which is not an FFT, and each of inverse transform stages 29 and 31 implements a corresponding inverse transform (a frequency-to-time domain transform) or synthesis filter bank.
  • a typical conventional harmonic transposer operates on a time domain signal having full sampling rate (44.1 kHz or 48 kHz), and employs an FFT (e.g., of size equal to roughly 1024 to 4096 lines) to generate (in the frequency domain) output audio indicative of frequency transposed samples of the input signal.
  • FFT e.g., of size equal to roughly 1024 to 4096 lines
  • Such a typical transposer also employs an inverse FFT to generate time domain output audio in response to the frequency domain output.
  • the samples of the single, critically sampled (or nearly critically sampled) channel can be efficiently transformed into the frequency domain by an FFT transform of much smaller size (e.g., an FFT with block size of 32-256 samples) than the FFT transform (e.g., of block size equal to 1024 to 4096) that would be needed if the raw, unfiltered time-domain input data were transformed directly into the frequency domain.
  • an FFT transform of much smaller size e.g., an FFT with block size of 32-256 samples
  • the FFT transform e.g., of block size equal to 1024 to 4096
  • Performing frequency transposition directly on the sub-bands of the hybrid data (the input to stage 1 of Fig. 2 ), and combining the resulting transposed data, is a suboptimal option. This is because, each of the low frequency hybrid sub-bands (shown as the input to stage 1 of Fig. 2 ) is oversampled data, and if stage 1 of Fig. 2 were omitted, each of the low frequency hybrid sub-bands would be transformed into the frequency domain, so that the processing power required for each of the hybrid sub-bands would be as high as the processing power required for the single CQMF band (channel 0) in the Fig. 2 system.
  • the inventive system When performing frequency transposition on a single CQMF band (e.g., channel 0), the inventive system preferably changes the phase response that would be needed if the transposition were performed directly on the CQMF sub-bands (frequency transposition in the CQMF domain is indeed possible.
  • the frequency resolution provided by the sub-band samples of the CQMF bank is inadequate for virtual bass processing in accordance with the invention).
  • this phase response compensation is applied by element 2 of the Fig. 2 system.
  • the phase relations between the neighboring channels in a CQMF bank will not be correct when performing an FFT split (in element 19 of the Fig. 2 system). Therefore, a phase compensation factor needs to be applied (in element 37 of the Fig. 2 system).
  • the general CQMF analysis modulation may have the expression
  • T e.g., in stage 9 of the Fig.
  • CQMF channel 1 of the output (the signal output from stage 35 of Fig. 2 ) needs a multiplication by e -i ⁇ /2 to preserve the phase relationship and emulate that it has passed a CQMF analysis stage. This multiplication is performed in element 37 of Fig. 2 .
  • the 8-channel Nyquist filter bank has pass-bands with center frequencies 47 Hz, 141 Hz, 234 Hz, 328 Hz, 422 Hz, 516 Hz , -141 Hz, and -47 Hz.
  • the Nyquist filter bank uses complex-valued arithmetic and operates on complex-valued CQMF samples (channel 0) as input.
  • the first 4 pass-bands (0-3) constitute the pass-band of CQMF channel 0, while the last 4 pass-bands filters the CQMF transition regions: channel 4 and 5 filters the overlap/transition region of CQMF channel 0 towards CQMF channel 1, and channel 6 and 7 filters the transition region to negative frequencies of CQMF channel 0.
  • the output from the Nyquist filter bank is simply band-passed versions of the input CQMF signal.
  • stage 1 adds the eight streams of Nyquist samples back together (Nyquist synthesis), the result is an exact reconstruction of the CQMF channel 0, which is critically sampled in terms of sampling frequency (actually the CQMF bank may be oversampled by a factor of 2 due to the complex-valued sub-band samples, while the real part only of its output may be critically sampled (maximally decimated)).
  • the Nyquist synthesis step (implemented in a typical implementation of stage 1 of the Fig. 2 system) is particularly straightforward since it is just a simple summation of the samples from the 8 lowest hybrid channels of the sub-banded input data for each CQMF time slot.
  • the summation generates a conventional CQMF channel 0 signal, which is input to element 2 of the Fig. 2 system (or to compressor 45, in implementations in which the optional compressor 45 is included in the Fig. 2 system).
  • the output signals from the inventive transposer are two CQMF signals (the outputs of elements 33 and 35 of Fig. 2 ), containing the bass enhancement signal (sometime referred to as a virtual bass signal) to be mixed (in stage 43) with an appropriately delayed version of the original input signal.
  • Both output signals are filtered through 8- and 4-channel Nyquist analysis stages (stages 39 and 41 of Fig. 2 ) respectively to convert them back to the original hybrid sub-banded domain.
  • Stage 39 implements 8-channel analysis to output, in parallel, 8 sub-band channels in response to the CQMF signal (CQMF channel 0) asserted to its input.
  • Stage 41 implements 4-channel analysis to output, in parallel, four sub-band channels in response to the CQMF signal (CQMF channel 1) asserted to its input.
  • the CQMF channel 0 signal (produced in stage 1 of Fig. 2 ) optionally undergoes dynamic range compression (e.g., in compressor 45 of Fig. 2 ).
  • dynamic range compression e.g., in compressor 45 of Fig. 2
  • compression is used in a broad sense to denote either broadening of the dynamic range (sometimes referred to dynamic range expansion) or narrowing of the dynamic range, so that compressor 45 may be what is sometimes referred to as a compander (compressor/expander).
  • a low pass filtered, down-mixed (mono) version of the CQMF channel 0 signal can be used as the control signal for the compressor.
  • stage 1 of the Fig. 2 system can sum the lowest four sub-bands of the hybrid, sub-banded input data, and assert the control signal to compressor 45.
  • compressor 45 (or element 1B of the Fig. 4 system, to be described below) performs an averaged energy calculation, and computes the compression gain required to perform the appropriate dynamic range compression.
  • stage 3 performs the following operations on the complex-valued CQMF channel 0 samples asserted thereto (to implement frequency domain oversampling by a factor of 2):
  • stage 5 performs a 128-point complex FFT on each windowed, zero-padded block.
  • stage 19 splits (in a manner to be described in more detail below) each block of the processed coefficients into two half sized blocks (each comprising 64 coefficients): a first block indicative of content in the frequency range 0-375 Hz; and a second block indicative of content in the frequency range 375-750 Hz.
  • stage 29 performs a 64-point IFFT on each first block
  • stage 31 performs a 64-point IFFT on each second block.
  • Windowing and overlap/adding stage 33 discards the first and last 16 samples from each transformed block output from stage 29, windows the remaining 32 samples with a 32-point synthesis window, and overlap-adds the resulting samples, to generate a conventional CQMF channel 0 signal indicative of the transposed content in the range 0 to 375 Hz.
  • Element 37 performs the above-described phase shift on this signal to generate a conventional CQMF channel 1 signal indicative of the transposed content in the range 375 to 750 Hz.
  • the block size of the input to stage 3 is quite small (32-256 samples per block).
  • the block size of the forward transform implemented by stage 5 is typically larger, and the specific forward transform block size depends on the frequency domain oversampling (typically a factor of 2, but sometimes a factor of 4).
  • the inventive system uses asymmetric analysis and synthesis windows for the forward (e.g., FFT) and inverse (e.g., IFFT) transforms in contrast to the symmetric windows used in typical implementations.
  • the size (number of points) of the analysis window (e.g., the window applied in stage 3) and the forward transform (e.g., the transform applied by stage 5) may be different from that of the synthesis window (e.g., the window applied in stage 33 or 35) and the inverse transform (e.g., the inverse transform applied in stage 29 or 31).
  • the shape and size of each window and size of each transform maybe chosen so as to achieve adequate frequency resolution while lowering the inherent algorithmic delay of the transposer.
  • computational complexity is reduced by processing only the signal of interest (e.g., the CQMF channel 0 data, generated in stage 1 of the Fig. 2 system in response to hybrid, sub-banded input data, are critically sampled).
  • the signal of interest e.g., the CQMF channel 0 data, generated in stage 1 of the Fig. 2 system in response to hybrid, sub-banded input data, are critically sampled.
  • the inventive system comprises a preprocessing stage (e.g., summation stage 1 of the Fig. 2 system), coupled to receive input audio indicative of low frequency audio content (in a range from 0 to B Hz, so that B is the bandwidth of the low frequency audio content) and configured to generate critically sampled audio indicative of the low frequency audio content (e.g., the CQMF channel 0 signal output from stage 1 of Fig. 2 ); a bass enhancement stage (including a harmonic transposer) coupled and configured to generate a bass enhancement signal (e.g., the output of stages 39 and 41 of the Fig. 2 system) in response to the critically sampled audio; and a bass enhanced audio generation stage (e.g., stage 43 of the Fig.
  • a preprocessing stage e.g., summation stage 1 of the Fig. 2 system
  • receive input audio indicative of low frequency audio content in a range from 0 to B Hz, so that B is the bandwidth of the low frequency audio content
  • the bass enhanced audio signal is a full frequency range signal generated by mixing the bass enhancement signal output from stages 39 and 41 of Fig. 2 ), and the input audio (sub-bands 0-7 of the hybrid sub-band signal) asserted to the summation stage, and also the other sub-bands (e.g., sub-bands 8-76) of the hybrid signal.
  • the preprocessing stage e.g., summation stage 1 of Fig.
  • the at least substantially critically sampled signal is indicative of the low frequency audio content (in the range from 0 to B Hz), and has sampling frequency Fs/Q, where Fs is the sampling frequency of the input audio, and Q is a downsampling factor.
  • Q is the largest factor which makes Fs/Q at least substantially equal to (but not less than) two times the bandwidth B of the input signal (i.e., Q ⁇ Fs/2B).
  • Transposed frequency components produced in the bass enhancement stage
  • the downsampling factor Q preferably forces the output signal of the summation stage to be critically sampled or close to critically sampled.
  • the 2nd order "base" transposer (stage 9 of Fig. 2 ) of the inventive system extends the bandwidth of the input signal by a factor of two, thus generating harmonic components of 2 nd order, and transposers of other orders (e.g., stage 11 of Fig. 2 ) generate harmonics of greater factors.
  • the frequency-transposed output of the inventive virtual bass system typically does not need to include frequency components above about 500 Hz (otherwise, the audio signal frequency range to be transposed would extend above what is considered the bass range).
  • the first CQMF channel (channel 0), whose bandwidth is from 0 to 375 Hz (at 48 kHz), has bandwidth which is typically more than adequate for the virtual bass synthesis system input.
  • the first two CQMF channels (channel 0 and 1) have combined bandwidth (0 to 750 Hz at 48 kHz) that is typically sufficient for the virtual bass synthesis system output.
  • each complex coefficient output from transform stage 5 corresponds to a frequency identified by index k .
  • Element 7 of Fig. 2 multiplies each complex coefficient by e i ⁇ k .
  • Stage 5 and element 7, considered together, are a subsystem (which may be referred to as a transform stage) which implements a single time-to-frequency domain transform.
  • Element 7 is used to center the analysis window at time 0 in the FFT, an important step in a transposer (or phase vocoder).
  • the Fig. 2 system also includes transposers of other orders (e.g., fifth and optionally also higher orders), not shown in Fig. 2 .
  • Each of such optional transposers operates in parallel with stages 9 and 11, and multiplies the phase of each complex coefficient asserted thereto by a transposition factor T, where T is an integer greater than 4, either directly or by interpolation of coefficients, so as to produce a harmonic (or corresponding order) of such coefficient.
  • phase multiplier stages 9 and 11 implement nonlinear processing which determines contributions to different frequency bands (e.g., different frequency bands of the enhanced low frequency audio output from stages 39 and 41) in response to one frequency band of the input low frequency audio to be enhanced (i.e., in response to a complex coefficient generated by transform stage 5 having a single frequency index k, or in response to complex coefficients generated by transform stage 5 having frequency indices, k, in a range).
  • different frequency bands e.g., different frequency bands of the enhanced low frequency audio output from stages 39 and 41
  • one frequency band of the input low frequency audio to be enhanced i.e., in response to a complex coefficient generated by transform stage 5 having a single frequency index k, or in response to complex coefficients generated by transform stage 5 having frequency indices, k, in a range.
  • the interpolation scheme for transposition orders higher than 2 enables the use of a single, common time-to-frequency transform or analysis filter bank (including transform stage 5) and a single common frequency-to-time transform or synthesis filter bank (including inverse transform stages 29 and 31) for all orders of transposition, thereby significantly reducing the computational complexity when using multiple harmonic transposers.
  • the overall gains for the coefficients to which different transposition factors have been applied are set independently (in stages 13-15).
  • Gain stage 13 sets the gain of the coefficients output from stage 9
  • gain stage 15 sets the gain of the coefficients output from stage 11
  • an additional gain stage (not shown in Fig. 2 ) for each other phase multiplier stage sets the gain of the coefficients output from the corresponding phase multiplier stage.
  • One such additional gain stage is gain stage 14 of Fig. 4 , which sets the gain of the coefficients output from stage 10 of Fig. 4 .
  • the coefficients output from the gain stages 13-15 are summed in element 17, generating a single stream of frequency-transposed (and level adjusted) coefficients which is indicative of the enhanced audio (virtual bass) determined in accordance with the invention. This single stream of frequency-transposed coefficients is asserted to the input of element 19.
  • the gains can be set to approximate the well-known Equal Loudness Contours (ELCs), since the ELCs can be adequately modeled by a straight line on a logarithmic scale for frequencies below 400 Hz.
  • ELCs Equal Loudness Contours
  • the odd order harmonics (the 3 rd order harmonic, 5 th order harmonic, etc.) can sometimes be perceived as being more harsh than the even order harmonics (the 2 nd order harmonic, 4 th order harmonic, etc.), although their presence is typically important (or vital) for the virtual bass effect.
  • the odd order harmonics may be attenuated (in stages 13-15) by more than the amount determined by the ELCs.
  • each gain stage may apply (to one of the streams of transposed coefficients) a slope gain, i.e.
  • a roll-off attenuation factor (e.g., measured in Decibels per octave). This attenuation is applied on a per bin basis (i.e., an attenuation value is applied independently for each frequency index, k).
  • a control signal indicative of a tonality metric (indicated in Fig. 2 , although this signal is not applied in some implementations) for CQMF channel 0 is asserted to the gain stages, and the gain stages apply gain on a per bin basis in response to the control signal.
  • the slope gain may be applied (e.g., increased by 6 dB or some other amount per octave) so that the roll-off is steeper. This can improve the listening experience for audio (e.g., music) with bass (e.g., bass guitar) sounds consisting of strong harmonic series, which otherwise would result in an over-exaggerated virtual bass effect.
  • a control signal indicative of a tonality measure is asserted to the gain stages (e.g., stages 13-15), and the gain stages apply gain on a per bin basis in response to the control signal.
  • the tonality measure has been obtained by the conventional method used for CQMF subband samples in conventional HE-AAC audio encoding, where LPC coefficients are used to calculate the relation between the predictable part of the signal and the prediction error (the un-predictable part).
  • a control (correction) function is typically used.
  • this level compensation operation is performed in the hybrid sub-band
  • index i is the block index, i.e. the index of the blocks that are made up of subsequent hybrid sub-band samples over which the averaging is performed.
  • a block consists of 4 hybrid sub-band samples.
  • nrg vb ( c,i,b ) is a function of energy, E vb ( c, n, b ) , of the transposed signal contained in the hybrid sub-band sample in channel c, sub-band time slot n, and hybrid sub-band b , and is calculated in the way in which nrg org ( c,i,b ) is determined in equation (6), with E vb ( c, n, b ) replacing E org ( c, n, b).
  • the correction function of Eq. 5 is illustrated in Fig.
  • V(c, i, b) is plotted on the axis labeled "Level compensation factor”
  • energy E vb ( c, n, b ) is plotted on the axis labeled "VB energy”
  • energy E org ( c, n, b ) is plotted on the axis labeled "Original energy.”
  • the frequency-transposed data asserted from the output of element 17 of Fig. 2 is preferably transformed into a CQMF channel 0 signal and a CQMF channel 1 signal.
  • Stage 19 is configured to split each block of frequency-transposed coefficients (typically comprising 128 coefficients) that is output from element 17 into two half sized blocks: a first half sized block (typically comprising 64 coefficients) indicative of content in the frequency range 0-375 Hz; and a second half sized block (typically comprising 64 coefficients) indicative of content in the frequency range 375-750 Hz.
  • Stages 21 and 23 perform CQMF prototype filter frequency response compensation in the frequency domain.
  • the CQMF response compensation performed in stage 21 changes the gains of the 0-375 Hz components output from stage 19 to match the normal profile produced in conventional processing of CQMF data
  • the CQMF response compensation performed in stage 23 changes the gains of the 375-750 Hz components output from stage 19 to match the normal profile produced in conventional processing of CQMF data.
  • the CQMF compensations are applied to the frequency components indicative of the overlapping regions between CQMF channel 0 and CQMF channel 1 (e.g., for the frequency components of CQMF channel 0 indicative of the middle of the pass band and upwards in frequency, and for the frequency components of CQMF channel 1 indicative of the middle of the pass band and downwards in frequency).
  • the levels of compensation are set to distribute the energy of the overlapping parts of the spectrum in a manner that a conventional CQMF analysis filter bank would do between CQMF channel 0 and CQMF channel 1 in the absence of the FFT splitting stage 19 of Fig. 2 .
  • Element 25 multiplies each complex coefficient output from stage 21 (and having frequency index k ) by e -i ⁇ k , to cancel the shift applied by element 7.
  • Element 27 multiplies each complex coefficient output from stage 23 (and having frequency index k) by e -i ⁇ k , to cancel the shift applied by element 7.
  • Stage 29 performs a frequency-to-time domain transform (e.g., an IFFT, where stage 5 had performed an FFT) on each block of the coefficients output from element 25.
  • Stage 31 performs a frequency-to-time domain transform (e.g., an IFFT, where stage 5 had performed an FFT) on each block of the coefficients output from element 27.
  • Windowing and overlap/adding stage 33 discards the first and last m samples (where m is typically equal to 16) from each transformed block output from inverse transform stage 29, windows the remaining samples, and overlap-adds the resulting samples, to generate a conventional CQMF channel 0 signal indicative of the transposed content in the range 0 to 375 Hz.
  • windowing and overlap/adding stage 35 discards the first and last m samples (where m is typically equal to 16) from each transformed block output from inverse transform stage 31, windows the remaining samples, and overlap-adds the resulting samples, to generate a signal indicative of the transposed content in the range 375 to 750 Hz.
  • Element 37 performs the above-described phase shift on this signal to generate a conventional CQMF channel 1 signal indicative of the transposed content in the range 375 to 750 Hz.
  • the output signals of elements 33 and 37 are filtered in Nyquist 8- and 4-channel analysis stages (stages 39 and 41 of Fig. 2 ) respectively to convert them back to the original hybrid sub-banded domain.
  • Stage 39 implements 8-channel analysis to output, in parallel, 8 sub-band channels in response to the CQMF channel 0 signal asserted to its input.
  • Stage 41 implements 4-channel analysis to output, in parallel, four sub-band channels in response to the CQMF channel 1 signal asserted to its input.
  • the outputs of stages 39 and 41 together comprise a bass enhancement signal (i.e., when mixed together, they determine the bass enhancement signal) which has been generated in the bass enhancement stage of the Fig. 2 system.
  • the bass enhancement stage includes a harmonic transposer configured to apply transpositions having several transposition factors to low frequency content of input audio (i.e., to sub-bands 0-7 of the hybrid sub-banded input audio, whose content is in the range from 0 Hz to 375 Hz).
  • the bass enhancement signal (including content in the range from 0 Hz to 750 Hz) is combined (e.g., mixed) with the input audio in bass enhanced audio generation stage 43 to generate a bass enhanced audio signal (the output of stage 43).
  • the high frequency content (sub-bands 8-76) of the hybrid sub-banded input audio is also mixed with the bass enhancement signal in stage 43.
  • the output of stage 43 is full range audio (the bass enhanced audio signal) which has been bass enhanced in accordance with an embodiment of the inventive virtual bass synthesis method.
  • FIG. 4 is a block diagram of an implementation of the Fig. 2 system. Elements of the Fig. 4 implementation that are identical to corresponding elements of the Fig. 2 system are identically numbered in Figs. 2 and 4 , and the description of them above will not be repeated with reference to Fig. 4 .
  • Fig. 4 includes input data buffer 110, which buffers the hybrid, sub-banded input audio data, whose sub-bands 0-7 are input to stage 1.
  • Fig. 4 also includes Nyquist synthesis stage 1A which is coupled to buffer 110 and configured to implement simple summation of the samples from the e.g. 4 lowest sub-bands (sub-bands 0-3) of the sub-banded input audio data in buffer 110, for each hybrid sub-band time slot.
  • a stereo or a multi-channel signal would also be mixed down to a mono signal by the stage 1A.
  • the output of stage 1A is indicative of a low-passed, mixed down for all input speaker channels, version of the CQMF sub-band signal of channel 0 (i.e., the output from stage 1).
  • the output of stage 1A is employed by compression gain determination stage 1B to generate a control signal for compressor 45.
  • stage 1B In response to the output of stage 1A, stage 1B performs an averaged energy calculation, and computes the compression gain required to perform appropriate dynamic range compression on the corresponding segments of the output of stage 2. Stage 1B asserts (to compressor 45) the control signal to cause compressor 45 to perform such dynamic range compression.
  • compressor 45 is buffered in buffer 111 (coupled between elements 45 and 3 as shown in Fig. 4 ), and then asserted to stage 3 for windowing and zero-padding.
  • stage 112 (coupled between elements 5 and stages 9-11 as shown in Fig. 4 , if included), the complex coefficients output from transform stage 5 are employed to calculate cross-products which can be used in some implementations of phase multiplication stages 9-11, as described in the paper by Lars Villemoes, Per Ekstrand, and Per Hedelin, entitled "Methods for Enhanced Harmonic Transposition," 2011 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, October 16-19, 2011.
  • element 113 coupled between elements 5 and stages 13-15 as shown in Fig. 4 , if included, the complex coefficients output from transform stage 5 are employed to determine spectrum magnitudes, which are in turn used to generate control signals which are asserted to stages 13-15 to control the gains (applied by stages 13-15) for the coefficients to which transposition factors have been applied by phase multiplier stages 9-11.
  • the Fig. 4 system also includes output buffer 116 (coupled between element 33 and stage 39 as shown in Fig. 4 ) for the CQMF channel 0 data output from element 33), and output buffer 117 (coupled between element 37 and stage 41 as shown in Fig. 4 ) for the CQMF channel 1 data output from element 37.
  • the Fig. 4 system optionally includes limiter 114 (coupled between element 33 and buffer 116 as shown in Fig. 4 , if included), and limiter 115 (coupled between element 37 and buffer 117 as shown in Fig. 4 , if included).
  • limiter 114 coupled between element 33 and buffer 116 as shown in Fig. 4 , if included
  • limiter 115 coupled between element 37 and buffer 117 as shown in Fig. 4 , if included.
  • Such limiters would function to limit the magnitudes of the transposed samples output from elements 33 and 37, e.g., to maintain averaged values of the magnitudes within predetermined limiting values.
  • the invention is a virtual bass generation method, including steps of:
  • the harmonic transposition performed in step (a) employs combined transposition to generate harmonics, including a second order (“base”) transposer and at least one higher order transposer (typically, a third order transposer and a fourth order transposer, and optionally also at least one transposer of order higher than four), of each of the low frequency components, such that all of the harmonics (and typically also the transposed data) are generated in response to frequency-domain values determined by a single, common time-to-frequency domain transform stage (e.g., by performing phase multiplication, either direct or by interpolation, on frequency coefficients resulting from a single time-to-frequency domain transform, for example, implemented by transform stage 5 and element 7 of the Fig.
  • base second order
  • at least one higher order transposer typically, a third order transposer and a fourth order transposer, and optionally also at least one transposer of order higher than four
  • the harmonic transposition is performed using integer transposition factors (e.g., the factors two, three, and four applied respectively by stages 9, 10, and 11 of Fig. 4 ), which eliminates the need for unstable (or inexact) phase estimation, phase unwrapping and/or phase locking techniques (e.g., as implemented in conventional phase vocoders).
  • integer transposition factors e.g., the factors two, three, and four applied respectively by stages 9, 10, and 11 of Fig. 4 .
  • step (a) is performed on low frequency components of the input audio signal which have been generated by performing a frequency domain oversampled transform on the input audio signal (e.g., frequency domain oversampling as implemented by stage 3 of Fig. 2 ), by means of generating windowed, zero-padded samples, and performing a time-to-frequency domain transform on the windowed, zero-padded samples.
  • the frequency domain oversampling typically improves the quality of the virtual bass generation in response to impulse-like (transient) signals.
  • the method includes a step to generate critically sampled audio indicative of the low frequency components (e.g., as implemented by stage 1 of Fig. 2 ), and step (a) is performed on the critically sampled audio.
  • the input audio signal is a complex-valued QMF domain (CQMF) signal
  • the critically sampled audio is indicative of a set of low frequency sub-bands (e.g., sub-bands 0-7) of the hybrid signal.
  • CQMF complex-valued QMF domain
  • the input audio signal is indicative of low frequency audio content (in a range from 0 to B Hz, where B is a number less than 500)
  • the critically sampled audio is an at least substantially critically sampled (critically sampled or close to critically sampled) signal indicative of the low frequency audio content, and has sampling frequency Fs/Q, where Fs is the sampling frequency of the input audio signal, and Q is a downsampling factor.
  • Q is the largest factor which makes Fs/Q at least substantially equal to (but not less than) two times the bandwidth B of the input signal (i.e., Q ⁇ Fs/2B).
  • step (a) is performed in a subsampled (downsampled) domain, which is the first (lowest frequency) band (channel 0) of a CQMF bank for the transposer analysis stage (input), and the first two (lowest frequency) bands (channels 0 and 1) of a CQMF bank for the transposer synthesis stage (output).
  • the separation of CQMF channels 0 and 1 is accomplished by a splitting of the transposed data (e.g., as in element 19 of Fig.
  • each frequency-to-time domain transform e.g., the transform implemented by stage 29 of Fig. 2 and the transform implemented by stage 31 of Fig.
  • the first set of frequency components and the second set of frequency components are magnitude compensated to account for the CQMF channel 0 and CQMF channel 1 frequency responses.
  • the transposed data are energy adjusted (e.g., attenuated), for example, as in elements 13-15 of Fig. 2 .
  • the transposed data may be attenuated in a manner determined by the well-known Equal Loudness Contours (ELCs) or an approximation thereof.
  • ELCs Equal Loudness Contours
  • the transposed data indicative of each generated harmonic overtone spectrum may have an additional attenuation (e.g., a slope gain in dB per octave) applied thereto.
  • the attenuation may depend on a tonality metric (e.g., for the frequency range of the low frequency components of the input audio signal), e.g., so that a strong tonality results in a larger attenuation (in dB per octave) within each generated harmonic overtone.
  • a tonality metric e.g., for the frequency range of the low frequency components of the input audio signal
  • data indicative of the harmonics are energy adjusted (e.g., attenuated) in accordance with a control function which determines a gain to be applied to each hybrid sub-band of the transposed data.
  • the invention is a system or device (e.g., device having physically-limited or otherwise limited bass reproduction capabilities, such as, for example, a notebook, tablet, mobile phone, or other device with small speakers) configured to perform any embodiment of the inventive method on an input audio signal.
  • Device 200 of Fig. 5 is an example of such a device.
  • Device 200 includes a virtual bass synthesis subsystem 201, which is coupled to receive an input audio signal and configured to generate enhanced audio in response thereto in accordance with any embodiment of the inventive method, rendering subsystem 202, and left and right speakers (L and R), connected as shown.
  • Subsystem 201 may (but need not) have the structure and functionality of the above-described Fig. 2 or Fig. 4 embodiment of the invention.
  • Rendering subsystem 202 is configured to generate speaker feeds for speakers L and R in response to the enhanced audio signal generated in subsystem 201.
  • the inventive system is or includes a general or special purpose processor (e.g., an implementation of subsystem 201 of Fig. 5 , or an implementation of Fig. 2 or Fig. 4 ) programmed with software (or firmware) and/or otherwise configured to perform an embodiment of the inventive method.
  • the inventive system is a general purpose processor, coupled to receive input audio data, and programmed (with appropriate software) to generate output audio data in response to the input audio data by performing an embodiment of the inventive method.
  • the inventive system is a digital signal processor (e.g., an implementation of subsystem 201 of Fig. 5 , or an implementation of Fig. 2 or Fig. 4 ), coupled to receive input audio data, and configured (e.g., programmed) to generate output audio data in response to the input audio data by performing an embodiment of the inventive method.

Landscapes

  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • Stereophonic System (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Auxiliary Devices For Music (AREA)

Claims (15)

  1. Virtuelles Basserzeugungsverfahren, aufweisend einen Schritt des Vorverarbeitens von Samples eines Teilband-CQMF- (komplexwertiger Quadraturspiegelfilter) -Eingangsaudiosignals, um kritisch abgetastetes Audio zu erzeugen, das auf Inhalt von einem Satz von Niederfrequenzteilbändern des Eingangsaudiosignals hinweist, und das die Schritte aufweist:
    (a) Ausführen von Oberschwingungstransposition an dem kritisch abgetasteten Audio, um transponierte Daten zu erzeugen, die auf Oberschwingungen hinweisen, wobei erwartet wird, dass die Oberschwingungen während der Wiedergabe einer verbesserten Version des Eingangsaudio, das die Oberschwingungen aufweist, hörbar sind;
    (b) Erzeugen eines Verstärkungssignals als Reaktion auf die transponierten Daten; und
    (c) Erzeugen eines verbesserten Audiosignals durch Kombinieren des Verstärkungssignals mit dem Eingangsaudiosignal,
    wobei die Oberschwingungstransposition, die in Schritt (a) ausgeführt wird, kombinierte Transposition verwendet, sodass die Oberschwingungen eine Oberschwingung zweiter Ordnung und mindestens eine Oberschwingung höherer Ordnung von jeder der Niederfrequenzkomponenten aufweisen, und sodass alle Oberschwingungen als Reaktion auf Frequenzbereichswerte, die durch eine einzelne, gemeinsame Zeit-zu-Frequenzbereich-Umwandlungsstufe bestimmt werden, und eine anschließend ausgeführte Rückumwandlung, die durch eine einzelne, gemeinsame Frequenz-zu-Zeitbereich-Umwandlungsstufe bestimmt wird, erzeugt werden.
  2. Verfahren nach Anspruch 1, wobei das Eingangsaudiosignal auf Niederfrequenzaudioinhalt in einem Bereich von 0 bis B Hz hinweist, wobei B eine Zahl kleiner als 500 ist und das kritisch abgetastete Audio ein mindestens im Wesentlichen kritisch abgetastetes Signal ist, das auf den Niederfrequenzaudioinhalt hinweist.
  3. Virtuelles Basserzeugungsverfahren, aufweisend einen Schritt des Vorverarbeitens von Samples eines Eingangsaudiosignals, um kritisch abgetastetes Audio zu erzeugen, das auf Niederfrequenzkomponenten des Eingangsaudiosignals hinweist, und das die Schritte aufweist:
    (a) Ausführen von Oberschwingungstransposition an dem kritisch abgetasteten Audio, um transponierte Daten zu erzeugen, die auf Oberschwingungen hinweisen, wobei erwartet wird, dass die Oberschwingungen während der Wiedergabe einer verbesserten Version des Eingangsaudio, das die Oberschwingungen aufweist, hörbar sind;
    (b) Erzeugen eines Verstärkungssignals als Reaktion auf die transponierten Daten; und
    (c) Erzeugen eines verbesserten Audiosignals durch Kombinieren des Verstärkungssignals mit dem Eingangsaudiosignal,
    wobei die in Schritt (a) ausgeführte Oberschwingungstransposition kombinierte Transposition verwendet, sodass die Oberschwingungen eine Oberschwingung zweiter Ordnung und mindestens eine Oberschwingung höherer Ordnung von jeder der Niederfrequenzkomponenten aufweisen, und sodass alle Oberschwingungen als Reaktion auf Frequenzbereichswerte, die durch eine einzelne, gemeinsame Zeit-zu-Frequenzbereich-Umwandlungsstufe bestimmt werden, und eine anschließend ausgeführte Rückumwandlung, die durch eine einzelne, gemeinsame Frequenz-zu-Zeitbereich-Umwandlungsstufe bestimmt wird, erzeugt werden, wobei das kritisch abgetastete Audio ein CQMF-Kanal 0-Signal ist und das in Schritt (b) erzeugte Verstärkungssignal ein CQMF-Kanal 0-Verstärkungssignal und ein CQMF-Kanal 1-Verstärkungssignal aufweist.
  4. Virtuelles Basserzeugungsverfahren, das die Schritte aufweist:
    (a) Ausführen von Oberschwingungstransposition an Niederfrequenzkomponenten eines Eingangsaudiosignals, um transponierte Daten zu erzeugen, die auf Oberschwingungen hinweisen, wobei erwartet wird, dass die Oberschwingungen während der Wiedergabe einer verbesserten Version des Eingangsaudio, das die Oberschwingungen aufweist, hörbar sind;
    (b) Erzeugen eines Verstärkungssignals als Reaktion auf die transponierten Daten; und
    (c) Erzeugen eines verbesserten Audiosignals durch Kombinieren des Verstärkungssignals mit dem Eingangsaudiosignal,
    wobei die Oberschwingungstransposition, die in Schritt (a) ausgeführt wird, kombinierte Transposition verwendet, sodass die Oberschwingungen eine Oberschwingung zweiter Ordnung und mindestens eine Oberschwingung höherer Ordnung von jeder der Niederfrequenzkomponenten aufweisen, und sodass alle Oberschwingungen als Reaktion auf Frequenzbereichswerte, die durch eine einzelne, gemeinsame Zeit-zu-Frequenzbereich-Umwandlungsstufe bestimmt werden, und eine anschließend ausgeführte Rückumwandlung, die durch eine einzelne, gemeinsame Frequenz-zu-Zeitbereich-Umwandlungsstufe bestimmt wird, erzeugt werden,
    ebenfalls aufweisend den Schritt des Erzeugens der Niederfrequenzkomponenten durch Ausführen einer überabgetasteten Frequenzbereichsumwandlung an dem Eingangsaudiosignal durch Erzeugen gefensterter, mit Nullen aufgefüllter Samples und Ausführen einer Zeit-zu-Frequenzbereich-Umwandlung an den gefensterten, mit Nullen aufgefüllten Samples, um die Niederfrequenzkomponenten zu erzeugen, und wobei Schritt (b) einen Schritt des Aufteilens von verarbeiteten Frequenzkomponenten in einen ersten Satz von Frequenzkomponenten in einem ersten Frequenzband und einen zweiten Satz von Frequenzkomponenten in einem zweiten Frequenzband aufweist und das Ausführen einer ersten Frequenzzu-Zeitbereich-Umwandlung an dem ersten Satz von Frequenzkomponenten und einer zweiten Frequenz-zu-Zeitbereich-Umwandlung an dem zweiten Satz von Frequenzkomponenten, wobei jede der ersten Frequenz-zu-Zeitbereich-Umwandlung und der zweiten Frequenz-zu-Zeitbereich-Umwandlung eine Blockgröße aufweist, die kleiner ist als die Zeit-zu-Frequenzbereich-Umwandlung.
  5. Verfahren nach Anspruch 4, wobei das erste Frequenzband das Frequenzband von CQMF-Kanal 0 ist und das zweite Frequenzband das Frequenzband von CQMF-Kanal 1 ist, und optional
    wobei der erste Satz von Frequenzkomponenten und der zweite Satz von Frequenzkomponenten größenordnungskompensiert sind, um entsprechend Frequenzantworten von CQMF-Kanal 0 und CQMF-Kanal 1 zu berücksichtigen.
  6. Virtuelles Basserzeugungsverfahren, das die Schritte aufweist:
    (a) Ausführen von Oberschwingungstransposition an Niederfrequenzkomponenten eines Eingangsaudiosignals, um transponierte Daten zu erzeugen, die auf Oberschwingungen hinweisen, wobei erwartet wird, dass die Oberschwingungen während der Wiedergabe einer verbesserten Version des Eingangsaudio, das die Oberschwingungen aufweist, hörbar sind;
    (b) Erzeugen eines Verstärkungssignals als Reaktion auf die transponierten Daten; und
    (c) Erzeugen eines verbesserten Audiosignals durch Kombinieren des Verstärkungssignals mit dem Eingangsaudiosignal,
    wobei die in Schritt (a) ausgeführte Oberschwingungstransposition kombinierte Transposition verwendet, sodass die Oberschwingungen eine Oberschwingung zweiter Ordnung und mindestens eine Oberschwingung höherer Ordnung von jeder der Niederfrequenzkomponenten aufweisen, und sodass alle Oberschwingungen als Reaktion auf Frequenzbereichswerte, die durch eine einzelne, gemeinsame Zeit-zu-Frequenzbereich-Umwandlungsstufe bestimmt werden, und eine anschließend ausgeführte Rückumwandlung, die durch eine einzelne, gemeinsame Frequenz-zu-Zeitbereich-Umwandlungsstufe bestimmt wird, erzeugt werden, und
    wobei die Zeit-zu-Frequenzbereich-Umwandlung und die Rückumwandlung asymmetrische Analyse- und Synthesefenster verwenden.
  7. Verfahren nach Anspruch 1, ebenfalls aufweisend den Schritt des Erzeugens der Niederfrequenzkomponenten durch Ausführen einer überabgetasteten Frequenzbereichsumwandlung an dem Eingangsaudiosignal durch Erzeugen gefensterter, mit Nullen aufgefüllter Samples und Ausführen einer Zeit-zu-Frequenzbereich-Umwandlung an den gefensterten, mit Nullen aufgefüllten Samples, um die Niederfrequenzkomponenten zu erzeugen.
  8. Verfahren nach Anspruch 1, wobei das verbesserte Audiosignal einen erhöhten Wahrnehmungspegel des Bassinhaltes während der Wiedergabe des verbesserten Audiosignals durch mindestens einen Lautsprecher bereitstellt, der die Niederfrequenzkomponenten nicht physikalisch reproduzieren kann.
  9. Verfahren nach Anspruch 1, ebenfalls aufweisend einen Schritt der Wiedergabe des verbesserten Audiosignals durch Lautsprecher, welche die Niederfrequenzkomponenten nicht physikalisch reproduzieren können.
  10. Verfahren nach Anspruch 1, wobei die Niederfrequenzkomponenten des Eingangsaudiosignals Bassfrequenzkomponenten sind, von denen erwartet wird, dass sie während der Wiedergabe des Eingangsaudiosignals unter Verwendung eines erwarteten Lautsprechers oder Lautsprechersatzes unhörbar sind.
  11. Verfahren nach Anspruch 1, wobei die transponierten Daten auf amplitudenmodifizierte Versionen der Oberschwingungen, wie z. B. amplitudenmodifizierte Versionen der Oberschwingungen, hinweisen, deren Werte mindestens annähernd durch Kurven gleicher Lautstärke (ELCs) bestimmt werden.
  12. Verfahren nach Anspruch 1, wobei Schritt (a) einen Schritt des Abschwächens der Oberschwingungen auf eine Weise, die durch eine Tonalitätsmetrik bestimmt wird, aufweist, um die transponierten Daten zu bestimmen.
  13. Verfahren nach Anspruch 1, wobei mindestens einer von den Schritten (a) und (b) einen Schritt des Abschwächens von Daten aufweist, die auf die Oberschwingungen gemäß einer Steuerfunktion hinweisen, wobei die Steuerfunktion eine Verstärkung bestimmt, die auf jedes Frequenzteilband der transponierten Daten anzuwenden ist.
  14. Verfahren nach Anspruch 13, wobei die Steuerfunktion eine Verstärkung g(b) bestimmt, die auf Oberschwingungskoeffizienten in Frequenzteilband b anzuwenden ist, und die Form aufweist: g(b)= H[(G*nrgorig(b) nrgvb(b)) / (G*nrgorig(b) + nrgvb(b))] + B,
    wobei H, G und B Konstanten sind, nrgorig(b) auf Energie des Eingangsaudiosignals in dem Teilband b hinweist und nrgvb(b) auf Energie der transponierten Daten oder des Verstärkungssignals in dem Teilband b hinweist.
  15. Virtuelles Basserzeugungssystem, das konfiguriert ist, das virtuelle Basserzeugungsverfahren nach einem der vorstehenden Ansprüche auszuführen.
EP13188415.7A 2012-10-15 2013-10-14 Virtuelle Basssynthese mit harmonischer Transposition Active EP2720477B1 (de)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US13/652,023 US8971551B2 (en) 2009-09-18 2012-10-15 Virtual bass synthesis using harmonic transposition

Publications (2)

Publication Number Publication Date
EP2720477A1 EP2720477A1 (de) 2014-04-16
EP2720477B1 true EP2720477B1 (de) 2016-03-02

Family

ID=49293633

Family Applications (2)

Application Number Title Priority Date Filing Date
EP13771123.0A Active EP2907324B1 (de) 2012-10-15 2013-09-27 System und verfahren zur reduzierung der latenzzeit in transposerbasierten virtuellen basssystemen
EP13188415.7A Active EP2720477B1 (de) 2012-10-15 2013-10-14 Virtuelle Basssynthese mit harmonischer Transposition

Family Applications Before (1)

Application Number Title Priority Date Filing Date
EP13771123.0A Active EP2907324B1 (de) 2012-10-15 2013-09-27 System und verfahren zur reduzierung der latenzzeit in transposerbasierten virtuellen basssystemen

Country Status (4)

Country Link
EP (2) EP2907324B1 (de)
JP (1) JP5894347B2 (de)
CN (1) CN104704855B (de)
WO (1) WO2014060204A1 (de)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023280357A1 (en) * 2021-07-09 2023-01-12 Soundfocus Aps Method and loudspeaker system for processing an input audio signal
WO2023280356A1 (en) * 2021-07-09 2023-01-12 Soundfocus Aps Method and transducer array system for directionally reproducing an input audio signal
RU2819779C1 (ru) * 2020-03-20 2024-05-24 Долби Интернешнл Аб Усиление низких частот для громкоговорителей

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105280189B (zh) * 2015-09-16 2019-01-08 深圳广晟信源技术有限公司 带宽扩展编码和解码中高频生成的方法和装置
JP7270836B2 (ja) 2019-08-08 2023-05-10 ブームクラウド ３６０ インコーポレイテッド 音響心理学的周波数範囲拡張のための非線形適応フィルタバンク
CN115299075B (zh) * 2020-03-20 2023-08-18 杜比国际公司 扬声器的低音增强
JP2023130644A (ja) * 2022-03-08 2023-09-21 アルプスアルパイン株式会社 音響信号処理装置、音響システム及び低音感の増強方法

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
SE0101175D0 (sv) 2001-04-02 2001-04-02 Coding Technologies Sweden Ab Aliasing reduction using complex-exponential-modulated filterbanks
TWI339991B (en) * 2006-04-27 2011-04-01 Univ Nat Chiao Tung Method for virtual bass synthesis
US8036903B2 (en) * 2006-10-18 2011-10-11 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Analysis filterbank, synthesis filterbank, encoder, de-coder, mixer and conferencing system
JP4983694B2 (ja) * 2008-03-31 2012-07-25 株式会社Ｊｖｃケンウッド 音声再生装置
KR101589942B1 (ko) * 2009-01-16 2016-01-29 돌비 인터네셔널 에이비 외적 향상 고조파 전치
CN101505443B (zh) * 2009-03-13 2013-12-11 无锡中星微电子有限公司 一种虚拟重低音增强方法及系统
GB0906594D0 (en) * 2009-04-17 2009-05-27 Sontia Logic Ltd Processing an audio singnal
KR101613684B1 (ko) * 2009-12-09 2016-04-19 삼성전자주식회사 음향 신호 보강 처리 장치 및 방법
US8638953B2 (en) * 2010-07-09 2014-01-28 Conexant Systems, Inc. Systems and methods for generating phantom bass
PL3288032T3 (pl) * 2010-07-19 2019-08-30 Dolby International Ab Przetwarzanie sygnałów audio podczas rekonstrukcji wysokiej częstotliwości
JP5375861B2 (ja) * 2011-03-18 2013-12-25 ヤマハ株式会社 オーディオ再生の効果付加方法およびその装置
CN102354500A (zh) * 2011-08-03 2012-02-15 华南理工大学 一种基于谐波控制的虚拟低音增强处理方法
TWI575962B (zh) * 2012-02-24 2017-03-21 杜比國際公司 部份複數處理之重疊濾波器組中的低延遲實數至複數轉換

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2819779C1 (ru) * 2020-03-20 2024-05-24 Долби Интернешнл Аб Усиление низких частот для громкоговорителей
WO2023280357A1 (en) * 2021-07-09 2023-01-12 Soundfocus Aps Method and loudspeaker system for processing an input audio signal
WO2023280356A1 (en) * 2021-07-09 2023-01-12 Soundfocus Aps Method and transducer array system for directionally reproducing an input audio signal

Also Published As

Publication number Publication date
JP2015531575A (ja) 2015-11-02
CN104704855B (zh) 2016-08-24
EP2907324A1 (de) 2015-08-19
EP2907324B1 (de) 2016-11-09
JP5894347B2 (ja) 2016-03-30
CN104704855A (zh) 2015-06-10
EP2720477A1 (de) 2014-04-16
WO2014060204A1 (en) 2014-04-24

Similar Documents

Publication Publication Date Title
US8971551B2 (en) Virtual bass synthesis using harmonic transposition
US20230074883A1 (en) Apparatus and method for processing an input audio signal using cascaded filterbanks
US10043526B2 (en) Harmonic transposition in an audio coding method and system
EP2720477B1 (de) Virtuelle Basssynthese mit harmonischer Transposition
US9640187B2 (en) Method and an apparatus for processing an audio signal using noise suppression or echo suppression
JP4934427B2 (ja) 音声信号復号化装置及び音声信号符号化装置
US8886346B2 (en) Oversampling in a combined transposer filter bank
EP2334103B1 (de) Vorrichtung und verfahren zur tonverbesserung
US20060053018A1 (en) Advanced processing based on a complex-exponential-modulated filterbank and adaptive time signalling methods
KR20090028755A (ko) 압축된 서브밴드 필터 임펄스 응답들을 형성하는 필터 압축기 및 방법
JP2007011341A (ja) 高調波信号の周波数拡張
US9514767B2 (en) Device, method and computer program for freely selectable frequency shifts in the subband domain
SG183966A1 (en) Improved magnitude response and temporal alignment in phase vocoder based bandwidth extension for audio signals
EP2476115A1 (de) Verfahren und vorrichtung zur verarbeitung von tonsignalen
JP7260101B2 (ja) 情報処理装置、これを用いたミキシング装置、及びレイテンシ減少方法
US8788277B2 (en) Apparatus and methods for processing a signal using a fixed-point operation
JP2023518794A (ja) スピーカのための低音強調
RU2782168C1 (ru) Система и способ для генерирования ряда сигналов высокочастотных поддиапазонов
RU2805938C1 (ru) Система и способ для генерирования ряда сигналов высокочастотных поддиапазонов
EP3783912B1 (de) Mischvorrichtung, mischverfahren und mischprogramm

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17P Request for examination filed

Effective date: 20141016

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 21/02 20130101ALI20150715BHEP

Ipc: G10L 21/038 20130101ALI20150715BHEP

Ipc: H04R 3/04 20060101AFI20150715BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20150918

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 778700

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160315

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602013005230

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20160302

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 778700

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160302

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160603

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160602

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 4

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160702

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160704

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602013005230

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

26N No opposition filed

Effective date: 20161205

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160602

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161031

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161014

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161014

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20131014

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161031

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160302

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 602013005230

Country of ref document: DE

Owner name: DOLBY INTERNATIONAL AB, IE

Free format text: FORMER OWNER: DOLBY INTERNATIONAL AB, AMSTERDAM, NL

Ref country code: DE

Ref legal event code: R081

Ref document number: 602013005230

Country of ref document: DE

Owner name: DOLBY INTERNATIONAL AB, NL

Free format text: FORMER OWNER: DOLBY INTERNATIONAL AB, AMSTERDAM, NL

REG Reference to a national code

Ref country code: DE

Ref legal event code: R081

Ref document number: 602013005230

Country of ref document: DE

Owner name: DOLBY INTERNATIONAL AB, IE

Free format text: FORMER OWNER: DOLBY INTERNATIONAL AB, DP AMSTERDAM, NL

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230512

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230920

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230920

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230920

Year of fee payment: 11