EP2612437A1 - Spectrally uncolored optimal croostalk cancellation for audio through loudspeakers - Google Patents

Spectrally uncolored optimal croostalk cancellation for audio through loudspeakers

Info

Publication number
EP2612437A1
EP2612437A1 EP11825672.6A EP11825672A EP2612437A1 EP 2612437 A1 EP2612437 A1 EP 2612437A1 EP 11825672 A EP11825672 A EP 11825672A EP 2612437 A1 EP2612437 A1 EP 2612437A1
Authority
EP
European Patent Office
Prior art keywords
audio
frequency
xtc
loudspeakers
audio signals
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP11825672.6A
Other languages
German (de)
French (fr)
Other versions
EP2612437B1 (en
EP2612437A4 (en
Inventor
Edgar Y. Choueiri
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Princeton University
Original Assignee
Princeton University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Princeton University filed Critical Princeton University
Priority claimed from PCT/US2011/050181 external-priority patent/WO2012036912A1/en
Publication of EP2612437A1 publication Critical patent/EP2612437A1/en
Publication of EP2612437A4 publication Critical patent/EP2612437A4/en
Application granted granted Critical
Publication of EP2612437B1 publication Critical patent/EP2612437B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00Circuits for transducers, loudspeakers or microphones
    • H04R3/04Circuits for transducers, loudspeakers or microphones for correcting frequency response
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04SSTEREOPHONIC SYSTEMS 
    • H04S1/00Two-channel systems
    • H04S1/002Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2430/00Signal processing covered by H04R, not provided for in its groups
    • H04R2430/03Synergistic effects of band splitting and sub-band processing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00Circuits for transducers, loudspeakers or microphones
    • H04R3/12Circuits for transducers, loudspeakers or microphones for distributing signals to two or more loudspeakers
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00Stereophonic arrangements
    • H04R5/04Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04SSTEREOPHONIC SYSTEMS 
    • H04S2420/00Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/01Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]

Definitions

  • Binaural audio with loudspeakers also known as transauralization, aims to reproduce, at the entrance of each of the listener's ear canals, the sound pressure signals recorded on only the ipsilateral channel of a stereo signal. That is, only the sound signal of the left stereo channel is reproduced at the left ear and only the sound signal of the right stereo channel is reproduced at the right ear.
  • the source signal was encoded with a head-related transfer function (HRTF) of the listener, or includes the proper interaural time difference (ITD) and interaural level difference (ILD) cues
  • HRTF head-related transfer function
  • ITD interaural time difference
  • ILD interaural level difference
  • Crosstalk occurs when the left ear (right ear) hears sounds from the right (left) audio channel, originating from the light speaker (left speaker). In other words, crosstalk occurs when the sound on one of the stereo channels is heard by the contralateral ear of the listener.
  • Crosstalk corrupts HRTF information and ITD or ILD cues so that a listener may not properly or completely comprehend the soundfield's binaural cues that are embedded in the recording. Therefore, approaching the goal of BAL requires an effective cancellation of this unintended crosstalk, i.e. crosstalk cancellation or XTC for short.
  • Dl Severe spectral coloration to the sound heard by the listener, even if that listener is sitting in the intended sweet spot.
  • D3 Severe dynamic range loss when the sound is processed through the XTC filter or processor (while avoiding distortion and/or clipping).
  • a method and system for calculating the frequency-dependent regularization parameter (FDRP) used in inverting the analytically derived or experimentally measured system transfer matrix for crosstalk cancellation (XTC) filter design is described.
  • the method relies on calculating the FDRP that results in a flat amplitude vs frequency response at the loudspeakers (as opposed to a flat amplitude vs frequency response at the ears of the listener, as inherently done in prior art methods) thus forcing XTC to be effected into the phase domain only and relieving the XTC filter from the drawbacks of audible spectral coloration and dynamic range loss.
  • XTC filters that yield optimal XTC levels over any desired portion of the audio band, impose no spectral coloration on the processed sound beyond the spectral coloration inherent in the playback hardware and/or loudspeakers, and cause no dynamic range loss.
  • XTC filters designed with this method and used in the system are not only optimal but, due to their being free from Drawbacks Dl , D2 and D3, allow for a most natural and spectrally transparent 3D audio reproduction of binaural or stereo audio through loudspeakers.
  • the method and system do not attempt to correct the spectral characteristics of the playback hardware, and therefore are best suited for use with audio playback hardware and loudspeakers that are designed to meet a desired spectral fidelity level without the help of additional signal processing for spectral correction.
  • FIG. 1 is a diagram of a listener and a two-source model
  • FIG. 2 is a plot of the frequency responses of the perfect XTC filter at the loudspeakers
  • FIG. 3 is a plot showing the effects of regularization on the envelope spectrum at the loudspeakers
  • FIG. 4 shows the effects of regularization on the crosstalk cancellation spectrum
  • FIG. 5 is a plot showing the envelope spectrum at the loudspeakers
  • FIG. 6 is a flow chart of the method of the present invention.
  • FIG. 7 shows four (windowed) measured impulse responses (IR) representing the transfer function in the time domain.
  • FIG. 8 is a graph showing measured spectra associated with a perfect XTC filter
  • FIG. 9 is a graph showing measured spectra for an XTC filter of the present invention.
  • an idealized situation consisting of two point sources (idealized loudspeakers) 12, 14 in free space (no sound reflections) and two listening points 16, 18 corresponding to the location of the ears of an idealized listener 20 (no HRTF).
  • actual data corresponding to the impulse responses of real loudspeakers in a real room measured at the ear canal entrances of a dummy head will be used.
  • /, and / 2 are the path lengths between any of the two sources 12, 14 and the ipsilateral and contralateral ear, respectively, as shown in Figure 1.
  • uppercase letters represent frequency variables
  • lowercase represent time-domain variables
  • uppercase bold letters represent matrices
  • lowercase bold letters represent vectors
  • is the effective distance between the entrances of the ear canals, and / is the distance between either source and the interaural mid-point of the listener.
  • 2 ⁇ is the loudspeaker span.
  • Arsin( ⁇ ) as in many loudspeaker-based listening set-ups, which leads to Another important parameter is the time delay, defined as the time it takes a sound wave to traverse the path length difference ⁇ / .
  • the received signal at the listener's left ear 16 and the received signal at the listener's right ear 18 may be written in vector form as:
  • a (8) which, in the time domain, is a transmission delay (divided by the constant /, ) that does not affect the shape of the received signal.
  • R i.e., R LL (ico) and R I ⁇ R (ico)
  • R LL (ico) and R I ⁇ R (ico) represent the ipsilateral transmission of the recorded sound signal to the ears
  • off-diagonal elements i.e., R RL (ioj) and R l R ⁇ ico)
  • R RL (ioj) and R l R ⁇ ico) represent the undesired contralateral transmission, i.e., the crosstalk.
  • S(a>) is the envelope spectrum that describes the maximum amplitude that could be expected at the loudspeakers, and is given by
  • a perfect crosstalk cancellation (P-XTC) filter may be defined as one that, theoretically, yields infinite crosstalk cancellation at the ears of the listener, for all frequencies.
  • the spectra has a frequency varying behavior at the sources
  • Fig. 2 shows the frequency responses of a Perfect XTC filter at the loudspeakers: amplitude envelope (curve 22), side image (curve 24), and central image (curve 26).
  • the envelope peaks correspond to a boost of
  • the peaks and minima in the condition number occur at the same frequencies as those of the amplitude envelope spectrum at the loudspeakers, $ m .
  • the minima have a condition number of unity (the lowest possible value), which implies that the XTC filter resulting from the inversion of C is most robust (i.e., least sensitive to errors in the transfer
  • the matrix inversion resulting in the P- XTC filter becomes ill-conditioned, or in other words, infinitely sensitive to errors.
  • the sliglitest misalignment for instance, of the listener's head, would thus result in a severe loss in XTC control at the ears (at and near these frequencies) which, in turn, causes the severe spectral coloration in S l?i (fe , ') to be transmitted to the ears.
  • Regularization methods allow controlling the norm of the approximate solution of an ill-conditioned linear system at the price of some loss in the accuracy of the solution.
  • the control of the norm through regularization can be done subject to an optimization prescription, such as the minimization of a cost function.
  • Regularization may be discussed analytically in the context of XTC filter optimization, which may be defined as the maximization of XTC performance for a desired tolerable level of spectral coloration or, equivalently, the minimization of spectral coloration for a desired minimum XTC performance.
  • the envelope spectrum, 5 ⁇ ( ⁇ 3, is plotted in Fig. 3 for three values of ⁇ .
  • Two features can be noted in that plot: 1 ) increasing the regularization parameter attenuates the peaks in the spectrum without affecting the minima, and 2) with increasing ⁇ the spectral maxima split into doublet peaks (two closely-spaced peaks).
  • the first and second derivatives ⁇ are used to find the conditions for which the first derivative is nil and the second is negative. These conditions are summarized as follows: If ⁇ is below a threshold ⁇ " defined as
  • the peaks are singlets and occur at the same non-dimensional frequencies as for the envelope spectrum peaks of the P-XTC filter (J l j ), and have the following amplitude:
  • the method and system of the present invention rely on the use of a specific scheme for calculating the frequency-dependent regularization parameter (FDRP) that would result in the flattening of the amplitude vs frequency spectrum measured at the loudspeakers and not at the ears of the listeners as is implicit in previous XTC filter designs that are based on the inversion of the system transfer matrix.
  • FDRP frequency-dependent regularization parameter
  • a frequency-dependent regularization parameter is calculated that would cause the envelope spectrum 5(a ) ) to be flat at a desired level ⁇ (in dB) over the frequency bands where the perfect filter's envelope spectrum exceeds ⁇ . Outside these bands (i.e., where the S i P] (o)) is below ⁇ ), we apply no regularization. This can be stated symbolically as:
  • step 30 the system's transfer matrix in the frequency domain (i.e. matrix C as in
  • Eq. (12) and the input 28) is inverted, either analytically (if it results from a tractable idealized model) or numerically (if it results from experimental measurements), using zero or a very small constant regularization parameter (large enough to avoid machine inversion problems) to obtain the corresponding perfect XTC filter, H 1 '' 1 .
  • Step 40 the FDRP thus obtained, ⁇ ( ⁇ ) , is used to calculate the pseudo-inverse of the system's transfer matrix (e.g. according to Eqn. (22)), which yields the sought regularized optimal XTC filter H t ?1 that has a flat frequency response at the loudspeakers.
  • a time domain version (impulse response) of the filter is obtained in step 44 by simply taking the inverse Fourier transform of H [ ?1 (output 42) .
  • Fig. 7 shows the four (windowed) measured impulse responses (IR) representing the transfer function in the time domain.
  • the x-axis of each plot in Fig. 7 is time in ms, and the_y- axis is the normalized amplitude of the measured signal.
  • the top left plot shows the IR of the left loudspeaker measured at the left ear of the dummy head, and the bottom left plot shows the IR of the left loudspeaker measured at the right ear of the dummy head.
  • the top right plot is the ER. of the right speaker -left ear transfer function and the bottom plot is the IR of the right speaker - right ear transfer function.
  • Fig 8 shows relevant spectra where the x-axis is frequency in Hz and the _y-axis is amplitude in dB.
  • the curve 48 in that plot is the frequency response C LL that corresponds to the left speaker-left ear transfer function in the frequency domain obtained by panning the test sound completely to the left channel.
  • the ripples in curve 48 above 5 kHz are due to the HRTF of the head and the left ear pinna.
  • Curve 50 is the response at the left loudspeaker and shows a dynamic range loss of 31.45 dB (difference between the maximum and minimum in that curve).
  • Curve 52 is the frequency response at the left (ipsilateral) ear, E SJ , which, as expected from a perfect XTC filter, is essentially flat over the entire audio band.
  • the curve 54 is the corresponding frequency response measured at the right (contralateral) ear, E S( . , and shows significant attenuation with respect to curve 52 due to XTC.
  • the difference in amplitude between the curves 52 and 54 linearly averaged over frequencies is the average XTC level, which for this case is 21.3 dB.
  • the filter designed with the method of the present invention imposes no audible coloration to the sound of the playback system, has no dynamic range loss, and yields an XTC level that is essentially the same as that of a perfect XTC filter.
  • the method described herein may be implemented in software, or firmware incorporated in a computer-readable storage medium for execution by a general purpose computer or a processor, such as a DSP chipset.
  • suitable computer-readable storage mediums include a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).
  • Embodiments of the present invention may be represented as instructions and data stored in a computer-readable storage medium.
  • aspects of the present invention may be implemented using Verilog, which is a hardware description language (HDL).
  • Verilog data instructions may generate other intermediary data, (e.g., netlists, GDS data, or the like), that may be used to perform a manufacturing process implemented in a semiconductor fabrication facility.
  • the manufacturing process may be adapted to manufacture semiconductor devices (e.g., processors) that embody various aspects of the present invention.
  • Suitable processors include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, a graphics processing unit (GPU), a DSP core, a controller, a microcontroller, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) , any other type of integrated circuit (IC), and/or a state machine, or combinations thereof.
  • DSP digital signal processor
  • GPU graphics processing unit
  • DSP core DSP core
  • controller a microcontroller
  • ASICs application specific integrated circuits
  • FPGAs field programmable gate arrays

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Stereophonic System (AREA)

Abstract

A method and system for calculating the frequency-dependent regularization parameter (FDRP) used in inverting the analytically derived or experimentally measured system transfer matrix for designing and/or producing crosstalk cancellation (XTC) filters relies on calculating the FDRP that results in a flat amplitude vs frequency response at the loudspeakers, thus forcing XTC to be effected into the phase domain only and relieving the XTC filter from the drawbacks of audible spectral coloration and dynamic range loss. When the method and system are used with any effective optimization technique, it results in XTC filters that yield optimal XTC levels over any desired portion of the audio band, impose no spectral coloration on the processed sound beyond the spectral coloration inherent in the playback hardware and/or loudspeakers, and cause no (or arbitrarily low) dynamic range loss.

Description

SPECTRALLY UNCOLORED OPTIMAL CROSSTALK CANCELLATION FOR
AUDIO THROUGH LOUDSPEAKERS
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. provisional application No.
61/379,831 entitled "OPTIMAL CROSSTALK CANCELLATION FOR BINAURAL AUDIO WITH TWO LOUDSPEAKERS" filed on September 3rd, 2010, the contents of which are hereby incorporated by reference herein.
BACKGROUND
[0002] Binaural audio with loudspeakers (BAL), also known as transauralization, aims to reproduce, at the entrance of each of the listener's ear canals, the sound pressure signals recorded on only the ipsilateral channel of a stereo signal. That is, only the sound signal of the left stereo channel is reproduced at the left ear and only the sound signal of the right stereo channel is reproduced at the right ear. For example, if the source signal was encoded with a head-related transfer function (HRTF) of the listener, or includes the proper interaural time difference (ITD) and interaural level difference (ILD) cues, then delivering the signal on each of the channels of the stereo signal to the ipsilateral ear, and only to that ear, would ideally guarantee that the car-brain system receives the cues it needs to hear an accurate 3 -dimensional (3-D) reproduction of a recorded soundfield.
[0003] However, an unintended consequence of binaural audio playback through loudspeakers is crosstalk. Crosstalk occurs when the left ear (right ear) hears sounds from the right (left) audio channel, originating from the light speaker (left speaker). In other words, crosstalk occurs when the sound on one of the stereo channels is heard by the contralateral ear of the listener.
[0004] Crosstalk corrupts HRTF information and ITD or ILD cues so that a listener may not properly or completely comprehend the soundfield's binaural cues that are embedded in the recording. Therefore, approaching the goal of BAL requires an effective cancellation of this unintended crosstalk, i.e. crosstalk cancellation or XTC for short.
[0005] While there are various techniques for effecting some level of crosstalk cancellation
(XTC) for a two loudspeaker system, they all have one or more of the following drawbacks: Dl : Severe spectral coloration to the sound heard by the listener, even if that listener is sitting in the intended sweet spot.
D2: Useful XTC levels are reached only at limited frequency ranges of the audio band.
D3: Severe dynamic range loss when the sound is processed through the XTC filter or processor (while avoiding distortion and/or clipping).
[0006] The above drawbacks can be seen by analyzing XTC using the most fundamental formulation of the XTC problem - that is by looking at the inverse of the system transfer matrix (as will be shown and discussed below) that describes sound propagation from the loudspeakers to the ears of the listener.
[0007] While the technique of constant parameter (non-frequency dependent) regularization, commonly used in XTC filter design to make the inversion of the system transfer matrix better behaved, may alleviate some of Drawback D3, it inherently introduces spectral artifice of its own (specifically, at the expense of reducing the amplitude of the spectral peaks in the inverted transfer matrix, constant-parameter regularization results in undesirable narrow-band artifacts at higher frequencies and arolloff at lower frequencies at the loudspeakers) and does little to alleviate the other two drawbacks (Dl and D2).
[0008] Prior art frequency-dependent regularization, even when coupled with an effective optimization scheme, is not enough to deal away with Drawbacks Dl, D2 and D3.
[0009] Previous XTC filter design methods based on system transfer matrix inversion
(with or without regularization) strive to maintain a flat amplitude vs. frequency response at the ears of the listener by imposing a non-flat amplitude vs frequency response at the loudspeakers (as explained below), which causes a loss in the dynamic range of the processed sound, and, for reasons that will be explained below, leads to a spectral coloration of the sound as heard by the listener, even if the listener is sitting in the intended sweet spot.
[0010] Therefore, while previous methods are useful for designing XTC filters that can inherently correct for non-idealities in the amplitude vs frequency response of the playback hardware and loudspeakers, they do not address all of Drawbacks Dl , D2 and D3. SUMMARY
[0011] A method and system for calculating the frequency-dependent regularization parameter (FDRP) used in inverting the analytically derived or experimentally measured system transfer matrix for crosstalk cancellation (XTC) filter design is described. The method relies on calculating the FDRP that results in a flat amplitude vs frequency response at the loudspeakers (as opposed to a flat amplitude vs frequency response at the ears of the listener, as inherently done in prior art methods) thus forcing XTC to be effected into the phase domain only and relieving the XTC filter from the drawbacks of audible spectral coloration and dynamic range loss. When the method is used with any effective optimization scheme it results in XTC filters that yield optimal XTC levels over any desired portion of the audio band, impose no spectral coloration on the processed sound beyond the spectral coloration inherent in the playback hardware and/or loudspeakers, and cause no dynamic range loss. XTC filters designed with this method and used in the system are not only optimal but, due to their being free from Drawbacks Dl , D2 and D3, allow for a most natural and spectrally transparent 3D audio reproduction of binaural or stereo audio through loudspeakers. The method and system do not attempt to correct the spectral characteristics of the playback hardware, and therefore are best suited for use with audio playback hardware and loudspeakers that are designed to meet a desired spectral fidelity level without the help of additional signal processing for spectral correction.
DESCRIPTION OF THE DRAWINGS
[0012] A more detailed understanding of the present invention may be had from the following detailed description which should read in light of the accompanying drawings wherein:
[0013] FIG. 1 is a diagram of a listener and a two-source model;
[0014] FIG. 2 is a plot of the frequency responses of the perfect XTC filter at the loudspeakers,
[0015] FIG. 3 is a plot showing the effects of regularization on the envelope spectrum at the loudspeakers,
[0016] FIG. 4 shows the effects of regularization on the crosstalk cancellation spectrum,
[0017] FIG. 5 is a plot showing the envelope spectrum at the loudspeakers, [0018] FIG. 6 is a flow chart of the method of the present invention.
[0019] FIG. 7 shows four (windowed) measured impulse responses (IR) representing the transfer function in the time domain.
[0020] FIG. 8 is a graph showing measured spectra associated with a perfect XTC filter
[0021] FIG. 9 is a graph showing measured spectra for an XTC filter of the present invention.
DETAILED DESCRIPTION
[0022] In order to explain the advantages of the method and system of the present invention an analytical formulation of the fundamental XTC problem in an idealized situation will be described and the "perfect XTC filter" will be defined, which will serve as a benchmark illustrating the severe problem of audible spectral coloration inherent to all XTC filters.
[0023] In the following description, for the sake of clarity and to allow analytical insight, an idealized situation will be used consisting of two point sources (idealized loudspeakers) 12, 14 in free space (no sound reflections) and two listening points 16, 18 corresponding to the location of the ears of an idealized listener 20 (no HRTF). However, in the example given following the description of the invention, , actual data corresponding to the impulse responses of real loudspeakers in a real room measured at the ear canal entrances of a dummy head will be used.
Formulation of the Fundamental XTC Problem
[0024] In the frequency domain, the air pressure at a free-field point located a distance r from a point source (monopole) radiating a sound wave of frequency ω , under the idealizing assumptions that sound propagation occurs in a free field (with no diffraction or reflection from the head and pinnae of the listener or any other physical objects), and that the loudspeakers radiate like point sources, is given by:
ia>p0q e~ikr
P{r, ia>) = -f* ,
Απ r
where pa is the air density, k = 2πΙ λ = ωΙ cs is the wavenumber, λ is the wavelength, cs is the speed of sound (340.3 m/s), and q is the source strength (in units of volume per unit time). Defining the mass flow rate of air from the center of the source, V, as:
v - icoP^
4π which is the time derivative of , in the symmetric two-source geometry shown in Figure 1 the n
air pressure due to the two sources 12, 14, under the above stated assumptions, add up as
PL M = - - VL {ico) +——VR {ico). ( 1 )
Similarly, at the right ear 18 of the listener 20 the following is the sensed pressure:
^ M = -r- VL {ico) +—-—VR {ico). (2)
Here, /, and /2 are the path lengths between any of the two sources 12, 14 and the ipsilateral and contralateral ear, respectively, as shown in Figure 1.
[0025] Throughout this specification, uppercase letters represent frequency variables, lowercase represent time-domain variables, uppercase bold letters represent matrices, and lowercase bold letters represent vectors, and define
Δ/≡/, - /, and g≡l l2 (3)
as the path length difference and path length ratio, respectively.
[0026] Because the contralateral distance in the geometry of Fig. 1 is greater than the ipsilateral distance, then 0 < g < 1 . Further, from the geometry in Figure 1 , the two distances may be expressed as:
ΔΓ
- Ar/sin(6>), (4)
where ΔΓ is the effective distance between the entrances of the ear canals, and / is the distance between either source and the interaural mid-point of the listener. As defined in Fig. 1 , Θ = 2Θ is the loudspeaker span. Note that for / » Arsin(< ) , as in many loudspeaker-based listening set-ups, which leads to Another important parameter is the time delay, defined as the time it takes a sound wave to traverse the path length difference Δ/ .
[0027] Using equations (1) and (2), the received signal at the listener's left ear 16 and the received signal at the listener's right ear 18 may be written in vector form as:
p - aCv
where a (8) which, in the time domain, is a transmission delay (divided by the constant /, ) that does not affect the shape of the received signal. The source vector at the loudspeaker comprising a left channel, VL , and a right channel, VR , is written in vector form as v = [ , (ico), VR (i<:o) . v may be obtained from the two channels of "recorded" signals, denoted d = pL (ico),DR (ia)) , using the transformation
v = Hd (9) where
is the sought 2 x 2 filter or transformation matrix for XTC. Therefore, from Eq. (7), the following result may be obtained
p = aCHd ( 1 1 ) where p = [Ρ1 (ίω), Ρ1{(ίω)]ί is the vector of pressures at the ears, and C is the system's transfer matrix
~ιωτ
1 ge
(12) ge l
which is symmetric due to the symmetry of the geometry shown in Fig. I .
[0028] In summary, the transformation from the signal d , through the filter H , to the source variables v , then tlirough wave propagation from the loudspeaker sources to pressure, p , at the ears of the listener, can be written as p - aCHd
R (13) p = aRd
where the performance matrix, R, is defined as
RLL (ico) R. ico)
= CH (14)
lRRL (ico) RRR {ico)_
[0029] The diagonal elements of R (i.e., RLL (ico) and RI{R (ico) ) represent the ipsilateral transmission of the recorded sound signal to the ears, and the off-diagonal elements (i.e., RRL (ioj) and Rl R {ico) ) represent the undesired contralateral transmission, i.e., the crosstalk.
Performance Metrics
[0030] A set of metrics by which to judge the spectral coloration and performance of XTC filters will now be described. The amplitude spectrum (to a factor s ) of a signal fed to only one (either left or right) of the two inputs of the system, as heard at the ipsilateral ear is
Esii °)≡
where the subscripts "si" and 11 stand for "side image" and "ipsilateral ear (with respect to the input signal)", respectively, since Esj.. , as defined, is the frequency response (at the ipsilateral ear) for the side image that would result from the input being panned to one side. Similarly, at the contralateral ear to the input signal (subscript X), the following is the side-image frequency response:
'·.·, )= H^H
The system's frequency response at either ear when the same signal is split equally between left and right inputs is another spectral coloration metric:
Here the subscript "ci" stands for "center image" since Eci , as defined, is the frequency response (at either ear) for the center image that would result from the input being panned to the center.
[0031] Also of importance are the frequency responses that would be measured at the sources (i.e., the loudspeakers), which are denoted by S and may be obtained from the elements of the filter matrix H : HlL (iO)) + HLR {ico) HRL (ico) + HRR (i o)
They are given using the same subscript convention used with the amplitude spectrum above (with "II" and "A"' referring to the loudspeakers that are ipsilateral and contralateral to the input signal, respectively). An intuitive interpretation of the significance of the above metrics is that a signal panned from a single input to both inputs to the system will result in frequency responses going from Esj to Ecj at the ears, and Ssj to Scj at the loudspeakers.
[0032] Two other spectral coloration metrics are the frequency responses of the system to in-phase and out-of-phase inputs to the system. These two responses are given by:
5,.(<a)≡
S0(a))≡
The subscripts i and o denote the in-phase and out-of-phase responses, respectively. Note that, as defined, 5,. is double (i.e., 6 dB above) Sci , as the latter describes a signal of amplitude 1 panned to center (i.e., split equally between L and R inputs), while the former describes two signals of amplitude 1 fed in phase to the two inputs of the system.
[0033] Since a real signal can comprise various components having different phase relationships, it is useful to combine S^co) and S0(co) into a single metric, S(a>) , which is the envelope spectrum that describes the maximum amplitude that could be expected at the loudspeakers, and is given by
S( )≡max[Si(oj),S0( )]. It is relevant to note that S(oS) is equivalent to the 2-norm of H, ||H|| , and that S, and S0 are the two singular values of H.
[0034] Finally, an important metric that will allow for the evaluation and comparison of the XTC performance of various filters is ώ) , the crosstalk cancellation spectrum:
It is the ratio of the amplitude spectrum at the ipsilateral ear to the amplitude spectrum at the contralateral ear and, therefore, the greater the value of the crosstalk cancellation spectrum, χ{ω) , the more effective is the crosstalk cancellation filter. The above definitions give a total of eight metrics, (Esiii,Esjx,Ecj,Ss i,Ssix ,Scj,S,z), all real functions of frequency, by which to evaluate and compare the spectral coloration and XTC performance of XTC filters.
Benchmark: Perfect Crosstalk Cancellation
[0035] A perfect crosstalk cancellation (P-XTC) filter may be defined as one that, theoretically, yields infinite crosstalk cancellation at the ears of the listener, for all frequencies. Crosstalk cancellation requires that the received signal at each of the two ears be that which would have resulted from the ipsilateral signal alone. Therefore, in order to achieve perfect cancellation of the crosstalk, Eq. (13) requires that R = CH = I , where I is the unity matrix (identity matrix), and thus, as per the definition of R in Eq. (14) , the P-XTC filter is the inverse of the system transfer matrix expressed in Eq. (12), and may be expressed exactly: where the superscript p] denotes perfect XTC. For this filter, the eight metrics defined above become:
_i. ^o- pin = 1
'
ge g
¾ )=
1-gV jg4-2g2cos(2ojTc) + l
1
(16)
2 ~2gcos(or) + l
χ ω) =∞
(17)
[0036] The perfect XTC filter (χΙΡι =∞ ) gives flat frequency responses at the ears (as evidenced by the constant E[p] , E[P] , and E[ ]) and is effective at canceling crosstalk as evidenced by El - 0 , while preserving the ipsilateral signal as evidenced by an amplitude spectrum of 1 , E[PJ - 1 . However, the spectra has a frequency varying behavior at the sources
( {a>), Sl (co), and Sl P](a>) ) that constitute severe spectral coloration, which, as we shall see below, only in an ideal world (i.e. under the idealized assumptions of the model) is not heard at the ears.
[0037] The extent of spectral coloration at the loudspeakers is plotted in Fig. 2 which shows the frequency responses of a Perfect XTC filter at the loudspeakers: amplitude envelope (curve 22), side image (curve 24), and central image (curve 26). The dotted horizontal line marks the envelope ceiling, which for this case (g=.985) is 36.5 dB. The non-dimensional frequency ω I TC is given on the bottom axis, and the corresponding frequency in Hz, shown on the top axis, is to illustrate a particular (typical) case of zc=3 samples at the redbook CD sampling rate of 44.1 kHz. (which would be the case, for instance, of a set-up with = 15 Cm, ϊ = 1·6 m, and Θ = 18· _)
[0038] The peaks in the Ss lPJ( ), and S[ p] (a>) spectra occur shown in Fig.
2 at frequencies for which the amplitude of the signal at the loudspeakers must be boosted in order to effect XTC at the ears while compensating for the destructive interference at that location. Similarly, minima in the spectra occur when the amplitude must be attenuated due to constructive interference.
[0039] Using the first and second derivatives (with respect to ωτε ) of the expressions for the various spectra, the amplitudes and frequencies for the associated peaks, denoted by the superscript * , and minima, denoted by the superscript , , are given by:
1
si n & ηττ ith n
Q
six ηττ , with n
,[P] t 1
ci tit
1 with II
- ¾
.[P] l 1.
b ci = .
£> at O,TC = nw, with ». = 0, 2, 4. 6, , . .
,[P] 1 1 t , = nx. with n = 0, 1. 2. 3, 4, , . .
1 _
5 <[P] < 1
= , at U,T„ = n~ 2 . with ?i = 1. 3. 5,. 7, . . .
V /.l + f?2
( 19)
[0040] For a typical listening set-up, g » l , say, a reference ,·? = -985 case shown in Fig. 2,
. h"'] !
the envelope peaks (i.e., ) correspond to a boost of
1 - .
(and the peaks in the other spectra, " c*i ' , correspond to boosts of about
30.5 dB.) While these boosts have equal fi-equency widths across the spectmm, when the spectrum is plotted logarithmically (as is appropriate for human sound perception), the low-frequency boost is most prominent in its perceived frequency extent. This low fi-equency (i.e., bass boost) has been recognized as an intrinsic problem in XTC. While the high-frequency peaks could, in principle, be pushed out of the audio range by decreasing Tc (which, as can be seen from Eqs. (4) to (6), is achieved by increasing / and/or decreasing the loudspeaker span, Θ , as is done in the so-called ''Stereo Dipole" configuration, where Θ maybe 10° ), the "low frequency boost" ofthe P-XTC filter would remain problematic.
[0041] The severe spectral coloration associated with these high-amplitude peaks presents three practical problems: 1) it would be heard by a listener outside the sweet spot, 2) it would cause a relative increase (compared to unprocessed sound playback) in the physical strain on the playback transducers, and 3) it would correspond to a loss in the dynamic range.
[0042] These penalties might be a justifiable price if infinitely good XTC performance
( X = c ) and perfectly flat frequency response (E m(of) = constant ) that the perfect XTC filter promises were guaranteed at the ears of a listener in the sweet spot. However, in practice, these theoretically promised benefits are unachievable due to the solution's sensitivity to unavoidable errors. This problem can best be appreciated by evaluating the condition number of the transfer matrix C.
[0043] It is well known that in matrix inversion problems the sensitivity of the solution to errors in the system is given by the condition number of the matrix. The condition number K(C) of the matrix C is given by
K ( C ) = | | | | I I C- ' H = | | C [ | \ \H [P] \ \ .
(It is also, equivalently, the ratio of largest to smallest singular values of the matrix.) Therefore, we have
Using the first and second derivatives of this function, as was done for the previous spectra, the following are the maxima and minima:
1 + g
K { C ) = -Λ at ωτ, = ηπ, with n = 0,1,2,3,4, ...
- " (20) κ· ( C ) = 1 at ωτβ = n— , with i = 1,3,5,7, ...
2 [Z 1 )
First, it is noted that the peaks and minima in the condition number occur at the same frequencies as those of the amplitude envelope spectrum at the loudspeakers, $m . Second, it is noted that the minima have a condition number of unity (the lowest possible value), which implies that the XTC filter resulting from the inversion of C is most robust (i.e., least sensitive to errors in the transfer
IT 3 IT 5tr
matrix) at the non-dimensional frequencies £,J c ~ ?, ' 2 ' 2 ' "" . Conversely, the condition number can reach very high values (e.g., κ C = 132.3 for typical case of β = -985 ) at the non- dimensional frequencies w-rc = Ο,π, 2π, Ίπ ... . As 8 1 the matrix inversion resulting in the P- XTC filter becomes ill-conditioned, or in other words, infinitely sensitive to errors. The sliglitest misalignment, for instance, of the listener's head, would thus result in a severe loss in XTC control at the ears (at and near these frequencies) which, in turn, causes the severe spectral coloration in Sl?i(fe,') to be transmitted to the ears.
Deficiencies of Constant-Parameter Regularization
[0044] Regularization methods allow controlling the norm of the approximate solution of an ill-conditioned linear system at the price of some loss in the accuracy of the solution. The control of the norm through regularization can be done subject to an optimization prescription, such as the minimization of a cost function. Regularization may be discussed analytically in the context of XTC filter optimization, which may be defined as the maximization of XTC performance for a desired tolerable level of spectral coloration or, equivalently, the minimization of spectral coloration for a desired minimum XTC performance.
[0045] A pseudoinverse representing a nearby solution to the matrix inversion problem is sought:
where the superscript " denotes the Hermitian operator, and β is the regularization parameter which essentially causes a departure from H(P ), the exact inverse of C . β is taken to be a constant, 0 < β « 1 . The pseudoinverse matrix, Ht ?1 , is the regularized filter, and the superscript [ β] is used to denote constant-parameter regularization. The regularization stated in Eq. (22) corresponds to a minimization of a cost function, J {id) ,
J{ioS) = e {ϊω)β{ίω) + βν {ίώ)ν{ίω) (23)
where the vector e represents a perfon-nance metric that is a measure of the departure from the signal reproduced by the perfect filter. Physically, then, the first tenia in the sum constituting the cost function represents a measure of the performance error, and the second term represents an "effort penalty," which is a measure of the power exerted by the loudspeakers. For β > 0 , Eq. (22) leads to an optimum, which corresponds to the least-square minimization of the cost function J{i o) . [0046] Therefore, an increase of the regularization parameter β leads to a minimization of the effort penalty at the expense of a larger performance error and thus to an abatement of the peaks in the norm of H, i.e., the coloration peaks in the 5 (ω) spectra, at the price of a decrease in XTC performance at and near the frequencies where the system is ill-conditioned.
[0047] Using the explicit form for C given by Eq. (12), the frequency response of the constant parameter regularization XTC filter becomes:
where
'
9e gi g2 + ;i ) ei3u
g2 ei- Te + g2— [ (^2 + ) 4. 2 3 + l ] '
126 ;
The eight metric spectra we defined herein become:
+ 3g2— 2g2 cosier, ') + /? + 1
"SI,, —2g2 cos(2u rc) + i( + β) + 2 + 1 2 [o2 + 2 COS(n:Tr ) + £5 + 1 1 '
- 2( + l )g2 c
(u?) V /g os(2u?re) + [β + I)2
S1!i -2g2 cost'^r-) + ( o2 + i?)2 + 23 + 1
It is worth noting that as β→ 0 , Ht/J)— > H ; /'' and the spectra of the perfect XTC filter are recovered from the expressions above as expected.
[0048] The envelope spectrum, 5^(ω3, is plotted in Fig. 3 for three values of β . Two features can be noted in that plot: 1 ) increasing the regularization parameter attenuates the peaks in the spectrum without affecting the minima, and 2) with increasing β the spectral maxima split into doublet peaks (two closely-spaced peaks).
[0049] To get a measure of peak attenuation and the conditions for the formation of doublet peaks, the first and second derivatives ωτ,. are used to find the conditions for which the first derivative is nil and the second is negative. These conditions are summarized as follows: If β is below a threshold β" defined as
β < β'≡(£ - ϊ)*. (29)
the peaks are singlets and occur at the same non-dimensional frequencies as for the envelope spectrum peaks of the P-XTC filter (J l j ), and have the following amplitude:
at 0JT,- — η , with n = 0/1,2,3, , ...
[0050] If the condition
β"≤β = ι (30)
is satisfied, the maxima are doublet peaks located at the following non-dimensional frequencies: 6JTC = ηπ ± cos 1 1 with n = 0,1,2,3,4, ...
2ff / (31) and have an amplitude
2 (32)
which does not depend on β , (The superscripts and T* denote singlet and doublet peaks, respectively.) The attenuation of peaks in the spectrum due to regularization can be obtained by dividing the amplitude of the peaks in the P-XTC (i.e., ,5 = 0 ) spectrum by that of peaks in the regularized spectrum. For the case of singlet peaks, the attenuation is
and for doublet peaks, it is given by
10051J For the typical case of $ = ·9δ5 illustrated in Fig. 2 , we have β' = 2.225 x icr* , and for β = ·005 and 0.05 we get doublet peaks that are attenuated (with respect to the peaks in the P-XTC spectrum) by 19.5 and 29.5 dB, respectively, as marked on that plot. Therefore, increasing the regularization parameter above this (typically low) threshold causes the maxima in the envelope spectrum to split into doublet peaks shifted by a frequency
Δ^ωΓρ) = cos 1
i *8 i to either side of the peaks in the response of the perfect XTC filter.
(For an illustrative case of ,9 - .985 , it is found that β" = 2.225 x 1CT« and Δ(ωτ, ); 0.225 for β = · 05 ). Due to the logarithmic nature of frequency perception for humans, these doublet peaks are perceived as narrow-band artifacts at high frequencies (i.e., for n = t 'i, ,.. )5 but the first doublet peak centered at r-— 0 is perceived as a wide-band low-frequency rolloff of typically many dB, as can be clearly seen in Fig. 3. Therefore, constant-/5 regularization transforms the bass boost of the perfect XTC filter into a bass roll-off.
[0052] Since regularization is essentially a deliberate introduction of error into system inversion, it is expected that both the XTC spectrum and the frequency responses at the ears will suffer (i.e., depart from their ideal P-XTC filter levels of ∞ and 0 dB, respectively) with increasing β . The effects of constant-parameter regularization on responses at the ears are illustrated in Fig. 4 which shows the effects of regularization on the crosstalk cancellation spectrum, ^β](ω) (top two curves), and the ipsilateral frequency response at the ear for a side image, Esj/i (co) . The black horizontal bars on the top axis mark the frequency ranges for which an
XTC level of 20~dB or higher is reached with β=.05, and the grey bars represent the same for the case of fi =.005. (Other parameters are the same as for Fig. 2).
[0053] The black curves in that plot represent the crosstalk cancellation spectra and show that XTC control is lost within frequency bands centered around the frequencies where the system is ill-conditioned ( a3 Tc = with = 0,1,2,3Α ... ) and whose frequency extent widens with increasing regularization. For example, increasing β to .05 limits XTC of 20 dB or higher to the frequency ranges marked by black horizontal bars on the top axis of that figure, with the first range extending only from 1.1 to 6.3 kHz and the second and third ranges located above 8.4 kHz. In many practical applications, such high (20 dB) XTC levels may not be needed or achievable (e.g., because of room reflections and/or mismatch between the HRTF of the listener and that used (e.g. dummy head) to design the filter, and the higher values of β needed to tame the spectral coloration peaks below a required level at the loudspeakers may be tolerated.
• F :3 (
[0054] The " si„ "3' 1 responses at the ears, shown as the bottom curves in Fig. 4, depart only by a few dB from the corresponding P-XTC (i.e.. '5 = 0 ) filter response (which is a flat curve at 0 dB). More precisely and generally, the maxima and minima of the » spectrum are given by:
£ ; J = — : at ^Tc = i— . with n = 1. -i, a. , . .
at. ? Tc = η π. with n - 0. 1. 2. Ά.4
For the typical (.# = ·985 ) example shown in the figure, for
8 - .05. Ε[β] ί = - .2 dB and E[ ] 1 = -6.1 dB , . , , . ,
si II M il , showing that even relatively aggressive regularization results in a spectral coloration at the ears that is quite modest compared to the spectral coloration the perfect XTC filter imposes at the loudspeakers. [0055] In sum, while constant-parameter regularization, a commonly used technique in the design of XTC filters, is effective at reducing the amplitude of peaks (including the 'Tow- frequency boost") in the envelope spectrum at the loudspeakers, it typically results in undesirable narrow-band artifacts at higher frequencies and a rolloff of the lower frequencies at the loudspeakers. This non-optimal behavior can be avoided if the regularization parameter is allowed to be a function of the frequency, as described herein.
Spectral Flattening through Frequency-Dependent Regularization
[0056] The method and system of the present invention rely on the use of a specific scheme for calculating the frequency-dependent regularization parameter (FDRP) that would result in the flattening of the amplitude vs frequency spectrum measured at the loudspeakers and not at the ears of the listeners as is implicit in previous XTC filter designs that are based on the inversion of the system transfer matrix.
[0057] Flattening of the amplitude vs frequency spectrum measured at the loudspeakers, as opposed to at the ear of the listener, forces XTC to result from phase effects only, and not from amplitude effects, since the amplitude is flat with frequency at the loudspeakers. This means that any inherent spectral (i.e. amplitude vs frequency) coloration in the loudspeaker and/or playback hardware will not be corrected for (as is inherently done in previous inversion-based XTC filter design methods where the XTC filter aims to reproduce at the ears the same amplitude vs frequency response of the recorded the signal).
[0058] Flattening of the amplitude vs frequency spectrum measured at the loudspeakers, results in the listener hearing the same amplitude vs frequency response that would be heard without processing the sound through the XTC filter. This implies that the listener would not hear any spectral coloration beyond that due to the playback hardware and loudspeakers without the filter. Equally important is the fact that such a flat filter response at the loudspeakers also means no dynamic range loss in the processed audio.
[0059] hi order to explain method and system of the present invention, an idealized analytical description of how to calculate a frequency-dependent regularization parameter will be described that results in the specific goal of flattening the XTC filter response at the loudspeakers.
Description of the method of the present invention in the context of the idealized model
[0060] For the sake of clarity, the same optimization scheme described with respect to the minimization of the cost function expressed in Eq. (23)) will be used, keeping in mind that the method and system of the present invention are completely independent of the adopted optimization scheme
[0061] hi order to avoid the frequency-domain artifacts discussed above and illustrated in
Fig. 3, a frequency-dependent regularization parameter is calculated that would cause the envelope spectrum 5(a)) to be flat at a desired level Γ (in dB) over the frequency bands where the perfect filter's envelope spectrum exceeds Γ . Outside these bands (i.e., where the Si P] (o)) is below Γ ), we apply no regularization. This can be stated symbolically as:
= r if S{ p] (ω)≥γ (33)
where the P-XTC envelope spectrum, S[ P] (OJ) , is given by Eq. (16) , and
r = l (f'20 (35)
with Γ given in dB. Γ cannot exceed the magnitude of the peaks in the S[ ] (a>) spectrum, r is bounded by: γ < ~^- (36) where the bound is the maxima of the S[ p] spectra, SlP ] , given by Eq. ( 18).
[0062] The frequency-dependent regularization parameter needed to effect the spectral flattening required by Eq. (33) is obtained by setting Sm(o)) , given by Eq. (27), equal to γ and solving for β(ω) , which is now a function of frequency. Since the regularized spectral envelope,
~ffii I J-f ίβ] I
?ΐζω), (which is also 1 1 , the 2-norm of the regularized XTC filter) is the maximum of two functions, two solutions for are obtained:
- '
β , ( ω) = - g Λ + 25cos{ ωτ, ) +
The first solution, β E (ω3 ; applies for frequency bands where the out-of-phase response of the perfect filter (i.e. , the second singular value, which is the second argument of the max u function in Eq. (16) ) dominates over the in-phase response (i.e., the first argument of that function):
1 - 2^C OS(WTC) + 1
[0063] Similarly, regularization with β n ίω3 applies for frequency bands where ^f3 - .
Therefore, we must distinguish between three branches of the optimized solution; two regularized branches corresponding to β ~ β i and β = βη (w) , and one non-regularized (perfect- filter) branch corresponding to β = 0 . We call these Branch I, II and P, respectively, and sum up the conditions associated with each as follows:
Branch I; applies where Sm( ) > γ and Sf > S }., and requires setting (ω)—γ, β = β / (ω);
Bran ch 11: applies where Sm((a)≥ γ and Sf*≥ SJP* and requires setting 5(ω) - γ, β = βπ (ω){
Branch Ρ: applies where Sm{oj) < γ,
and requires setting 5(&») = 5^(ω), β— 0.
[0064] Following this three-branch division, the envelope spectrum at the loudspeakers,
$■>->) , for the case of frequency-dependent regularization is plotted as the thick black curve in Fig. 5 for Γ-7 dB. This value was chosen because it corresponds to the magnitude of the (doublet)
peaks in the β = ·05 spectrum (i.e., \ v ), which is also plotted (light solid curve) as a reference for the corresponding case of constant-parameter regularization. (We call a spectrum obtained with frequency-dependent regularization and one obtained with constant-^ regularization "corresponding spectra," if the peaks in -?^(ω) 5 whether singlets or doublets, are equal to Y .)
[0065] It is seen from that figure that the low-frequency boost and the high-frequency peaks of the perfect XTC spectrum, which would be transformed into a low-frequency roll-off and narrow-band artifacts, respectively, by constant-^ regularization, are now flat at the desired maximum coloration level, Γ. The rest of the spectrum, i.e., the frequency bands with amplitude below Γ, is allowed to benefit from the infinite XTC level of the perfect XTC filter and the robustness associated with relatively low condition numbers. [0066] In the method of the present invention Y is specifically chosen to be at or below the value equal to the lowest value of the 5¾¾ω) spectrum, i.e.
as this would insure that the entire spectrum 5 (ω) js flat (j.e- the inequality in (34) does not hold and Branch P disappears) and XTC would be forced to be effected through phase effects only, resulting in no amplitude coloration due to XTC filtering and no dynamic range loss, all while insuring the minimization of whatever cost function is prescribed by the adopted optimization scheme (in this particular example, Eq. (23)).
Generalized Method
[0067] The above leads us to a general description of the method of the present invention in terms of specific steps that are taken in the XTC filter design procedure (the steps are also shown schematically in Fig. 6 along with the associated input and output for each step):
[0068] In step 30, the system's transfer matrix in the frequency domain (i.e. matrix C as in
Eq. (12) and the input 28) is inverted, either analytically (if it results from a tractable idealized model) or numerically (if it results from experimental measurements), using zero or a very small constant regularization parameter (large enough to avoid machine inversion problems) to obtain the corresponding perfect XTC filter, H1''1.
[0069] In step 34 Γ is set equal to F*,be the lowest value (in dB) reached by the amplitude vs frequency response at the loudspeakers, S[P]i in Step 34. This is found from either Eq. (19) (or a similar equation resulting from another tractable analytical model) or from plotting the H '" spectra (if the inversion was done numerically using actual measurements as in the example given further below) then calculate γ from γ* = 10r* 20 (36).
[0070] In Step 38, the frequency-dependent regularization parameter (FD P) β(ω) that would result in a flat frequency response at the loudspeakers is calculated, so that S[Pi(co) = constant < γ* (as, for instance, is done by using Equations (37) and (38)) thus forcing XTC to be caused by phase effects only.
[0071] In Step 40, the FDRP thus obtained, β(ω) , is used to calculate the pseudo-inverse of the system's transfer matrix (e.g. according to Eqn. (22)), which yields the sought regularized optimal XTC filter Ht ?1 that has a flat frequency response at the loudspeakers. (Finally, if needed for applying the resulting filter through a time-base convolution, as is often done in practical XTC implementation), a time domain version (impulse response) of the filter is obtained in step 44 by simply taking the inverse Fourier transform of H[ ?1 (output 42) .)
[0072] It should be noted that in Step 38, if the FDRP is calculated so that Si m(co) = constant < /*, the spectral flattening occurs for a side image (i.e. a sound panned to either the left or right channel and thus would be perceived by a listener to be located at or near his or left or right ear when the XTC level is sufficiently high ). However, the same method can be used to flatten the response at the loudspeakers for an image that is not a pure side image by simply requiring that S[m(oj) = constant < γ*, where S[(i](cQ) is the XTC filter's frequency response for an image of source panned anywhere between the left and right channels. For instance, to flatten for a central image, we set S{p] ci{w), (given, for instance, by the equation preceding Eqn. 27) to a constant < γ*, and proceed with the steps of the method as outlined above. In this context it is relevant to mention that for some applications, for instance pop music recording where the lead vocal audio is panned dead center, it might be desirable to flatten the response for a center image, i.e. Sc!(a>) , (or an image of any other desired panning) in order to avoid coloration of that image. It should also be noted in that context that since Sl/i](a))≥ S[P]( ) only flattening the side image (i.e. setting = constant < γ*) would result in no dynamic range loss due to the XTC filter. In other words, flattening for anything but the side image would incur a dynamic range loss that must be balanced by the benefit of a reduced spectral coloration for the desired panned image. For instance, for binaural recordings of real acoustic soundfields, which typically contain no dead- center panned images, flattening of the side image is advisable as this leads to no dynamic range loss.
Example using a Measured Transfer Function.
[0073] An example based on the transfer function of two loudspeakers in a room measured by microphones placed at the ear canal entrances of a dummy head (Neumann KU- 100) will now be described. The loudspeakers had a span of 60 degrees at the listening position, which was about 2.5 meters from each loudspeaker.
[0074] Fig. 7 shows the four (windowed) measured impulse responses (IR) representing the transfer function in the time domain. The x-axis of each plot in Fig. 7 is time in ms, and the_y- axis is the normalized amplitude of the measured signal. The top left plot shows the IR of the left loudspeaker measured at the left ear of the dummy head, and the bottom left plot shows the IR of the left loudspeaker measured at the right ear of the dummy head. The top right plot is the ER. of the right speaker -left ear transfer function and the bottom plot is the IR of the right speaker - right ear transfer function.
[0075] Fig 8, shows relevant spectra where the x-axis is frequency in Hz and the _y-axis is amplitude in dB. The curve 48 in that plot is the frequency response CLL that corresponds to the left speaker-left ear transfer function in the frequency domain obtained by panning the test sound completely to the left channel. The ripples in curve 48 above 5 kHz are due to the HRTF of the head and the left ear pinna. The other curves 50, 52 54 in that plot are the measured frequency responses associated with the perfect XTC filter, that is an XTC filter obtained by inverting the transfer function with essentially no regularization (β=10"3), In particular. Curve 50 is the response at the left loudspeaker and shows a dynamic range loss of 31.45 dB (difference between the maximum and minimum in that curve). Curve 52 is the frequency response at the left (ipsilateral) ear, ESJ , which, as expected from a perfect XTC filter, is essentially flat over the entire audio band. The curve 54 is the corresponding frequency response measured at the right (contralateral) ear, ES(. , and shows significant attenuation with respect to curve 52 due to XTC.
The difference in amplitude between the curves 52 and 54 linearly averaged over frequencies is the average XTC level, which for this case is 21.3 dB.
[0076] We contrast these curves with those curves in Fig. 9 which shows the responses due to a filter designed in accordance with the present invention. By design, curve 60, representing
, S[l3](co) , the response at the left loudspeaker, is completely flat over the entire audio spectrum.
Consequently, the frequency response at the left ear , curve 62, matches very well the corresponding measured system transfer function, C , shown in curve 64,. Since Sm(oj) is flat, there is no dynamic range loss associated with this filter. The average XTC level for this filter
(obtained by taking the linear average of the difference between curve 62 and 66) is 19.54 dB, which is only 1.76 dB lower than the XTC level obtained with the perfect filter, testifying to the optimal nature of the regularized filter. [In sum, the filter designed with the method of the present invention, imposes no audible coloration to the sound of the playback system, has no dynamic range loss, and yields an XTC level that is essentially the same as that of a perfect XTC filter.
[0077] The method described herein may be implemented in software, or firmware incorporated in a computer-readable storage medium for execution by a general purpose computer or a processor, such as a DSP chipset. Examples of suitable computer-readable storage mediums include a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs).
[0078] Embodiments of the present invention may be represented as instructions and data stored in a computer-readable storage medium. For example, aspects of the present invention may be implemented using Verilog, which is a hardware description language (HDL). When processed, Verilog data instructions may generate other intermediary data, (e.g., netlists, GDS data, or the like), that may be used to perform a manufacturing process implemented in a semiconductor fabrication facility. The manufacturing process may be adapted to manufacture semiconductor devices (e.g., processors) that embody various aspects of the present invention.
[0079] Suitable processors include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, a graphics processing unit (GPU), a DSP core, a controller, a microcontroller, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) , any other type of integrated circuit (IC), and/or a state machine, or combinations thereof.
[0080] While the foregoing invention has been described with reference to its preferred embodiments, various alterations and modifications will occur to those skilled in the art. All such alterations and modifications are intended to fall within the scope of the appended claims.

Claims

1. A method for filtering audio signals to cancel crosstalk in an audio system comprising the steps of
inverting a transfer matrix or function of the audio system;
using information from the inverted transfer matrix or function to calculate a frequency-dependent regularization parameter that when applied to audio signals produces a flat frequency response at any of the loudspeakers of the audio system over an audio band or a portion thereof;
using said calculated frequency-dependent regularization parameter to calculate the pseudo inverse of said transfer matrix.
2. The method for filtering audio signals to cancel crosstalk of claim 1 wherein said flat frequency response is effected only though phase effects over said audio band or portion thereof.
3. The method for filtering audio signals to cancel crosstalk of claim 1 , wherein said frequency-dependent regularization parameter when applied to audio signals produces a flat frequency response at one or more of the loudspeakers for a desired image panned anywhere between left and right channels.
4. The method for filtering audio signals to cancel crosstalk of claim 1 wherein said audio system is a binaural audio system.
5. The method for filtering audio signals to cancel crosstalk of claim 1 wherein said audio system is a stereo audio system.
6. A method for designing crosstalk cancellation filters for i audio applications comprising the steps of
inverting a transfer matrix or function of an audio system;
using information from the inverted transfer matrix or function to calculate a frequency-dependent regularization parameter that when applied to audio signals produces a flat frequency response at any of the loudspeakers of the audio system over an audio band or a portion thereof;
using said calculated frequency-dependent regularization parameter to calculate the pseudo inverse of said transfer matrix.
7. The method for designing crosstalk cancellation filters for audio applications of claim 6 wherein frequency-dependent regularization causes crosstalk cancellation to be effected only though phase effects over said audio band or portion thereof.
8. The method for designing crosstalk cancellation filters for audio applications of claim 6, wherein said step of calculating said frequency-dependent regularization parameter lead to a filter that when applied to audio signals produces a flat frequency response at one of the loudspeakers for a desired image panned anywhere between left and right channels.
9. The method for filtering audio signals to cancel crosstalk of claim 6 wherein said audio system is a binaural audio system
10. The method for filtering audio signals to cancel crosstalk of claim 6 wherein said audio system is a stereo audio system
1 1 . A system for filtering audio signals to cancel crosstalk in an audio system comprising:
an audio input stage;
a processor for
inverting a transfer matrix of the audio system
calculating a frequency-dependent regularization parameter that when applied to audio signals produces a flat frequency response at any of the loudspeakers of the audio system over an audio band or a portion thereof;
calculating the pseudo inverse of said transfer matrix using said calculated frequency-dependent regularization parameter.
12. The system for filtering audio signals to cancel crosstalk in an audio system of claim 1 1 wherein said flat frequency response is effected by said processor only though phase effects over said audio band or portion thereof.
13. The system for filtering audio signals to cancel crosstalk in an audio system of claim 1 1 wherein said processor has the capability of applying said frequency-dependent regulanzation parameter to filter audio signals to produce a flat frequency response at one or more of the loudspeakers for a desired image panned anywhere between left and right channels.
14. A system for producing crosstalk cancellation filters for audio applications that involves
an audio input stage;
a processor for
inverting a transfer matrix of the audio system;
calculating a frequency-dependent regulanzation parameter that leads to a filter that when applied to audio signals produces a flat frequency response at any of the loud speakers of an audio system over an audio band or a portion thereof; and
calculating the pseudo inverse of said transfer matrix using said calculated frequency-dependent regulanzation parameter.
15. The system for producing crosstalk cancellation filters for audio applications of claim 14 wherein frequency-dependent regularization is used so that crosstalk cancellation is effected only though phase effects over said audio band or portion thereof.
16. The system for filtering audio signals to cancel crosstalk in an audio system of claim 14 wherein said processor has the capability of applying said fi-equency-dependent regularization parameter to produce a filter that when applied to the audio signals produces a flat frequency response at one or more of the loudspeakers for a desired image panned anywhere between left and right channels.
EP11825672.6A 2010-09-03 2011-09-01 Spectrally uncolored optimal crosstalk cancellation for audio through loudspeakers Active EP2612437B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US37989110P 2010-09-03 2010-09-03
PCT/US2011/050181 WO2012036912A1 (en) 2010-09-03 2011-09-01 Spectrally uncolored optimal croostalk cancellation for audio through loudspeakers

Publications (3)

Publication Number Publication Date
EP2612437A1 true EP2612437A1 (en) 2013-07-10
EP2612437A4 EP2612437A4 (en) 2014-04-02
EP2612437B1 EP2612437B1 (en) 2015-11-18

Family

ID=54290154

Family Applications (1)

Application Number Title Priority Date Filing Date
EP11825672.6A Active EP2612437B1 (en) 2010-09-03 2011-09-01 Spectrally uncolored optimal crosstalk cancellation for audio through loudspeakers

Country Status (1)

Country Link
EP (1) EP2612437B1 (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2556663A (en) * 2016-10-05 2018-06-06 Cirrus Logic Int Semiconductor Ltd Method and apparatus for acoustic crosstalk cancellation
US10595150B2 (en) 2016-03-07 2020-03-17 Cirrus Logic, Inc. Method and apparatus for acoustic crosstalk cancellation
CN112235691A (en) * 2020-10-14 2021-01-15 南京南大电子智慧型服务机器人研究院有限公司 Hybrid small-space sound reproduction quality improving method

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6668061B1 (en) * 1998-11-18 2003-12-23 Jonathan S. Abel Crosstalk canceler
US20100202629A1 (en) * 2007-07-05 2010-08-12 Adaptive Audio Limited Sound reproduction systems

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6668061B1 (en) * 1998-11-18 2003-12-23 Jonathan S. Abel Crosstalk canceler
US20100202629A1 (en) * 2007-07-05 2010-08-12 Adaptive Audio Limited Sound reproduction systems

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
BAI MINGSIAN R ET AL: "Optimal design of loudspeaker arrays for robust cross-talk cancellation using the Taguchi method and the genetic algorithm", THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA, AMERICAN INSTITUTE OF PHYSICS FOR THE ACOUSTICAL SOCIETY OF AMERICA, NEW YORK, NY, US, vol. 117, no. 5, 1 May 2005 (2005-05-01), pages 2802-2813, XP012072930, ISSN: 0001-4966, DOI: 10.1121/1.1880852 *
Edgar Y Choueiri: "Optimal Crosstalk Cancellation for Binaural Audio with Two Loudspeakers", , 13 November 2010 (2010-11-13), XP055103354, Princeton University Retrieved from the Internet: URL:http://www.princeton.edu/3D3A/Publications/BACCHPaperV4d.pdf [retrieved on 2014-02-20] *
See also references of WO2012036912A1 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10595150B2 (en) 2016-03-07 2020-03-17 Cirrus Logic, Inc. Method and apparatus for acoustic crosstalk cancellation
US11115775B2 (en) 2016-03-07 2021-09-07 Cirrus Logic, Inc. Method and apparatus for acoustic crosstalk cancellation
GB2556663A (en) * 2016-10-05 2018-06-06 Cirrus Logic Int Semiconductor Ltd Method and apparatus for acoustic crosstalk cancellation
US10111001B2 (en) 2016-10-05 2018-10-23 Cirrus Logic, Inc. Method and apparatus for acoustic crosstalk cancellation
CN112235691A (en) * 2020-10-14 2021-01-15 南京南大电子智慧型服务机器人研究院有限公司 Hybrid small-space sound reproduction quality improving method

Also Published As

Publication number Publication date
EP2612437B1 (en) 2015-11-18
EP2612437A4 (en) 2014-04-02

Similar Documents

Publication Publication Date Title
WO2012036912A1 (en) Spectrally uncolored optimal croostalk cancellation for audio through loudspeakers
US10104485B2 (en) Headphone response measurement and equalization
US8965546B2 (en) Systems, methods, and apparatus for enhanced acoustic imaging
US9577595B2 (en) Sound processing apparatus, sound processing method, and program
CN110024419B (en) Gain Phase Equalization (GPEQ) filter and tuning method
US20140334650A1 (en) Methods and devices for reproducing surround audio signals
CN110035376A (en) Come the acoustic signal processing method and device of ears rendering using phase response feature
CN110557710B (en) Low complexity multi-channel intelligent loudspeaker with voice control
TWI646530B (en) Method for generating first sound and second sound, audio processing system, and non-transitory computer readable medium
WO2004103025A1 (en) Loudspeaker system for virtual sound synthesis
CN107039029B (en) Sound reproduction with active noise control in a helmet
US8873762B2 (en) System and method for efficient sound production using directional enhancement
TW202027517A (en) Spectral defect compensation for crosstalk processing of spatial audio signals
EP2612437A1 (en) Spectrally uncolored optimal croostalk cancellation for audio through loudspeakers
Choueiri Binaural audio through loudspeakers
WO2023010691A1 (en) Earphone virtual space sound playback method and apparatus, storage medium, and earphones
JP2011259299A (en) Head-related transfer function generation device, head-related transfer function generation method, and audio signal processing device
Cecchi et al. Crossover Networks: A Review
EP4231668A1 (en) Apparatus and method for head-related transfer function compression
Schultz et al. An extraaural headphone for optimized binaural reproduction (Ein extraauraler Kopfhörer für die optimierte Wie-dergabe binauraler Signale)
KR20240145040A (en) Device and method for head transfer function compression
WO2023156274A1 (en) Apparatus and method for reducing spectral distortion in a system for reproducing virtual acoustics via loudspeakers
CN116389972A (en) Audio signal processing method, system, chip and electronic equipment

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20130329

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602011021507

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: H03B0029000000

Ipc: H04S0001000000

A4 Supplementary search report drawn up and despatched

Effective date: 20140304

RIC1 Information provided on ipc code assigned before grant

Ipc: H04S 1/00 20060101AFI20140226BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20150508

RIN1 Information on inventor provided before grant (corrected)

Inventor name: CHOUEIRI, EDGAR, Y.

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 762069

Country of ref document: AT

Kind code of ref document: T

Effective date: 20151215

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602011021507

Country of ref document: DE

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: JACOBACCI AND PARTNERS S.P.A., CH

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 762069

Country of ref document: AT

Kind code of ref document: T

Effective date: 20151118

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160218

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160318

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160318

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160219

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602011021507

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

26N No opposition filed

Effective date: 20160819

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160901

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160901

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20110901

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160930

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20151118

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230524

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20230810

Year of fee payment: 13

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20231001

Year of fee payment: 13

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20240705

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240702

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20240701

Year of fee payment: 14

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240702

Year of fee payment: 14