EP2189010B1 - Eine vorrichtung und ein verfahren zur ermittlung eines komponentensignals in hoher genauigkeit - Google Patents

Eine vorrichtung und ein verfahren zur ermittlung eines komponentensignals in hoher genauigkeit Download PDF

Info

Publication number
EP2189010B1
EP2189010B1 EP08801826.2A EP08801826A EP2189010B1 EP 2189010 B1 EP2189010 B1 EP 2189010B1 EP 08801826 A EP08801826 A EP 08801826A EP 2189010 B1 EP2189010 B1 EP 2189010B1
Authority
EP
European Patent Office
Prior art keywords
loudspeakers
delay
audio signal
wave field
interpolation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP08801826.2A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP2189010A1 (de
Inventor
Andreas Franck
Sandra Brix
Thomas Sporer
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Original Assignee
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Publication of EP2189010A1 publication Critical patent/EP2189010A1/de
Application granted granted Critical
Publication of EP2189010B1 publication Critical patent/EP2189010B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00—Stereophonic arrangements
    • H04R5/04—Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/008—Systems employing more than two channels, e.g. quadraphonic in which the audio signals are in digital form, i.e. employing more than two discrete digital channels
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/13—Application of wave-field synthesis in stereophonic audio systems

Definitions

  • the present invention relates to an apparatus and method for detecting high accuracy component signals for a WFS (Wave Field Synthesis) system, and more particularly to an efficient algorithm for delay interpolation for wave field synthesis display systems.
  • WFS Wide Field Synthesis
  • Wave field synthesis is an audio reproduction method developed at TU Delft for the spatial reproduction of complex audio scenes.
  • the spatially correct rendering is not limited to a small area, but extends over a wide viewing area.
  • WFS is based on a well-founded mathematical-physical basis, namely the principle of Huygens and the Kirchhoff-Helmholtz integral.
  • a WFS reproduction system consists of a large number of loudspeakers (so-called secondary sources).
  • the loudspeaker signals are formed from delayed and scaled input signals. Since many audio objects (primary sources) are typically used in a WFS scene, many such operations are required to generate the loudspeaker signals. This requires the high computing power required for wave field synthesis.
  • WFS also offers the possibility of realistically mapping moving sources. This feature is used in many WFS systems and is, for example, for the use in the cinema, virtual reality applications or live performances of great importance.
  • a primary goal is the development of signal processing algorithms for the playback of moving sources using WFS.
  • the real-time capability of the algorithms is an important condition.
  • the most important criterion for evaluating the algorithms is the objective perceived audio quality.
  • WFS is a very expensive audio reproduction process in terms of processing resources. This is mainly due to the large number of speakers in a WFS setup and the often high number of virtual sources used in WFS scenes. For this reason, the efficiency of the algorithms to be developed is of paramount importance.
  • WFS scaling is performed on any combination of virtual source and speaker, as well as on each audio sample, it forms the bulk of the resource requirements of a WFS system, even with very little complexity of the single operation.
  • the natural Doppler effect ie the frequency shift of a moving source, is not considered an artifact here, as it is a property of the primary sound field that is to be reproduced by a WFS system. Nevertheless, this is undesirable in various applications.
  • delay interpolation The operation of obtaining the value of a time-discrete sampled signal at arbitrary times is referred to as delay interpolation or fractional-delay interpolation.
  • fractional delay algorithms are implemented as discrete filters which have as input a discrete-time signal and as an output an approximation of the delayed signal.
  • any delays can be generated with a fractional delay filter.
  • the delay d int is implemented by an index shift in the input signal.
  • Adherence to the ideal operating range requires a minimal value of the delay, which must not be undercut to maintain causality.
  • methods for delay interpolation, especially high-quality FD algorithms with large filter lengths also increase the system latency. However, this does not exceed an order of magnitude of 20 to 50 samples, even in the case of extremely complex procedures. However, this is generally low compared to other systemic latencies of a typical WFS playback system.
  • Wave field synthesis requires delay interpolation for each combination of virtual source and loudspeaker. Combined with the complexity of delay interpolation necessary for good rendering quality, good quality real-time implementation is not practical.
  • WFS renderers implement several simple methods for delay interpolation.
  • the used class hierarchy allows easy exchange of procedures become.
  • the temporal interpolation of the WFS parameters Delay (and Scale) also influences the quality of the sample rate conversion. In the traditional renderer structure, these parameters are updated only in a fixed grid (currently in time with 32 audio samples).
  • the U.S. Patent No. 6,600,495 B1 discloses a filter structure that allows the use of a filter as either a continuous-delay Farrow filter or as a selectable-delay polyphase filter.
  • the AES Convention Paper by Bleda a. a. May 28-31, Barcelona, Spain, "Design and Implementation of a Compatible Wave Field Synthesis Authoring Tool” discloses a user-friendly editing tool having a data acquisition functionality, a virtual source processor, an auralization processor for accommodating room acoustics, a Functionality for estimating auralization filters, and a final mixing stage.
  • the present invention has the object to provide an apparatus and a method which determines component signals for a wave field synthesis system with significantly higher accuracy. This object is achieved by a device according to claim 1 or 10, a method according to claim 15 or 16, or a computer program according to claim 18.
  • the gist of the present invention is that a higher quality component signal can be achieved by first pre-processing the audio signal belonging to a virtual source, the preprocessing being independent of the WFS parameter, such that a better interpolation is achieved.
  • the component signal thus has a higher accuracy, wherein the component signal represents the component generated by a virtual source for a loudspeaker signal.
  • the present invention includes improved interpolation of WFS parameters such as delay or delay Scaling values obtained with a low parameter sampling frequency.
  • embodiments of the present invention provide an apparatus for determining a component signal for a WFS system having an array of loudspeakers, the WFS system configured to provide an audio signal associated with a virtual source represented as a discrete signal sampled at an audio sampling frequency. and source positions associated with the virtual source to calculate component signals for the loudspeakers due to the virtual source, taking speaker positions into account.
  • the apparatus of the invention comprises means for providing WFS parameters for a component signal using a source position and using the speaker position, the parameters being determined at a parameter sampling frequency that is less than the audio sampling frequency.
  • the apparatus further comprises a WFS parameter interpolator for interpolating the WFS parameters to produce an interpolated WFS parameter having a parameter interpolation frequency greater than a parameter sampling frequency, the interpolated WFS parameters having interpolated fractions that have a higher accuracy than specified by the audio sampling frequency.
  • the device comprises an audio signal processing device which is designed to apply the interpolated fractional parts to the audio signal in such a way that the component signal is processed with the higher accuracy.
  • the delay interpolation algorithm is partitioned so that it has a part for calculating intermediate values in a) and b) a efficient algorithm for calculating the final results is divided.
  • the Farrow structure is a variable digital filter for continuously variable variable delays. It consists of a set of P sub-filters. The input signal is filtered by each of these sub-filter and provides P different outputs c p output signal is obtained by evaluating a polynomial in d, where d is the fractional portion or fraction of the desired delay the outputs of the subfilters, c p, the coefficients of the polynomial form.
  • the proposed algorithm preprocesses the outputs of the sub-filters for each sample of the input signal. These P values are written to the delay line. The output signals are generated by accessing the P values in the delay line and the evaluation of the polynomial. This efficient operation is done for each speaker.
  • the audio signal processing device is designed to carry out the methods (i) and / or (ii).
  • the audio signal processing means is adapted to perform an oversampling of the audio signal such that the oversampling is performed up to an oversampling rate which ensures a desired accuracy.
  • Embodiments of the present invention describe a WFS delay interpolation that is particularly advantageous for audio engineering and sound engineering in the context of wave field synthesis, since a significantly improved suppression of audible artifacts is achieved.
  • the improvement is achieved in particular by an improved delay interpolation, the use of fractional delays and asynchronous sampling rate conversion. It is thus
  • the wave field synthesis system has a speaker array 700 placed with respect to a demonstration area 702. Specifically, this includes in Fig. 11 shown speaker array which is a 360 ° array, four array sides 700a, 700b, 700c and 700d. If the demonstration area 702 z. As a movie theater, it is assumed that the cinema screen is on the same side of the screening area 702, on which the sub-array 700c is arranged with respect to the conventions front / back or right / left. In this case, the observer who is sitting at the so-called optimal point P in the demonstration area 702 would see to the front, ie to the screen.
  • Each loudspeaker array consists of a number of different individual loudspeakers 708, each of which is driven by its own loudspeaker signals transmitted by a wave field synthesis module 710 via an in-line loudspeaker signal Fig. 11 only schematically shown data bus 712 are provided.
  • the wave field synthesis module is configured to use the information about e.g. B.
  • the wave field synthesis module can also receive further inputs, such as information about the room acoustics of the demonstration area, etc.
  • Fig. 1 shows a device according to an embodiment of the present invention.
  • the virtual source source position 135 and the loudspeaker positions 145 are input to a device for providing WFS parameters 150.
  • the means for providing WFS parameters 150 may optionally include another input where other data 190 may be read.
  • the other data 190 may include, for example, the room acoustics and other scene data.
  • the means 150 for providing therefrom, with a parameter sampling frequency determines therefrom the WFS parameters 155 read in the WFS parameter interpolator 160. After interpolation, the interpolated WFS parameters are provided to the audio signal processor 170.
  • the audio signal processor 170 also has an input for an audio signal 125 and an output for component signals 115.
  • Each virtual source provides its own audio signal, which is processed into component signals for the various loudspeakers.
  • FIG. 12 shows a WFS system 200 having a WFS signal processing 210 and a WFS parameter calculation 220.
  • the WFS parameter calculation 220 has an input for scene data 225 relating, for example, to N source signals. Assuming that N signal sources (virtual sources) and M loudspeakers are available for the WFS system, the WFS parameter calculation 220 calculates NxM parameter values (scale and delay values). These parameters are output to the WFS signal processor 210.
  • the WFS signal processor 210 includes a WFS delay and scaling device 212, a summing device 214, and a delay line 216.
  • the delay line 216 is generally configured as a means for latching and may be given by, for example, a ring buffer.
  • the NxM parameters are read in by the WFS delay and scaling device 212.
  • the WFS delay and scaling device 212 also reads the audio signals from the delay line 216.
  • the audio signals in the delay line 216 in this case have an index that corresponds to a certain delay and is accessed by means of a pointer 217, so that the WFS delay and scaling device 212 by accessing an audio signal with a specific index, a Delay for the corresponding audio signal.
  • the index thus simultaneously serves as the address or addressing of the corresponding data in the delay line 216.
  • the delay line 216 receives audio input data from the N-source signals, which are stored in the delay line 216 according to their timing. By correspondingly accessing an index of the delay line 216, the WFS delay and scaling unit 212 can thus read out audio signals having a desired (calculated) delay value (index). Further
  • the WFS delay and scaling device 212 outputs corresponding component signals 115 to the means 214 for summing 214, and the means 214 for summing sums the component signals 115 of the respective N virtual sources to generate therefrom loudspeaker signals for the M-loudspeakers.
  • the speaker signals are provided at a sound output 240.
  • Embodiments thus relate to audio signal processing of a WFS rendering system 200.
  • This rendering system contains as input data the audio signals of the WFS sources (virtual sources), the index variable n counting the sources and N representing the number of sources. Typically, these data come from other system components such as audio players, possibly pre-filters, etc.
  • the block WFS parameter calculation 220 provides amplitude (scaling) and delay values (delay values) for each source / speaker combination (index variable: m , Number: M). This is usually done as a matrix, the corresponding values for the sources n and loudspeaker m are named delay (n, m) and scale (n.m) in the following.
  • the audio signals are first stored in the delay line 216 to allow later random access (i.e., with variable delay values).
  • Core component of the embodiments is the block "WFS delay and scaling" 212. This is sometimes referred to as WFS convolution or WFS convolution, but it is not a true convolution in terms of signal processing and therefore the term is usually avoided.
  • an output signal (component signal 115) is generated for each combination (n, m) of source and loudspeaker.
  • the method according to the invention or the device according to the invention is of little importance in practice.
  • the synthesized wave field deviates from a theoretically defined ideal case with a rounding of the delay values, these deviations are very small and are completely covered by other deviations that occur in practice, such as a spatial aliasing.
  • H be calculated for moving sources.
  • the algorithm is particularly interesting for moving sources, but errors do not only occur when samples are "swallowed” or used twice. Rather, approximation of sampled signals at arbitrary nodes always generates errors.
  • the methods for approximation between nodes are also referred to as fractional-delay interpolation.
  • the central point of the present invention is to enable the use of very high-quality delay interpolation methods by means of a corresponding structuring of the WFS signal processing, while at the same time keeping the calculation effort comparatively low.
  • the present invention it is not specific to respond to the movement of sources and to attempt to avoid errors by appropriately generated samples in this case.
  • the signal processing does not need information about source positions, but only delay and amplitude values (which are time-variant in the case of a moving source).
  • the errors described arise from the way in which these delay values are applied to the audio signals by the function unit WFS delay and scaling 212 (primarily: which method is used for delay interpolation).
  • WFS delay and scaling 212 primarily: which method is used for delay interpolation.
  • Playback can be done with the current WFS real-time rendering system, using various methods of delay interpolation.
  • the described algorithms for delay interpolation are used.
  • the source signals are simple, predominantly tonal signals, as they suggest an increased perceptibility of delay interpolation artifacts. Both signals below and above the system's spatial aliasing frequency are used to evaluate the visibility, both without aliasing influence and the influence of delay interpolation artefacts and aliasing disturbances on each other.
  • the perceived quality is evaluated informally and subjectively by some test persons.
  • the FD filters designed for a particular fractional delay can be examined using common discrete-system analysis techniques. Assessment measures such as complex frequency response, amplitude response, phase response, phase delay and group delay are used.
  • the ideal fractional delay element has a constant amplitude response with unity gain, a linear phase, and constant phase or group delay times corresponding to the desired delay.
  • the corresponding measures must be evaluated for different values of d.
  • Fig. 3 shows by way of example the amplitude response as well as the phase delay of a third-order Lagrange interpolator for different delay values d.
  • Fig. 3a make one Dependence of the amplitude on the normalized frequency and
  • Fig. 3b a dependence of the phase delay of the normalized frequency.
  • FIGS. 3a, 3b For example, different graphs are shown for different values of d.
  • THD + N Total harmonic distortion + noise
  • the subjective rating can be done on the single channel as well as in the WFS setup. Similar conditions are used as in the informal hearing test outlined above.
  • the use of objective measurement methods to evaluate the perceived signals especially the PEAQ (Perceptual Evaluation of Audio Quality) method, may be considered. In doing so, quite good correspondences with the subjectively determined quality of perception and with objective quality measures can be ascertained. Nevertheless, the results of further investigations are critical, because z. For example, the PEAQ test for other applications (audio coding) was designed and parameterized.
  • Fig. 4 shows an example of such a continuous impulse response generated from a discrete, variable FD filter.
  • the continuous impulse response of a continuous variable fractional delay filter can be used to describe the behavior of such a structure.
  • This continuous form of description can be generated by determining the discrete impulse responses for many values of d and combining them into a (quasi-) continuous impulse response.
  • inter alia the behavior of FD filters when used for asynchronous sample rate conversion, so z.
  • quality measures for variable delay interpolation algorithms can be derived. Based on this, it can be tested whether the quality of such a variable filter can be influenced by the targeted influence on the properties of the continuous impulse response.
  • the change in delay times required in the playback of moving sources results in asynchronous sample rate conversion of the audio signals.
  • the suppression of aliasing and imaging effects is the biggest problem to be solved when implementing a sample rate conversion.
  • the methods should be examined for their properties to suppress such baseband mirrored frequencies. It is to be analyzed how the fractional-delay algorithms can be examined for their suppression of alias and image components. Based on this, the algorithms to be designed have to be adapted.
  • Wave field synthesis requires delay interpolation for each combination of virtual source and loudspeaker. Combined with the complexity of delay interpolation necessary for good rendering quality, good quality real-time implementation is not practical.
  • Lagrange interpolation is one of the most widely used methods for fractional-delay interpolation - it is one of the most attractive algorithms and is recommended for the first-to-be-tested algorithm for most applications.
  • Lagrange interpolation is based on the concept of polynomial interpolation. For an Nth order method, a polynomial of order N is calculated which passes through N + 1 support points surrounding the searched location.
  • Fig. 5 shows a so-called worst-case amplitude response for a Lagrange interpolator of different order.
  • the quality at high frequencies improves even with increasing interpolation order only slowly.
  • FIGS. 6a to 6c show representations for an amplitude response and a delay interpolation d.
  • Fig. 6a shows an example of an amplitude A of an audio signal as a function of time t.
  • a sampling of the audio signal takes place at the times t10, t11, t12, ...., t20, t21, etc.
  • the sample rate is thus given by 1 / (t10-t11) (assuming a constant sample rate). With a much lower frequency, the delay values are recalculated.
  • the delay values are recalculated.
  • the delay values are calculated at the times t10, t20 and t30, wherein a delay value d1 was calculated at the time t10, a delay value d2 at the time t20 and a delay value of d3 at the time t30 ,
  • the times at which delay values are recalculated may vary, for example, a new delay value can be generated every 32 bars, or else more than 1,000 cycles may occur between the calculation of new delay values. Between the delay values, the delay values are interpolated for the individual measures.
  • Fig. 6b shows an example of how the interpolation of the delay values d can be made.
  • Various interpolation methods are possible. The simplest interpolation consists in a linear interpolation (Lagrangian interpolation 1st order). Better interpolations are based on polynomials of higher order (Lagrangian interpolation of higher order), whereby the corresponding calculation requires more computing time.
  • Fig. 6b 2 it is shown how the delay value d1 is assumed at the time t10, the delay value d2 is present at the time t20 and the delay value d3 is present at the time t30.
  • An interpolation results, for example, in that there is a delay value d13 at the time t13.
  • the interpolation is chosen such that the interpolation values occur at the times t10, t20, t30, .... as part of the interpolated curve.
  • Fig. 6c again shows the amplitude A of the audio signal as a function of time t, with the interval between t12 and t14 being shown.
  • the delay value d13 obtained by interpolation at the time t13 now causes the amplitude to be shifted by the delay value d13 to the time ta at the time t13.
  • the shift is to smaller values in time, which is only one specific embodiment and may be different in other embodiments. If d13 has a fractional fraction, ta is not at a sampling instant. In other words, the access to A2 need not occur at a clock time, and an approximation (eg, rounding) results in the above-described problems that the present invention solves.
  • a synchronous sampling rate conversion takes place by a fixed, integer factor L or by the oversampling rate L. This is done via an up-sampling (insertion of L-1 zero samples after each input value). and a subsequent low-pass filtering to avoid image spectra. This operation can be performed efficiently by means of polyphase filtering.
  • the second step there is a fractional-delay interpolation between oversampled values. This is done by means of a variable fractional delay filter of low order whose coefficients are calculated directly become. Particularly useful here is the use of Lagrange interpolators (see above).
  • a linear interpolation between the outputs of a polyphase filter bank can be made.
  • the primary goal is to reduce the storage and computational power requirements needed for near non-rational ("crooked", incommensurate) sample rate ratios.
  • variable fractional delay elements can be based on dedicated structures under which the so-called Farrow structure (see below) is important.
  • DAAU asynchronous sample rate conversion
  • DAAU asynchronous sample rate conversion
  • a synchronous sample rate converter oversampling or rational sampling rate conversion
  • a DA / AD conversion system typically is realized by a variable fractional delay filter.
  • the large reduction of the filter order of the variable part allows a significant reduction of the calculation effort.
  • the particular advantage of the proposed method for use in wave-field synthesis is that the oversampling operation must be performed only once for each input signal, while the result of this operation can be used for all loudspeaker signals calculated by this renderer unit. This can be correspondingly higher calculation effort placed on the oversampling, especially to keep the error over the entire audio playback range low.
  • the variable fractional-delay filtering which must be performed separately for each output signal, can be performed much more efficiently due to the lower filter order required.
  • one of the major drawbacks of FD filters with explicitly calculated coefficients ie, above all Lagrange FD filters
  • their poor behavior at high frequencies is compensated for by the fact that they only have to operate in a much lower frequency range.
  • Fig. 7 shows a concrete representation of an oversampling delay interpolation according to a first embodiment of the present invention, wherein a simultaneous readout by means of Lagrange interpolation.
  • the discrete audio signal data x s (from the audio source 215) is over-sampled in this embodiment by oversampling in the sampler 236 and then stored in the delay line 216 according to the time order. This results in each memory of the delay line 216, a sample of a predetermined time tm (see Fig. 6a ).
  • the corresponding oversampled values in the delay line 216 may then be read out by the WFS delay and scale 212, with the pointer 217 corresponding to the sample of the delay value. This means that a pointer 217 located in the Fig.
  • fractional delay filters 222 give the component signal 115 off.
  • the component signals 115 (y i ) are then subsequently summed for various virtual sources x s and output to the corresponding loudspeakers (loudspeaker signals).
  • the design of the filters can be done statically outside the runtime of the application. Efficiency requirements for the filter design are thus irrelevant; powerful tools and optimization methods can be used.
  • n corresponds to the sampling frequency of the oversampled signal.
  • transition bands or do not care bands which do not specify the frequency response specifications. These transition bands are defined by the audio frequency band specified above to achieve a desired Cushioning necessary filter length. The result is a transition region in the range 2f c ⁇ f ⁇ 2 (f s -f c ).
  • f c is the desired upper limit frequency
  • f s is the sampling frequency of the non-oversampled signal.
  • FIG. 8 Figure 11 shows a specification of the frequency response of an anti-imaging filter for oversampling, where transition band 310 is specified for only one base band.
  • FIG. 9 Figure 11 shows a specification of an anti-imaging filter for oversampling, wherein so-called don't-care regions are also determined for images 310a, 310b, 310c of the transition band 310.
  • the additional don't-care bands may be defined on the reflections (images) of the original transition region 310.
  • the anti-imaging filter is almost exclusively designed as a linear phase filter. Phase errors should definitely be avoided at this point since the purpose of delay interpolation is to target the phase of the input signal. When implemented as a polyphase system, however, the linear phase does not apply to the sub-filters, so that the corresponding complexity savings can not be utilized.
  • the oversampled signal results from the insertion of L - 1 zero samples, there is a gain by the factor or the oversampling rate L, so that the original signal amplitude is maintained. This is possible by multiplying the filter coefficients with this factor without additional calculation effort.
  • the filter coefficients of the prototype filters involved in the Lagrangian interpolation are determined, multiplied by the corresponding Lagrangian weights and after Application of the necessary index shifts summed.
  • the algorithm can be analyzed with the criteria described in section 4 (frequency response, phase delay, continuous impulse response), without the peculiarities of the multi-rate processing to be considered.
  • the algorithm presented is a practical and relatively easy-to-implement approach to improve the delay interpolation.
  • the performance increase compared to a method for delay interpolation with direct calculation of the coefficients is very low. This is opposed to a significant reduction in playback errors, especially at higher frequencies.
  • the direct methods such as Lagrange interpolation
  • Critical to the performance of the method is the efficient extraction of integer and fractional delay parameters, the calculation the Lagrangian coefficients and the execution of the filtering.
  • the design tools used to determine the performance determining parameters are quite simple: L, N pp and N can be determined by external constraints or by experiment.
  • the filter design of the prototype filter is done using standard methods for low-pass filters, possibly using additional don't-care regions.
  • the Farrow structure is a variable filter structure for implementing a variable fractional delay. It is a structure based on an FIR filter whose behavior can be controlled by an additional parameter. For the Farrow structure, the fractional part or fraction of the delay is used as a parameter to map a controllable delay.
  • the Farrow structure though independently developed, is one manifestation of a variable digital filter.
  • variable characteristic is achieved by forming the coefficients of the FIR filter by polynomials.
  • d is the controllable parameter.
  • the output of the Farrow structure can thus be implemented as a polynomial in d, where the coefficients of the polynomial are the outputs of M fixed sub-filters C m (z) in FIR structure.
  • the polynomial evaluation can be realized efficiently by using the Horner scheme.
  • the output signals of the fixed sub-filters C m (z) are independent of a concrete fractionally rational delay d.
  • these values are useful as intermediate results that can be used to evaluate the output signals for all secondary sources.
  • Fig. 10 schematically shows this algorithm, which can also be summarized as follows. Simultaneous readout takes place on the basis of a Farrow structure, wherein the data of an audio signal x s are input to a delay line 216. However, in this embodiment, the audio data itself is not input but instead the coefficients c p are calculated as output values 239 of the Farrow structure (sub-filter 237) and stored in the delay line 216 according to their timing - in contrast to the previously shown embodiment (see FIG. Fig. 7 ). As before, the access to the delay line 216 is made by a pointer 217 whose position is again selected according to the integer part of the delay d.
  • the corresponding (delayed) loudspeaker signal y i can be calculated therefrom by means of power series in the delay value or the fractional (non-integer) part of the delay value (in a polynomial evaluation device 250) ,
  • the application of the Farrow structure is not bound to specific design methods for the determination of the coefficients c nm .
  • a minimization of the error integral Q ⁇ ⁇ 0 ⁇ 1 ⁇ ⁇ ⁇ 0 ⁇ 1 ⁇ ⁇ n ⁇ m c nm ⁇ e jn ⁇ T - e j ⁇ T 2 ⁇ d ⁇ d ⁇ respectively. This corresponds to a least squares optimization problem.
  • the Weighted Least Squares (WLS) method additionally defines a weighting function that allows the error to be weighted in the integration area.
  • WLS Weighted Least Squares
  • iterative methods can be designed with which specific influence on the error in certain regions of the integration surface can be taken, for For example, to minimize the maximum error.
  • Most WLS methods have poor numerical conditioning. This is not due to inappropriate methods but results from the use of don't-care regions in filter design. Therefore, only Farrow structures of comparatively small subfilter length N and polynomial order M can be designed with these methods, since otherwise numerical instabilities limit the accuracy of the parameters or prevent a convergence of the method.
  • the work area is defined as the area spanned by the desired frequency range and the permitted range for the control parameter d.
  • This type of optimization is usually referred to as minimax or Chebyshev optimization.
  • Chebyshev or Minimax optimization problems can generally be solved by linear optimization techniques. These methods are orders of magnitude more expensive than those based on the Remez exchange algorithm. However, they allow a direct formulation and solution of the design problem for the Farrow structure subfilters. In addition, these methods allow the formulation of additional constraints in the form of equality or inequality conditions. This is considered a very important feature for the design of asynchronous sample rate converters.
  • a method for minimax design for Farrow structures is based on algorithms for limited optimization (optimization methods that allow specification of constraints are called constrained optimization).
  • a particular feature of these Farrow structure design techniques is that separate specifications for amplitude and phase errors can be given. For example, the maximum phase error can be minimized while specifying an allowed maximum amplitude error. Together with accurate tolerance specifications for amplitude and phase errors, resulting, for example, from the perception of corresponding errors, this represents a very powerful tool for application-specific optimization of the filter structures.
  • a further development of the Farrow structure represents the proposed modified Farrow structure.
  • the sub-filters of an optimal Farrow filter are linear-phase. They have, for even and odd m, alternately symmetric and antisymmetric coefficients, so that the number of coefficients to be determined is halved.
  • the linear-phase structure of the C m (z) also allows the use of more efficient algorithms for calculating the sub-filter outputs.
  • a method for designing the Farrow structure is possible.
  • a method is based on a singular value decomposition and, based on this, efficient structures for implementation have also been developed. This method provides higher filter design accuracy with reduced filter complexity compared to WLS techniques, but does not provide the ability to specify constraints or selectively affect amplitude or phase error barriers.
  • the primary goal of the filter design is to minimize the deviation from the ideal fractional delay. Either the maximum error or the (weighted) average error can be minimized. Depending on the method used, either the complex error or phase and amplitude response can be specified separately.
  • the shape of the associated continuous impulse response (see above) has a major impact on the quality and perceptual quality of asynchronous sample rate conversion. Therefore, the use of constraints directly related to the continuous impulse response should be investigated. For example, continuity requirements can be specified.
  • the Farrow structure provides a very powerful filter structure for delay interpolation.
  • efficient partitioning of the algorithm into preprocessing per source signal as well as a low complexity evaluating operation that will be performed for each output signal can be implemented.
  • the Farrow structure-based algorithm for WFS can be efficiently implemented.
  • prefiltering can exploit reductions in the complexity resulting from the linear-phase sub-filter of the modified Farrow structure.
  • the evaluation of the precalculated coefficients as a polynomial evaluation by the Horner scheme is extremely efficient.
  • a major advantage of this filter structure is also the presence of closed design methods that allow for a targeted design.
  • the prefiltering introduced above is efficiently performed as a polyphase operation.
  • the input data is simultaneously convoluted with L different sub-filters, the outputs of which are multiplexed into the up-sampled output signal.
  • the filtering can be done by linear convolution or by fast convolution based on the FFT.
  • the Fourier transformation of the input data must take place only once and can then be used several times for simultaneous convolution with the subfilters.
  • a Park-McLellan algorithm designed low-pass filter (Matlab function firpm) of length 192 has a stopband attenuation of over 150 dB. This corresponds to a subfilter length of 48, longer filters can no longer be designed numerically stable. In any case, the results of the sub-filter operations must be nested in the output data stream.
  • One way to efficiently implement such a filter operation is to use library functions for polyphase or multirate filtering, e.g. From the Intel IPP library.
  • the preprocessing of the algorithm based on the Farrow structure can also be carried out efficiently by means of such a library function for multirate processing.
  • the sub-filters must be combined by nesting (interleaving) into a prototype filter, the output values of the function represent the interleaved output values.
  • the linearity of the subfilters designed according to the modified Farrow structure can also be exploited for the number of operations for the filtering to reduce.
  • a separate implementation is very likely to be necessary.
  • sample-accurate a per-sample calculated value of the delay parameter
  • Fractional delay algorithms require the division of the desired delay into an integer and a fractionally rational component.
  • the range [0 ... 1) is not mandatory, but the range can be, for example, as [-1 ⁇ 2 ... 1 ⁇ 2] or [(N-1) / 2 ... (N + 1) / 2) in the Lagrange interpolation.
  • this does not change the basic operation.
  • this operation has to be performed for each elementary delay interpolation and therefore has a significant impact on performance. Therefore, an efficient implementation is very important.
  • the WFS audio signal processing consists of a delay operation and a scaling of the delayed values for each audio sample and each combination of source signal and speaker. For efficient implementation, these operations are performed together. If these operations are carried out separately, a significant reduction in performance due to the expense of passing parameters, additional control flow and degraded code and data locality is to be expected.
  • both methods can be implemented and compared under quality and performance aspects. There are trade-offs between these aspects.
  • the influence of the improved delay interpolation on the overall reproduction quality of the WFS reproduction system can be examined under the influence of the other known reproduction errors. It is necessary to determine up to which interpolation quality an improvement of the overall system can be achieved.
  • One goal is to design methods that provide a quality of delay interpolation with reasonable effort achieve no perceptible interference even without masking effects from other WFS artifacts. This would also ensure future improvements to the playback system that the delay interpolation has no negative impact on the quality of the WFS playback.
  • Prefilter pre-filter stage
  • the combination of the two filters also offers the possibility of reducing the phase delay of the system induced by (especially linear-phase) filters, if this is only necessary in one filter component.
  • embodiments provide an implementation of a high-quality method for delay interpolation. as can be used, for example, in wave field synthesis systems.
  • Embodiments also provide further developments of the algorithms for wave field synthesis reproduction systems. Particular attention is paid to methods for delay interpolation, since these have a great influence on the reproduction quality of moving sources. Due to the quality requirements and the extremely high influence of these algorithms on the performance of the entire reproduction system, novel signal processing algorithms for wave field synthesis are required. As explained in detail above, it is thus possible in particular to consider interpolated fractions with a higher accuracy. The higher accuracy is reflected in a significantly improved listening experience. As described above, because of the increased accuracy, artifacts that occur especially with moving sources are hard to hear.
  • embodiments describe two efficient methods that meet these requirements and that have been developed, implemented, and analyzed.
  • the inventive scheme can also be implemented in software.
  • the implementation may be on a digital storage medium, in particular a floppy disk or a CD with electronically readable control signals, which may interact with a programmable computer system such that the corresponding method is executed.
  • the invention thus also consists in a computer program product with program code stored on a machine-readable carrier for carrying out the method according to the invention when the computer program product runs on a computer.
  • the invention can thus be realized as a computer program with a program code for carrying out the method when the computer program runs on a computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Stereophonic System (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Obtaining Desirable Characteristics In Audible-Bandwidth Transducers (AREA)
EP08801826.2A 2007-09-19 2008-09-03 Eine vorrichtung und ein verfahren zur ermittlung eines komponentensignals in hoher genauigkeit Active EP2189010B1 (de)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
DE102007044687 2007-09-19
DE102007059597A DE102007059597A1 (de) 2007-09-19 2007-12-11 Eine Vorrichtung und ein Verfahren zur Ermittlung eines Komponentensignals in hoher Genauigkeit
PCT/EP2008/007201 WO2009036883A1 (de) 2007-09-19 2008-09-03 Eine vorrichtung und ein verfahren zur ermittlung eines komponentensignals in hoher genauigkeit

Publications (2)

Publication Number Publication Date
EP2189010A1 EP2189010A1 (de) 2010-05-26
EP2189010B1 true EP2189010B1 (de) 2013-10-16

Family

ID=40384478

Family Applications (1)

Application Number Title Priority Date Filing Date
EP08801826.2A Active EP2189010B1 (de) 2007-09-19 2008-09-03 Eine vorrichtung und ein verfahren zur ermittlung eines komponentensignals in hoher genauigkeit

Country Status (7)

Country Link
US (2) US8526623B2 (ja)
EP (1) EP2189010B1 (ja)
JP (1) JP5132776B2 (ja)
KR (1) KR101119254B1 (ja)
CN (1) CN101868984B (ja)
DE (1) DE102007059597A1 (ja)
WO (1) WO2009036883A1 (ja)

Families Citing this family (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102004057500B3 (de) * 2004-11-29 2006-06-14 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zur Ansteuerung einer Beschallungsanlage und Beschallungsanlage
US20090077149A1 (en) * 2007-09-14 2009-03-19 Texas Instruments Incorporated Asynchronous sampling rate conversion
US8515052B2 (en) 2007-12-17 2013-08-20 Wai Wu Parallel signal processing system and method
EP2261896B1 (en) * 2008-07-29 2017-12-06 Yamaha Corporation Performance-related information output device, system provided with performance-related information output device, and electronic musical instrument
US8737638B2 (en) * 2008-07-30 2014-05-27 Yamaha Corporation Audio signal processing device, audio signal processing system, and audio signal processing method
US8346021B2 (en) * 2009-05-05 2013-01-01 Analog Devices, Inc. Content adaptive scaler based on a farrow structure
JP2012533954A (ja) * 2009-07-22 2012-12-27 ストーミングスイス・ゲゼルシャフト・ミト・ベシュレンクテル・ハフツング ステレオ又は疑似ステレオオーディオ信号の最適化装置及び方法
US8507704B2 (en) 2009-09-08 2013-08-13 Air Products And Chemicals, Inc. Liquid composition containing aminoether for deposition of metal-containing films
JP5782677B2 (ja) 2010-03-31 2015-09-24 ヤマハ株式会社 コンテンツ再生装置および音声処理システム
BR112013009301A2 (pt) * 2010-10-21 2016-07-26 Acoustic 3D Holdings Ltd gerador de difusão acústica
EP2573761B1 (en) 2011-09-25 2018-02-14 Yamaha Corporation Displaying content in relation to music reproduction by means of information processing apparatus independent of music reproduction apparatus
KR20130093783A (ko) 2011-12-30 2013-08-23 한국전자통신연구원 오디오 객체 전송 장치 및 방법
JP5494677B2 (ja) 2012-01-06 2014-05-21 ヤマハ株式会社 演奏装置及び演奏プログラム
DE102012200512B4 (de) * 2012-01-13 2013-11-14 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Berechnen von Lautsprechersignalen für eine Mehrzahl von Lautsprechern unter Verwendung einer Verzögerung im Frequenzbereich
CN102711015B (zh) * 2012-05-29 2015-03-25 苏州上声电子有限公司 基于二次剩余序列组合的扬声器阵列声场控制方法和装置
US9913064B2 (en) 2013-02-07 2018-03-06 Qualcomm Incorporated Mapping virtual speakers to physical speakers
JP6216553B2 (ja) * 2013-06-27 2017-10-18 クラリオン株式会社 伝搬遅延補正装置及び伝搬遅延補正方法
DE102013218176A1 (de) 2013-09-11 2015-03-12 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und verfahren zur dekorrelation von lautsprechersignalen
US10679407B2 (en) 2014-06-27 2020-06-09 The University Of North Carolina At Chapel Hill Methods, systems, and computer readable media for modeling interactive diffuse reflections and higher-order diffraction in virtual environment scenes
US9977644B2 (en) * 2014-07-29 2018-05-22 The University Of North Carolina At Chapel Hill Methods, systems, and computer readable media for conducting interactive sound propagation and rendering for a plurality of sound sources in a virtual environment scene
US9571265B2 (en) * 2015-07-10 2017-02-14 Tempo Semicondutor, Inc. Sample rate converter with sample and hold
DE102015214950A1 (de) * 2015-08-05 2017-02-09 Innovationszentrum für Telekommunikationstechnik GmbH IZT Vorrichtung zum Verändern einer Abtastrate, System umfassend eine Vorrichtung zum Verändern einer Abtastrate und Verfahren zum Verändern einer Abtastrate
EP3139635A1 (en) * 2015-08-28 2017-03-08 Alex Volkov Synchronization of audio streams and sampling rate for wireless communication
KR101691367B1 (ko) * 2015-10-23 2016-12-30 조선대학교산학협력단 M채널 TI-ADCs에서 미스매치에 대한 디지털 후면 교정 방법 및 그 장치
US9497561B1 (en) * 2016-05-27 2016-11-15 Mass Fidelity Inc. Wave field synthesis by synthesizing spatial transfer function over listening region
US9980078B2 (en) 2016-10-14 2018-05-22 Nokia Technologies Oy Audio object modification in free-viewpoint rendering
US11096004B2 (en) 2017-01-23 2021-08-17 Nokia Technologies Oy Spatial audio rendering point extension
US10248744B2 (en) 2017-02-16 2019-04-02 The University Of North Carolina At Chapel Hill Methods, systems, and computer readable media for acoustic classification and optimization for multi-modal rendering of real-world scenes
US10531219B2 (en) * 2017-03-20 2020-01-07 Nokia Technologies Oy Smooth rendering of overlapping audio-object interactions
US11074036B2 (en) 2017-05-05 2021-07-27 Nokia Technologies Oy Metadata-free audio-object interactions
US10165386B2 (en) 2017-05-16 2018-12-25 Nokia Technologies Oy VR audio superzoom
US11395087B2 (en) 2017-09-29 2022-07-19 Nokia Technologies Oy Level-based audio-object interactions
US11172318B2 (en) 2017-10-30 2021-11-09 Dolby Laboratories Licensing Corporation Virtual rendering of object based audio over an arbitrary set of loudspeakers
US10542368B2 (en) 2018-03-27 2020-01-21 Nokia Technologies Oy Audio content modification for playback audio
EP3900284B1 (en) * 2018-12-17 2023-11-08 U-blox AG Estimating one or more characteristics of a communications channel
CN109889185B (zh) * 2019-02-28 2023-03-28 深圳信息职业技术学院 一种信号插值滤波方法及插值滤波器
WO2021129936A1 (en) * 2019-12-23 2021-07-01 Advantest Corporation A signal processing arrangement for providing a plurality of output samples on the basis of a plurality of input samples and a method for providing a plurality of output samples on the basis of a plurality of input samples
CN111950186A (zh) * 2020-08-20 2020-11-17 沈阳师范大学 一种无理分数阶系统的有理化方法
WO2023127225A1 (ja) * 2021-12-28 2023-07-06 アルプスアルパイン株式会社 フィルタ設計方法、及び、ｉｉｒ型全域通過フィルタ

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6600495B1 (en) * 2000-01-10 2003-07-29 Koninklijke Philips Electronics N.V. Image interpolation and decimation using a continuously variable delay filter and combined with a polyphase filter

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5274708A (en) * 1992-06-01 1993-12-28 Fusan Labs, Inc. Digital stereo sound enhancement unit and method
JP3951122B2 (ja) 2002-11-18 2007-08-01 ソニー株式会社 信号処理方法および信号処理装置
US7822496B2 (en) 2002-11-15 2010-10-26 Sony Corporation Audio signal processing method and apparatus
DE10321980B4 (de) 2003-05-15 2005-10-06 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Berechnen eines diskreten Werts einer Komponente in einem Lautsprechersignal
JP4007255B2 (ja) 2003-06-02 2007-11-14 ヤマハ株式会社 アレースピーカーシステム
DE10355146A1 (de) 2003-11-26 2005-07-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Erzeugen eines Tieftonkanals
DE102005033239A1 (de) * 2005-07-15 2007-01-25 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Steuern einer Mehrzahl von Lautsprechern mittels einer graphischen Benutzerschnittstelle

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6600495B1 (en) * 2000-01-10 2003-07-29 Koninklijke Philips Electronics N.V. Image interpolation and decimation using a continuously variable delay filter and combined with a polyphase filter

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
BLEDA SERGIO ET AL: "Design and Implementation of a Compatible Wave Field Synthesis Authoring Tool", AES CONVENTION 118; MAY 2005, AES, 60 EAST 42ND STREET, ROOM 2520 NEW YORK 10165-2520, USA, 1 May 2005 (2005-05-01), XP040507210 *
FARROW C W: "A continuously variable digital delay element", 19880607; 19880607 - 19880609, 7 June 1988 (1988-06-07), pages 2641 - 2645, XP010069457 *
MURPHY N P ET AL: "Implementation of wideband integer and fractional delay element", ELECTRONICS LETTERS, IEE STEVENAGE, GB LNKD- DOI:10.1049/EL:19941157, vol. 30, no. 20, 29 September 1994 (1994-09-29), pages 1658 - 1659, XP006001148, ISSN: 0013-5194 *

Also Published As

Publication number Publication date
JP2010539833A (ja) 2010-12-16
US8526623B2 (en) 2013-09-03
US20100208905A1 (en) 2010-08-19
JP5132776B2 (ja) 2013-01-30
US20130243203A1 (en) 2013-09-19
WO2009036883A1 (de) 2009-03-26
CN101868984A (zh) 2010-10-20
DE102007059597A1 (de) 2009-04-02
US8605910B2 (en) 2013-12-10
KR101119254B1 (ko) 2012-03-16
EP2189010A1 (de) 2010-05-26
CN101868984B (zh) 2013-11-20
KR20100063071A (ko) 2010-06-10

Similar Documents

Publication Publication Date Title
EP2189010B1 (de) Eine vorrichtung und ein verfahren zur ermittlung eines komponentensignals in hoher genauigkeit
EP3117631B1 (de) Vorrichtung und verfahren zum verarbeiten eines signals im frequenzbereich
DE69819090T2 (de) Kompensationsfilter
EP1872620B9 (de) Vorrichtung und verfahren zum steuern einer mehrzahl von lautsprechern mittels einer graphischen benutzerschnittstelle
EP1844627B1 (de) Vorrichtung und verfahren zum simulieren eines wellenfeldsynthese-systemes
US8539012B2 (en) Multi-rate implementation without high-pass filter
DE102006047197B3 (de) Vorrichtung und Verfahren zum Verarbeiten eines reellen Subband-Signals zur Reduktion von Aliasing-Effekten
EP2656633B1 (de) Vorrichtung und verfahren zum berechnen von lautsprechersignalen für eine mehrzahl von lautsprechern unter verwendung einer verzögerung im frequenzbereich
DE69833749T2 (de) Filterbankanordnung und verfahren zur filterung und trennung eines informationssignals in unterschiedlichen frequenzbändern, insbesondere für audiosignale in hörhilfegeräten
DE102006053919A1 (de) Vorrichtung und Verfahren zum Erzeugen einer Anzahl von Lautsprechersignalen für ein Lautsprecher-Array, das einen Wiedergaberaum definiert
DE102005033238A1 (de) Vorrichtung und Verfahren zum Ansteuern einer Mehrzahl von Lautsprechern mittels eines DSP
WO2006117089A2 (de) Vorrichtung und verfahren zur generierung und bearbeitung von toneffekten in räumlichen tonwiedergabesystemen mittels einer graphischen benutzerschnittstelle
EP2754151B1 (de) Vorrichtung, verfahren und elektroakustisches system zur nachhallzeitverlängerung
EP2280482B1 (de) Filterbankanordnung für eine Hörvorrichtung
DE69834728T2 (de) Rekursiver filter für 3d-schall mit initialisierung des abgriffs einer verzögerungsleitung
EP2357854B1 (de) Verfahren und Vorrichtung zur Erzeugung individuell anpassbarer binauraler Audiosignale
DE112006002548T5 (de) Vorrichtung und Verfahren zur Wiedergabe von virtuellem Zweikanal-Ton
DE10317701B4 (de) Verfahren und Digitalsignalverarbeitungseinheit zur Erzeugung von Filterkoeffizienten für Digitalfilter mit veränderlicher Bandbreite
JP4364598B2 (ja) フィルタ処理装置，フィルタ処理方法及びそのプログラム
DE60210479T2 (de) Audiokodierer mit unregelmässiger filterbank
EP2485504B1 (de) Erzeugung von stillen Gebieten innerhalb der Zuhörerzone vielkanaliger Wiedergabesysteme
DE60028769T2 (de) Anti-aliasierte begrenzung bei begrenzter modulation mit stufen-funktion
DE602004010632T2 (de) Systeme und verfahren zur implementierung eines abtastratenconverters unter verwendung von hardware und software zur maximierung von geschwindigkeit und flexibilität
DE112021003767T5 (de) Signalverarbeitungsvorrichtung und -Verfahren und Programm
EP2503799A1 (de) Verfahren und System zur Berechnung synthetischer Außenohrübertragungsfunktionen durch virtuelle lokale Schallfeldsynthese

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20100317

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

17Q First examination report despatched

Effective date: 20101019

DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20130422

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 636985

Country of ref document: AT

Kind code of ref document: T

Effective date: 20131115

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502008010828

Country of ref document: DE

Effective date: 20131212

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20131016

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140116

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140217

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502008010828

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

26N No opposition filed

Effective date: 20140717

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502008010828

Country of ref document: DE

Effective date: 20140717

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140903

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140930

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140930

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140930

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140903

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 636985

Country of ref document: AT

Kind code of ref document: T

Effective date: 20140903

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140903

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140117

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20080903

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131016

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230524

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230921

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230918

Year of fee payment: 16

Ref country code: DE

Payment date: 20230919

Year of fee payment: 16