EP2517196A1 - Verfahren zur künstlichen wiedergabe eines ausgangssignals eines nichtlinearen zeitinvarianten systems - Google Patents

Verfahren zur künstlichen wiedergabe eines ausgangssignals eines nichtlinearen zeitinvarianten systems

Info

Publication number
EP2517196A1
EP2517196A1 EP10814673A EP10814673A EP2517196A1 EP 2517196 A1 EP2517196 A1 EP 2517196A1 EP 10814673 A EP10814673 A EP 10814673A EP 10814673 A EP10814673 A EP 10814673A EP 2517196 A1 EP2517196 A1 EP 2517196A1
Authority
EP
European Patent Office
Prior art keywords
signal
time invariant
invariant system
linear time
output signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP10814673A
Other languages
English (en)
French (fr)
Other versions
EP2517196B1 (de
Inventor
Lamberto Tronchin
Andrea VENTURI
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Universita di Bologna
Original Assignee
Universita di Bologna
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Universita di Bologna filed Critical Universita di Bologna
Publication of EP2517196A1 publication Critical patent/EP2517196A1/de
Application granted granted Critical
Publication of EP2517196B1 publication Critical patent/EP2517196B1/de
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H3/00—Instruments in which the tones are generated by electromechanical means
    • G10H3/12—Instruments in which the tones are generated by electromechanical means using mechanical resonant generators, e.g. strings or percussive instruments, the tones of which are picked up by electromechanical transducers, the electrical signals being further manipulated or amplified and subsequently converted to sound by a loudspeaker or equivalent instrument
    • G10H3/14—Instruments in which the tones are generated by electromechanical means using mechanical resonant generators, e.g. strings or percussive instruments, the tones of which are picked up by electromechanical transducers, the electrical signals being further manipulated or amplified and subsequently converted to sound by a loudspeaker or equivalent instrument using mechanically actuated vibrators with pick-up means
    • G10H3/18—Instruments in which the tones are generated by electromechanical means using mechanical resonant generators, e.g. strings or percussive instruments, the tones of which are picked up by electromechanical transducers, the electrical signals being further manipulated or amplified and subsequently converted to sound by a loudspeaker or equivalent instrument using mechanically actuated vibrators with pick-up means using a string, e.g. electric guitar
    • G10H3/186—Means for processing the signal picked up from the strings
    • G10H3/187—Means for processing the signal picked up from the strings for distorting the signal, e.g. to simulate tube amplifiers
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/02—Means for controlling the tone frequencies, e.g. attack or decay; Means for producing special musical effects, e.g. vibratos or glissandos
    • G10H1/06—Circuits for establishing the harmonic content of tones, or other arrangements for changing the tone colour
    • G10H1/16—Circuits for establishing the harmonic content of tones, or other arrangements for changing the tone colour by non-linear elements

Definitions

  • the invention relates to a method for reproducing an output signal of a non-linear time invariant system, in particular a method used for example for artificially reproducing a particular acoustic effect going close to the real one.
  • acoustic effect can be, for example, the sound that can produce a sound chest of a particular musical instrument when it is played or a sound amplified by a non-linear amplifier such as a tube amplifier.
  • each of the aforesaid systems namely a sound chest of a musical instrument or a tube amplifier, or the sound produced by combinations of the aforesaid systems, is. a nonlinear system.
  • each of the aforesaid systems modifies the input signals sent, so the corresponding output signals are distorted with respect the respective input signals.
  • the output signal has different frequency components compared to the input signal.
  • the output signal may have a plurality of harmonics at frequencies that are different the one from each other and different from the frequency/frequencies of the input signal, even if the input signal has only one component at fundamental frequency.
  • each system characterizes the sound generated by each system, making the sound unique and recognizable among others. This means that each sound generated by a system differs from the sound generated by another system because of its harmonic content. It follows that a system is distinguished from another one because of the ' harmonic distortions it introduces in the sound produced by the system.
  • a distorting system can be an overdrive device that makes possible, by suitable amplifying means, to amplify an audio signal until the amplifier is in a saturation condition, generating an overloaded and distorted output signal.
  • Another distorting system can be a device that modifies the wave form of an audio signal sent to the input thereof, for example by subjecting it to a squaring process. It follows that the output audio signal is distorted compared to the input .
  • overdrive devices and distorting devices used in the musical field for example associated to an electrical guitar, intentionally reproduce distorting signals, by introducing in the spectrum of an output audio signal from the device, additional harmonics which are not present in the corresponding input audio signal from the overdrive device and/or the distorting device.
  • An object of the invention is to give a method for artificially reproducing an output signal of a non- linear time invariant system, such as for example the sound of a particular specimen instrument.
  • Another object is to obtain a method for artificially reproducing, economically, the output signal of a non-linear time invariant system, such as a tube amplifier that is typically very expensive.
  • Figure 1 is a scheme of a non-linear time invariant system
  • Figure 2 is a scheme showing a model (Hammerstein model) that represents the non-linear time invariant system in Figure 1 ;
  • Figure 3 is the spectrogram in linear scale of an input signal of the non-linear time invariant system, when such input signal is a signal of the exponential sine sweep type;
  • Figure 4 is the spectrogram like the one in Figure 3 using a logarithmic scale;
  • Figure 5 is the spectrogram in linear scale of an output signal of a non- linear time invariant system when the input signal is a signal of the exponential sine sweep type;
  • Figure 6 is the spectrogram like the one in Figure 5 using a logarithmic scale
  • Figure 7 is the spectrogram of the inverse of a signal of the exponential sine sweep type in logarithmic scale
  • Figure 8 is the spectrogram of the inverse convolution of a non-linear time invariant system subjected to an exponential sine sweep
  • Figure 9 is a diagram of the inverse convolution of a nonlinear time invariant system subjected to an exponential sine sweep
  • Figure 10 is the reproduction of a spectrogram of an output signal of the non-linear time invariant system, when an input signal of the exponential sine sweep type is sent to it;
  • Figure 11 shows the amplitude diagram of the frequency response of a signal of the Dirac Delta type and its waveform
  • Figure 12 shows how the problems relating to the phase deteriorates a Dirac Delta signal
  • Figure 13 shows the result of an emulation of a non-linear time invariant system without considering the phase problems
  • Figure 14 shows the impulse response of a FIR filter that is able to correct the phase problems once it is applied to the signal in Figure 12;
  • Figure 15 and Figure 16 shows the achieved results once the phase problems have been corrected.
  • a non-linear time invariant system 1 is schematically shown with a rectangle, the system having an input signal and an output signal, that are for example audio signals, expressed, in the time domain, as x(t) and y(t) respectively.
  • a non- linear time invariant system with memory such as for example the sound chest of a musical instrument, for example that o a violin, can be modeled by Volterra series:
  • the non-linear system 1 can be simplified in the series of two systems: a non-linear time invariant system without memory and a linear time invariant system with memory.
  • the output signal y(t) of such model (Hammerstein model) applies and is reported schematically in Figu 2:
  • the method for reproducing an output signal from the nonlinear time invariant system 1 uses a measurement technique of the impulse response of a linear system that uses as input signal x(t) a signal of the sine sweep type, i.e. a sine signal with frequency that varies from a starting frequency f 0 to a final frequency f 1 in T seconds.
  • the input signal x(t) thus is:
  • g(t) is a function defined as the integral of a function of the exponential type f(t) that has the following formu
  • Integrating f(t) gives the value of g(t), i.e.
  • Figures 3 and 4 show, respectively, the spectrograms of the input signal x(t) of the non-linear time invariant system 1 in linear and in logarithmic scale, when the input signal x(t) is a signal of the sine sweep type.
  • Figures 5 and 6 show, respectively, in linear and logarithmic scale the spectrograms of the output signal y(t) of the non- linear time invariant system 1 when the input signal x(t) of the non-linear time invariant system 1 is a signal of the exponential sine sweep ss(t) type.
  • a well known feature of the signal of the linear sine sweep type is that its reverse reproduction is also its "inverse”: defining as "inverse" of the waveform x(t) that waveform for which is valid , in which t 0
  • Figure 7 shows the spectrogram of the inverse of a signal of the sine sweep type
  • Figure 8 shows the spectrogram of a ' signal representing the inverse convolution of the output of a non-linear time invariant system subjected to an exponential sine sweep.
  • This latter graph is composed by a plurality of lines mutually parallel, which in a time- frequency plot are parallel with the frequency axis.
  • the input signal x(t) of the linear time invariant system 1 is a signal ss(t) of the exponential sine sweep type [that is ss( ⁇ (t)), in order to better underline that the frequency depends on the time] having the frequency which ranges from the starting value f 0 to a final frequency f 1 during T seconds, considering the (3) we will obtain:
  • A(t) is a constant term as well as the term that represents a DC offset, and can therefore be removed by- using a filter of the high-pass type, since it is not relevant for the calculation of the kernels of Volterra series expansion.
  • the method provides for moving from time domain to frequency domain using the Fourier transforms.
  • ⁇ ( ⁇ ) denotes the Fourier transform of
  • the non-linear time invariant system 1 allows the output signal of a non-linear system to be obtained, once the input signal is known, with excellent degree of approximation, and therefore it allows to artificially emulate the behavior of the non- linear time invariant system 1, for example by means of a data processing system.
  • the non-linear system 1 is a musical instrument, for example a particular specimen violin like a Stradivari or else, it is possible to obtain its characteristics by mechanically exciting the bridge of the violin with a stress having a sine sweep pattern, recording the produced sound and applying the aforementioned calculation method.
  • the first equation in (17) states that by convolving a sine sweep, e.g. of 15 seconds from 20Hz to 48kHz, with its inverse, it results into a waveform of the Dirac Delta type. This result is always verified as shown in Figure 11.
  • the amplitude diagram of the frequency response shows a flat spectrum and also the waveform has a shape which can be compared to a Dirac Delta.
  • the planned solution for solving these problems consists, in this case, in designing 4 FIR filters which, once they have been applied to the Dirac Deltas with the aforementioned problems (derived from the second, third, fourth and fifth, equations in (17) , respectively) , are able to "re-align" the phase, bringing back the signal to shapes of the type as shown in Figure 11.
  • Figure 14 shows first the impulse response of the FIR filter designed for correcting the phase problems of the signal shown in Figure 12, and then the Dirac Delta obtained after the phase correction.
  • Figure 16 shows a comparison between the emulation method obtained without the phase correction and with the correction here proposed.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Nonlinear Science (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Electrophonic Musical Instruments (AREA)
  • Telephonic Communication Services (AREA)
  • Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
EP10814673.9A 2009-12-23 2010-12-23 Verfahren zur künstlichen wiedergabe eines ausgangssignals eines nichtlinearen zeitinvarianten systems Not-in-force EP2517196B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
ITMO2009A000313A IT1397934B1 (it) 2009-12-23 2009-12-23 Metodo per riprodurre artificialmente un segnale d'uscita di un sistema non lineare tempo invariante.
PCT/IB2010/056059 WO2011077408A1 (en) 2009-12-23 2010-12-23 Method for artificially reproducing an output signal of a non-linear time invariant system

Publications (2)

Publication Number Publication Date
EP2517196A1 true EP2517196A1 (de) 2012-10-31
EP2517196B1 EP2517196B1 (de) 2014-06-18

Family

ID=42289387

Family Applications (1)

Application Number Title Priority Date Filing Date
EP10814673.9A Not-in-force EP2517196B1 (de) 2009-12-23 2010-12-23 Verfahren zur künstlichen wiedergabe eines ausgangssignals eines nichtlinearen zeitinvarianten systems

Country Status (5)

Country Link
US (1) US9171534B2 (de)
EP (1) EP2517196B1 (de)
DK (1) DK2517196T3 (de)
IT (1) IT1397934B1 (de)
WO (1) WO2011077408A1 (de)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9848262B2 (en) 2016-03-23 2017-12-19 Harman International Industries, Incorporated Techniques for tuning the distortion response of a loudspeaker
US10985951B2 (en) 2019-03-15 2021-04-20 The Research Foundation for the State University Integrating Volterra series model and deep neural networks to equalize nonlinear power amplifiers

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5802182A (en) * 1994-07-27 1998-09-01 Pritchard; Eric K. Audio process distortion

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2011077408A1 *

Also Published As

Publication number Publication date
WO2011077408A1 (en) 2011-06-30
DK2517196T3 (da) 2014-10-06
IT1397934B1 (it) 2013-02-04
ITMO20090313A1 (it) 2011-06-24
US9171534B2 (en) 2015-10-27
US20120328128A1 (en) 2012-12-27
EP2517196B1 (de) 2014-06-18

Similar Documents

Publication Publication Date Title
US7161515B2 (en) Calibration system and method for a linearity corrector using filter products
JP6552462B2 (ja) 楽器の特性をモデル化する方法
JP6452653B2 (ja) 楽器の特性をモデル化するシステム
CN101247671B (zh) 变换器参数的最优估计
US11164551B2 (en) Amplifier matching in a digital amplifier modeling system
Eichas et al. Block-oriented modeling of distortion audio effects using iterative minimization
US20110033057A1 (en) Method and Apparatus for Distortion of Audio Signals and Emulation of Vacuum Tube Amplifiers
EP2517196B1 (de) Verfahren zur künstlichen wiedergabe eines ausgangssignals eines nichtlinearen zeitinvarianten systems
Dietrich Uncertainties in acoustical transfer functions: modeling, measurement and derivation of parameters for airborne and structure-borne sound
JPS63234699A (ja) 音場補正装置
Carini et al. Room impulse response estimation using perfect sequences for Legendre nonlinear filters
Eichas System identification of nonlinear audio circuits
Eichas et al. Feature design for the classification of audio effect units by input/output measurements
DE102019005855B4 (de) Verfahren zum Anpassen eines Klangwandlers an einen Referenzklangwandler
GB2426404A (en) Measuring non-linear distortion in transducers
EP2080405A1 (de) Nichtlineare signalverarbeitung
JP6505546B2 (ja) 音響効果付与装置
JPH0438477A (ja) オーディオ増幅器の遅延時間測定方法
Dempwolf et al. The influence of small variations in a simplified guitar amplifier model
JP4244897B2 (ja) アナログ機器のシミュレータ
Tarr The Development of Audio Software with Distortion
JP4132693B2 (ja) イコライザ
Bargum et al. Differentiable Allpass Filters for Phase Response Estimation and Automatic Signal Alignment
Eichas System Identification of Nonlinear Audio Circuits: Von der Fakultät für Elektrotechnik der Helmut-Schmidt-Universität/Universität der Bundeswehr Hamburg zur Erlangung des akademischen Grades eines Doktor-Ingenieurs genehmigte Dissertation vorgelegt von
Axelson-Fisk Caring More About EQ Than IQ: Automatic Equalizing of Audio Signals

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20120716

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20140107

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 673733

Country of ref document: AT

Kind code of ref document: T

Effective date: 20140715

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602010016895

Country of ref document: DE

Effective date: 20140731

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

Effective date: 20141001

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140918

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140919

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20140618

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 673733

Country of ref document: AT

Kind code of ref document: T

Effective date: 20140618

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20141020

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20141018

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602010016895

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20150319

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20141223

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20141223

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20141231

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20141231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20101223

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 7

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FI

Payment date: 20171121

Year of fee payment: 8

Ref country code: DK

Payment date: 20171121

Year of fee payment: 8

Ref country code: FR

Payment date: 20171121

Year of fee payment: 8

Ref country code: DE

Payment date: 20171120

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20171121

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20171221

Year of fee payment: 8

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140618

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602010016895

Country of ref document: DE

REG Reference to a national code

Ref country code: DK

Ref legal event code: EBP

Effective date: 20181231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181223

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20181223

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181231

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181223

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190702

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181223

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181231