EP1994526B1 - Synthese et spatialisation sonores conjointes - Google Patents

Synthese et spatialisation sonores conjointes Download PDF

Info

Publication number
EP1994526B1
EP1994526B1 EP07731685A EP07731685A EP1994526B1 EP 1994526 B1 EP1994526 B1 EP 1994526B1 EP 07731685 A EP07731685 A EP 07731685A EP 07731685 A EP07731685 A EP 07731685A EP 1994526 B1 EP1994526 B1 EP 1994526B1
Authority
EP
European Patent Office
Prior art keywords
spatialization
parameters
source
channel
frequency component
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP07731685A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP1994526A1 (fr
Inventor
Grégory PALLONE
Marc Emerit
David Virette
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Orange SA
Original Assignee
France Telecom SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by France Telecom SA filed Critical France Telecom SA
Priority to PL07731685T priority Critical patent/PL1994526T3/pl
Publication of EP1994526A1 publication Critical patent/EP1994526A1/fr
Application granted granted Critical
Publication of EP1994526B1 publication Critical patent/EP1994526B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H7/00—Instruments in which the tones are synthesised from a data store, e.g. computer organs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2210/00—Aspects or methods of musical processing having intrinsic musical character, i.e. involving musical theory or musical parameters or relying on musical knowledge, as applied in electrophonic musical tools or instruments
    • G10H2210/155—Musical effects
    • G10H2210/265—Acoustic effect simulation, i.e. volume, spatial, resonance or reverberation effects added to a musical sound, usually by appropriate filtering or delays
    • G10H2210/295—Spatial effects, musical uses of multiple audio channels, e.g. stereo
    • G10H2210/301—Soundscape or sound field simulation, reproduction or control for musical purposes, e.g. surround or 3D sound; Granular synthesis
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
    • H04R2499/10—General applications
    • H04R2499/11—Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's

Definitions

  • the present invention relates to audio processing and, more particularly, to three-dimensional spatialization of synthetic sound sources.
  • non- parametric methods are known .
  • No particular parameter is used a priori to modify samples previously stored in memory.
  • the best-known representative of these methods is classical wave table synthesis.
  • transaural or " binaural
  • HRTFs Head Related Transfer Function
  • transfer functions representing the disturbance of acoustic waves by the morphology of an individual, these HRTFs functions being specific to this individual.
  • the sound reproduction is done in a manner adapted to the HRTFs of the listener, typically on two remote speakers (" transaural ") or from the two earpieces of a headset (" binaural ") .
  • Other techniques for example “ surround “ or “multichannel” (5.1 to 10.1 or more) rather provide a restitution on more than two speakers.
  • some HRTFs techniques use the separation of the frequency and position variables of the HRTFs, thus giving a set of p basic filters (corresponding to the first p eigenvalues of the covariance matrix of the HRTFs whose variables statistics are the frequencies), these filters being weighted by spatial functions (obtained by projection of the HRTFs on basic filters).
  • the spatial functions can then be interpolated, as described in the document US 5500900 .
  • Spatialization of many sound sources can be achieved through a multichannel implementation applied to the signal of each of the sound sources.
  • the gains of the spatialization channels are applied directly to the sound samples of the signal, often described in the time domain (but possibly also in the frequency domain). These samples sound are treated by a spatialization algorithm (with application of gains that depend on the desired position), regardless of the origin of these samples.
  • the proposed spatialization could apply to both natural and synthetic sounds.
  • each sound source must be synthesized independently (with a temporal or frequency signal), in order to then be able to apply independent spatialization gains.
  • N sound sources it is therefore necessary to perform N synthesis calculations.
  • the application of the gains to sound samples, whether they come from the time or frequency domain requires at least as many multiplications as there are samples.
  • M being the number of intermediate channels (surround channels for example)
  • N being the number of sources.
  • the so-called “ virtual loudspeakers” method makes it possible to encode the signals to be spatialized by applying them in particular gains, the decoding being done by convolution of the signals encoded by pre-calculated filters (Jérians Daniel, " Representation of acoustic fields, application to the transmission and reproduction of complex sound scenes in a multimedia context " , PhD Thesis, 2000).
  • an example embodiment that is referred to in this document WO-05/069272 and in which the sources are synthesized by associating amplitudes with frequencies constituting a " sound timbre " (for example a fundamental frequency and its harmonics), provides for grouping by identical frequencies synthesis signals, with a view to spatialization subsequent operating on the frequencies.
  • a sound timbre for example a fundamental frequency and its harmonics
  • amplitudes of a set at 0 j , a 1 j , ..., a p j to be assigned to a same source j can be zero if the corresponding frequencies are not represented in the sound signal of this source j .
  • the amplitudes a i 1 , ..., a i N relative to each frequency f i are grouped (" mixed ") to be applied, frequency by frequency, to the SPAT spatialization block for a frequency-based encoding (in binaural by example, then providing an interaural delay to apply to each source).
  • the signals of the channels c 1 ,..., C k , originating from the spatialization block SPAT, are then intended to be transmitted through one or more networks, or else stored, or other, for the purpose of a subsequent restitution (preceded by where appropriate, a suitable spatialization decoding).
  • the present invention improves the situation.
  • the present invention proposes for this purpose to first apply a spatialization encoding, then a " pseudo-synthesis " , the term “ pseudo " aiming at the fact that the synthesis applies in particular to the encoded parameters, derived from spatialization and not to usual synthetic sound signals.
  • a feature that the invention proposes is the spatial encoding of some synthesis parameters, rather than performing a spatial encoding of the signals corresponding directly to the sources.
  • This spatial encoding applies more particularly to synthesis parameters which are representative of an amplitude and it advantageously consists in applying to these few synthesis parameters spatialization gains which are calculated as a function of respective desired positions of the sources. It will thus be understood that the parameters multiplied by the gains in step b) and grouped in step c) are not really sound signals, as in the general prior art described above.
  • the present invention uses a mutual parametric synthesis where one of the parameters has the dimension of an amplitude. Unlike techniques of the prior art, it thus takes advantage of the advantages of such a synthesis to perform the spatialization.
  • the combination of synthesis parameter sets obtained for each of the sources advantageously makes it possible to globally control the encoded blocks of mutual parametric synthesis.
  • the present invention then makes it possible to spatialize simultaneously and independently of numerous synthesized sound sources from a parametric synthesis model, the spatialization gains being applied to the synthesis parameters rather than to the samples of the time or frequency domain. This embodiment thus ensures a substantial saving of the computing power required because it implies a low calculation cost.
  • the technique in the sense of the invention requires fewer calculations than the usual techniques in the sense of the prior art. For example, at the surround order 1 and in two dimensions (ie three intermediate channels), the invention already allows a calculation gain for only four sources to spatialize.
  • the present invention also makes it possible to reduce the number of gains to be applied. Indeed, the gains are applied to the synthesis parameters and not to the sound samples. Updating parameters such as the volume is generally less frequent than the sampling frequency of a signal, a calculation economy is thus achieved. For example, for a parameter update frequency (such as the volume in particular) of 200 Hz, a substantial saving in multiplication is achieved for a signal sampling frequency of 44100 Hz (in a ratio of about 200).
  • the fields of application of the present invention may concern both the musical field (including polyphonic ringtones of mobiles), the field of multimedia (including video game sound systems), the field of virtual reality (rendering of sound scenes). , simulators (synthesis of engine noise), or others.
  • new parameters p i m (i varying from 1 to N and m varying from 1 to M) are calculated by multiplying the parameters p i by the encoding gains g i m , obtained from the position of each of the sources.
  • the parameters p i m are combined (by summation in the example described) to provide the parameters p g m which feed M mutual parametric synthesis blocks.
  • These M blocks (referenced SYNTH (1) to SYNTH (M) on the figure 2 ) are constitutive SYNTH synthesis module, which delivers M time signals or frequency ss m (m ranging from 1 to M), obtained by synthesis from the parameters p g m .
  • These signals ss m can then feed a conventional block of spatial decoding, as will be seen later with reference to the figure 3 .
  • the synthesis used is an additive synthesis with application of an inverse Fourier transform (IFFT).
  • IFFT inverse Fourier transform
  • a set of N sources is characterized by a plurality of parameters p i, k representing the amplitude in the frequency domain of the k th frequency component for the i th source S i .
  • the parameter p i, k represents the amplitude of a given frequency component k for a given source S i .
  • p m i, k boy Wut m i - ⁇ p i , k , m varies from 1 to M.
  • the gains g m i are predetermined for a desired position for the source S i and according to the chosen spatialization encoding.
  • Each of the M time signals SS m (n) can then be supplied to a spatialization decoding block.
  • the adaptation filters from the surround format to the binaural format can be applied directly in the frequency domain, thus avoiding convolution in the time domain and a corresponding calculation cost.
  • each of the M frequency spectra ss m ( ⁇ ) is directly multiplied by the respective Fourier transforms of the temporal filters, noted Fg m ( ⁇ ) and Fd m ( ⁇ ) (adapted if necessary to have a coherent number of points), which is written:
  • Fg m ( ⁇ ) and Fd m ( ⁇ ) adapted if necessary to have a coherent number of points
  • the present invention also provides a device for generating synthetic and spatialized sounds, comprising in particular a processor, and in particular a working memory adapted to store instructions of the computer program product defined above.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Stereophonic System (AREA)
  • Golf Clubs (AREA)
  • Telephone Set Structure (AREA)
  • Telephone Function (AREA)
  • Circuit For Audible Band Transducer (AREA)
EP07731685A 2006-03-13 2007-03-01 Synthese et spatialisation sonores conjointes Active EP1994526B1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PL07731685T PL1994526T3 (pl) 2006-03-13 2007-03-01 Połączona synteza i uprzestrzennienie dźwięków

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR0602170 2006-03-13
PCT/FR2007/050868 WO2007104877A1 (fr) 2006-03-13 2007-03-01 Synthese et spatialisation sonores conjointes

Publications (2)

Publication Number Publication Date
EP1994526A1 EP1994526A1 (fr) 2008-11-26
EP1994526B1 true EP1994526B1 (fr) 2009-10-28

Family

ID=37400911

Family Applications (1)

Application Number Title Priority Date Filing Date
EP07731685A Active EP1994526B1 (fr) 2006-03-13 2007-03-01 Synthese et spatialisation sonores conjointes

Country Status (8)

Country Link
US (1) US8059824B2 (pl)
EP (1) EP1994526B1 (pl)
JP (1) JP5051782B2 (pl)
AT (1) ATE447224T1 (pl)
DE (1) DE602007002993D1 (pl)
ES (1) ES2335246T3 (pl)
PL (1) PL1994526T3 (pl)
WO (1) WO2007104877A1 (pl)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8786852B2 (en) 2009-12-02 2014-07-22 Lawrence Livermore National Security, Llc Nanoscale array structures suitable for surface enhanced raman scattering and methods related thereto
US9395304B2 (en) 2012-03-01 2016-07-19 Lawrence Livermore National Security, Llc Nanoscale structures on optical fiber for surface enhanced Raman scattering and methods related thereto
US9788135B2 (en) 2013-12-04 2017-10-10 The United States Of America As Represented By The Secretary Of The Air Force Efficient personalization of head-related transfer functions for improved virtual spatial audio
CN109691138A (zh) * 2016-10-04 2019-04-26 奥姆尼欧声音有限公司 立体声展开技术

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2679689B1 (fr) 1991-07-26 1994-02-25 Etat Francais Procede de synthese de sons.
JPH08502867A (ja) * 1992-10-29 1996-03-26 ウィスコンシン アラムニ リサーチ ファンデーション 指向性音を作る方法及び装置
US5596644A (en) * 1994-10-27 1997-01-21 Aureal Semiconductor Inc. Method and apparatus for efficient presentation of high-quality three-dimensional audio
FR2782228B1 (fr) 1998-08-05 2001-05-25 Ct Scient Tech Batiment Cstb Dispositif de simulation sonore et procede pour realiser un tel dispositif
FR2847376B1 (fr) * 2002-11-19 2005-02-04 France Telecom Procede de traitement de donnees sonores et dispositif d'acquisition sonore mettant en oeuvre ce procede
FI118247B (fi) * 2003-02-26 2007-08-31 Fraunhofer Ges Forschung Menetelmä luonnollisen tai modifioidun tilavaikutelman aikaansaamiseksi monikanavakuuntelussa
FR2851879A1 (fr) 2003-02-27 2004-09-03 France Telecom Procede de traitement de donnees sonores compressees, pour spatialisation.
WO2005069272A1 (fr) 2003-12-15 2005-07-28 France Telecom Procede de synthese et de spatialisation sonores
SE0400998D0 (sv) * 2004-04-16 2004-04-16 Cooding Technologies Sweden Ab Method for representing multi-channel audio signals
US8204261B2 (en) * 2004-10-20 2012-06-19 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Diffuse sound shaping for BCC schemes and the like
ES2380059T3 (es) * 2006-07-07 2012-05-08 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Aparato y método para combinar múltiples fuentes de audio codificadas paramétricamente
CN101681625B (zh) * 2007-06-08 2012-11-07 杜比实验室特许公司 用于从两个输入的音频信号获得两个环绕声音频通道的方法和设备

Also Published As

Publication number Publication date
DE602007002993D1 (de) 2009-12-10
PL1994526T3 (pl) 2010-03-31
US8059824B2 (en) 2011-11-15
JP2009530883A (ja) 2009-08-27
JP5051782B2 (ja) 2012-10-17
ATE447224T1 (de) 2009-11-15
ES2335246T3 (es) 2010-03-23
EP1994526A1 (fr) 2008-11-26
WO2007104877A1 (fr) 2007-09-20
US20090097663A1 (en) 2009-04-16

Similar Documents

Publication Publication Date Title
EP1600042B1 (fr) Procede de traitement de donnees sonores compressees, pour spatialisation
EP2000002B1 (fr) Procede et dispositif de spatialisation sonore binaurale efficace dans le domaine transforme
EP1992198B1 (fr) Optimisation d'une spatialisation sonore binaurale a partir d'un encodage multicanal
EP1999998B1 (fr) Procede de synthese binaurale prenant en compte un effet de salle
US8175280B2 (en) Generation of spatial downmixes from parametric representations of multi channel signals
EP2042001B1 (fr) Spatialisation binaurale de donnees sonores encodees en compression
US20060053018A1 (en) Advanced processing based on a complex-exponential-modulated filterbank and adaptive time signalling methods
KR102660704B1 (ko) 스펙트럼적 직교 오디오 성분 처리
EP1886535B1 (fr) Procede pour produire une pluralite de signaux temporels
EP1695335A1 (fr) Procede de synthese et de spatialisation sonores
EP1994526B1 (fr) Synthese et spatialisation sonores conjointes
EP3400599B1 (fr) Encodeur ambisonique ameliore d'une source sonore a pluralite de reflexions
EP3025514B1 (fr) Spatialisation sonore avec effet de salle
WO2006097633A1 (fr) Procede et systeme de spatialisation d'un signal sonore en fonction des qualites intrinseques de ce dernier

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20080828

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

17Q First examination report despatched

Effective date: 20090616

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 602007002993

Country of ref document: DE

Date of ref document: 20091210

Kind code of ref document: P

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2335246

Country of ref document: ES

Kind code of ref document: T3

LTIE Lt: invalidation of european patent or patent extension

Effective date: 20091028

REG Reference to a national code

Ref country code: PL

Ref legal event code: T3

NLV1 Nl: lapsed or annulled due to failure to fulfill the requirements of art. 29p and 29m of the patents act
PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100301

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100228

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100128

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

BERE Be: lapsed

Owner name: FRANCE TELECOM

Effective date: 20100331

26N No opposition filed

Effective date: 20100729

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20100331

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100129

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20100331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110331

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100429

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20100301

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20091028

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240220

Year of fee payment: 18

Ref country code: GB

Payment date: 20240220

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: PL

Payment date: 20240226

Year of fee payment: 18

Ref country code: FR

Payment date: 20240220

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20240402

Year of fee payment: 18