EP2515300A1 - Verfahren und System für die Geräuschunterdrückung - Google Patents

Verfahren und System für die Geräuschunterdrückung Download PDF

Info

Publication number
EP2515300A1
EP2515300A1 EP12163679A EP12163679A EP2515300A1 EP 2515300 A1 EP2515300 A1 EP 2515300A1 EP 12163679 A EP12163679 A EP 12163679A EP 12163679 A EP12163679 A EP 12163679A EP 2515300 A1 EP2515300 A1 EP 2515300A1
Authority
EP
European Patent Office
Prior art keywords
signal
module
output
sub
noise
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP12163679A
Other languages
English (en)
French (fr)
Other versions
EP2515300B1 (de
Inventor
Pascal Saguin
Karim Maouche
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Adeunis RF SA
Original Assignee
Adeunis RF SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Adeunis RF SA filed Critical Adeunis RF SA
Publication of EP2515300A1 publication Critical patent/EP2515300A1/de
Application granted granted Critical
Publication of EP2515300B1 publication Critical patent/EP2515300B1/de
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G10L25/18—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band

Definitions

  • the present invention relates to a noise reduction method and an associated noise reduction system.
  • It relates more particularly to a method and a system for reducing noise on a noisy acoustic signal y (t) from a microphone operating in a noisy environment.
  • the present invention finds particular application in full-duplex, wireless and mono-sensor audio communication systems, in other words mono-microphone, which make it possible to establish audio communication between several users, in an autonomous manner (ie ie without connection to a transmission base or to a network) and which is easy to use (that is to say not requiring any intervention of a technician to establish the communication).
  • Such communication systems are generally used in a noisy environment, such as a marine environment or a theater, or extremely noisy, such as a construction site or a hall or stadium hosting a sporting event.
  • the present invention aims to provide a method and a noise reduction system adapted to highly noisy environments and where the noise component is highly non-stationary and evolves relatively quickly over time.
  • the invention proposes to implement a multi-band spectral subtraction algorithm "SSMB" which consists of sharing the entire spectral band in sub-bands and to adapt in each subband the subtraction calculation between the noisy signal Y k, 1 and the noise component D k, 1 for extracting an output denoised signal SD k, l ; applying the spectral subtraction operation in each sub-band to improve the sensitivity of the noise reduction method.
  • SSMB multi-band spectral subtraction algorithm
  • step a the extraction of the phase ⁇ k, 1 , of the noisy signal Y k, 1 is optional. It is indeed possible, during step e), to determine the output denoised signal SD k, l from the square modules
  • the so-called "MCRA" method uses the following principle: on a finite horizon window and for a given frequency, the minimum of the power spectral density (DSP) of the noisy signal Y k, l corresponds to the value of the spectral density of power (DSP) of the noise component D k, l .
  • DSP power spectral density
  • Such an algorithm is particularly advantageous since it takes into account the value of the signal-to-noise ratio SNR k, l, i specific to each sub-band SB i to perform the spectral subtraction in each sub-band.
  • the implementation of this multi-band spectral subtraction algorithm "SSMB" allows processing with different parameters depending on the sub-band where the 'We are.
  • the noise component can be reduced more significantly in a sub-band if it is more dominant, unlike a full-band subtraction algorithm that will reduce noise equally all the sub-bands of the spectrum and will therefore be less precise and therefore less effective.
  • ⁇ c1 , ⁇ c2 , ⁇ c3 and ⁇ c4 are predetermined constants
  • SNR 1 and SNR 2 are predetermined thresholds.
  • ⁇ c1 , ⁇ c2 , ⁇ c3 and ⁇ c4 and SNR 1 and SNR 2 are chosen experimentally, in particular by numerical simulation.
  • SNR 1 - 5 dB
  • SNR 2 20 dB
  • ⁇ vs ⁇ 1 5
  • ⁇ vs ⁇ 2 4
  • ⁇ vs ⁇ 3 - 0.15
  • ⁇ vs ⁇ 4 1
  • step e) consists of determining the denoised signal X k, 1 from the square modules
  • 2 debruitées components of sub-band X k, l, i , and optionally phases ⁇ k, l extracted in step a), so that the output unedited signal SD k, l corresponds to the denoised signal X k , l , ie SD k, l X k, l .
  • the output unedited signal SD k, 1 corresponds to the denoised signal X k, l whose square modules
  • 2 are directly derived from step d) implementing the multi-band spectral subtraction algorithm "SSMB".
  • the processed signal i.e., the denoised signal X k, l
  • the processed signal may suffer in terms of the quality and intelligibility of a distortion problem.
  • This distortion whose origin is usually the noise reduction process, depends on the parameters of noise reduction algorithms ("MCRA” and "SSMB” algorithms) but also on the level and type of noise component to be reduced.
  • step d) implementing the multi-band spectral subtraction algorithm "SSMB"
  • a step of reinjection of the noisy signal Y k is carried out , l from the microphone in the denoised signal X k, l ; this reinjection being controlled by the amplification coefficient (otherwise called attenuation parameter) chosen very low, of the order of a few percent.
  • the reinjected signal corresponds to the signal ⁇
  • the step f.2) involves reconstructing the voice signal output denoised sd (t) only from the Sd output signals from the the step f.1), these output signals Sd l corresponding to the inverse Fourier transforms of the output denoised signal SD k, l specific to each time frame I.
  • the speech output speech signal sd (t) is reconstructed from the only output signals Sd I from step f.1).
  • This second embodiment allows somehow the deactivation of steps b), c) and d) noise reduction when the noise is not present. As a result, the distortions that can be brought about by the noise reduction processing, in this situation of absence or near absence of noise, will be eliminated.
  • This OLA overlap and add method is a typical time reconstruction method that uses juxtaposed weighting windows (ie overlapping or overlapping windows). then who Adds the output signals taking into account the overlap of the time frames.
  • the system implements a step of reinjection of a very small part of the noisy signal Y k, 1 coming from the microphone into the denoised signal X k, 1 to remedy at least in part the distortion problems induced by the estimation modules "MCRA” and "SSMB".
  • the system 1 comprises a conversion unit 2 of the noisy acoustic signal y (t) in the time domain into a noisy signal Y k, 1 in the frequency domain.
  • This conversion unit 2 comprises a switching module 21 of the noisy acoustic signal y (t) in sampled signals y 1 in successive time frames I.
  • the noisy acoustic signal y (t) is cut into frames of 240 samples, which at a sampling frequency of 8 kHz corresponds to time frames of 30 milliseconds.
  • the successive time frames overlap or overlap.
  • successive time frames overlap on 120 samples, which corresponds to fifty percent (50%) of overlap.
  • This overlap of the time frames is intended to allow the implementation of a method of recovery and addition called "OLA", which allows the initial temporal division of the noisy acoustic signal y (t) then the final restitution at the output of the system 1 in the time domain.
  • OVA a method of recovery and addition
  • This conversion unit 2 comprises, at the output of the chopper module 21, a windowing module 22 of the sampled signals y 1 by application of a weighting window, in particular of the Hanning window or Hamming window type, in order to output weighted sampled signals ⁇ y 1 ⁇ .
  • a weighting window in particular of the Hanning window or Hamming window type
  • time frames are then apodized with a weighting window, before applying a Fourier transform, in order to to minimize the edge effects due to the blanking cut performed by the cutting module 21.
  • This conversion unit 2 comprises, at the output of the windowing module 22, a calculation module 23 of a discrete Fourier transform which outputs the noisy signal Y k, l .
  • the switching module 21 receives as input the noisy acoustic signal y (t) and outputs the sampled signal y 1 , where I is the time index (or index of the time frame).
  • the calculation of the Discrete Fourier Transform is for example carried out by a Fast Fourier Transform (FFT) calculation with a size N which can be equal to 256 (N corresponds to the number of points of the Fourier transform).
  • FFT Fast Fourier Transform
  • the system 1 comprises, at the output of the conversion unit 2, a digital processing unit 3 in the frequency domain which carries out denoising or speech enhancement processing on the noisy signal Y k, l .
  • This digital processing unit 3 comprises a first extraction module 31 of the square module
  • this digital processing unit 3 comprises a second extraction module 32 of the phase ⁇ k, 1 of the noisy signal Y k, l . As detailed later, it is also conceivable to dispense with this second extraction module 32.
  • the digital processing unit 3 comprises, at the output of the first extraction module 31, an estimation module 33, called "MCRA", of a noise component D k, 1 contained in the noisy signal Y k, l from the square module
  • MCRA estimation module 33
  • the estimation module "MCRA” 33 thus outputs the noise component D k, l .
  • the frequency band can for example be divided into three frequency sub-bands, namely SB 1 for f i ⁇ 1000 Hz, SB 2 for 1000 Hz ⁇ f i ⁇ 2000 Hz and finally SB 3 for f i > 2000 Hz, where f i is the subband frequency.
  • the digital processing unit 3 comprises, at the output of the estimation module "MCRA” 33 and the module of division 34 of the frequency band, an estimation module 35, called “SSMB", of the square module
  • the principle of the multi-band spectral subtraction algorithm proceeds from a generalization of the spectral subtraction algorithm which consists in subtracting from the spectral power density of the noisy signal Y k, l coming from the microphone a portion of the spectral power density of the noise component estimated by the "MCRA" method.
  • This multi-band spectral subtraction algorithm is combined with the so-called OLA overlay and addition method.
  • ⁇ i ⁇ ⁇ vs ⁇ 1 if SNR k , l , i ⁇ SNR 1 ⁇ vs ⁇ 2 + ⁇ vs ⁇ 3 ⁇ SNR k , l , i if SNR 1 ⁇ SNR k , l , i ⁇ SNR 2 ⁇ vs ⁇ 4 if SNR k , l , i > SNR 2
  • ⁇ c1 , ⁇ c2 , ⁇ c3 and ⁇ c4 are predetermined constants
  • SNR 1 and SNR 2 are the predetermined thresholds.
  • ⁇ i ⁇ 5 if SNR k , l , i ⁇ - 5 4 - 3 20 ⁇ SNR k , l , i if - 5 ⁇ SNR k , l , i ⁇ 20 1 if SNR k , l , i > 20
  • ⁇ i ⁇ 1 if f i ⁇ 1000 first sub - bandaged SB 1 2.75 if 1000 ⁇ f i ⁇ 2000 second sub - bandaged SB 2 1.75 if f i > 2000 third sub - bandaged SB 3
  • the digital processing unit 3 also comprises, at the output of the estimation module "SSMB" 35 and the second extraction module 32, a module 36 for determining an output denoised signal SD k, l from the modules square
  • the system 1 implements a step of reinjection of a very small part of the noisy signal Y k , 1 , coming from the microphone, into the denoised signal X k, l ; the reinjected signal corresponding to the amplified signal ⁇
  • the distortion problems introduced by the "MCRA” 33 and "SSMB” estimation modules 35 are remedied.
  • the system 1 comprises, at the output of the digital processing unit 3, a conversion unit 4 in the time domain.
  • This conversion unit 4 comprises a calculation module 41 of an output signal Sd 1 specific to each time frame I by the application of an inverse Fourier transform of the output denoised signal SD k, l .
  • IFFT inverse fast Fourier Transform
  • This conversion unit 4 comprises, at the output of the calculation module 41, a reconstruction module 42 of a speech output signal sd (t) in the time domain from the output signals Sd 1 .
  • the time frame of the output denoised signal SD k, 1 passes through the calculation module 41 of the inverse Fourier transform, then its first half (N / 2 first sampling points). is added with the second half (N / 2 last sampling points) saved from the previous frame, while its second half (N / 2 first sampling points) is saved for the next block.
  • N / 2 which is equal to 128, being reminded that the recovery rate of two successive frames is fifty percent (50%).
  • the system 1 does not include the second extraction module 32 of the phase ⁇ k, 1 of the noisy signal Y k, l .
  • the recombination sub-module 362 of the de-banded components of sub-band X k, l, i calculates the denoised signal X k, l from the modules
  • System 1 of the figure 3 is a variant of the system 1 of the figure 2 without the second extraction module 32, but it is of course also conceivable to provide for the removal of the second extraction module 32 in the system 1 of the figure 1 .
EP20120163679 2011-04-22 2012-04-11 Verfahren und System für die Geräuschunterdrückung Not-in-force EP2515300B1 (de)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
FR1153507A FR2974443B1 (fr) 2011-04-22 2011-04-22 Procede et systeme de reduction du bruit

Publications (2)

Publication Number Publication Date
EP2515300A1 true EP2515300A1 (de) 2012-10-24
EP2515300B1 EP2515300B1 (de) 2014-05-14

Family

ID=44243603

Family Applications (1)

Application Number Title Priority Date Filing Date
EP20120163679 Not-in-force EP2515300B1 (de) 2011-04-22 2012-04-11 Verfahren und System für die Geräuschunterdrückung

Country Status (4)

Country Link
EP (1) EP2515300B1 (de)
ES (1) ES2492698T3 (de)
FR (1) FR2974443B1 (de)
PT (1) PT2515300E (de)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111854920A (zh) * 2020-07-24 2020-10-30 贵州电网有限责任公司 一种基于dvs振动监测信号的预处理方法及系统
CN113367676A (zh) * 2021-06-07 2021-09-10 广东电网有限责任公司 一种用于可穿戴式设备的ppg运动噪声滤波方法及装置
CN116495946A (zh) * 2023-06-27 2023-07-28 珠江水利委员会珠江水利科学研究院 一种用于城市雨污水截污调蓄和清淤综合处理方法及系统

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7016507B1 (en) * 1997-04-16 2006-03-21 Ami Semiconductor Inc. Method and apparatus for noise reduction particularly in hearing aids

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7016507B1 (en) * 1997-04-16 2006-03-21 Ami Semiconductor Inc. Method and apparatus for noise reduction particularly in hearing aids

Non-Patent Citations (8)

* Cited by examiner, † Cited by third party
Title
"Multi-band Spectral Subraction for Enhancing Speech Corrupted by Colored Noise", 2002 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING, vol. 4, pages 2002
BING-YIN XIA ET AL: "A modified spectral subtraction method for speech enhancement based on masking property of human auditory system", WIRELESS COMMUNICATIONS&SIGNAL PROCESSING, 2009. WCSP 2009. INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 13 November 2009 (2009-11-13), pages 1 - 5, XP031594664, ISBN: 978-1-4244-4856-2 *
COHEN I ET AL: "Speech enhancement for non-stationary noise environments", SIGNAL PROCESSING, ELSEVIER SCIENCE PUBLISHERS B.V. AMSTERDAM, NL, vol. 81, no. 11, 1 November 2001 (2001-11-01), pages 2403 - 2418, XP004308517, ISSN: 0165-1684, DOI: 10.1016/S0165-1684(01)00128-1 *
COHEN; B. BERDUGO: "Noise Estimation by Minima Controlled Recursive Averaging for Robust Speech Enhancement", IEEE SIGNAL PROCESSING LETTERS, vol. 9, no. 1, January 2002 (2002-01-01), pages 12 - 15, XP011428379, DOI: doi:10.1109/97.988717
I. COHEN; B. BERDUGO: "Speech enhancement for non-stationary noise environments", SIGNAL PROCESSING, vol. 81, 2001, pages 2403 - 2418, XP004308517, DOI: doi:10.1016/S0165-1684(01)00128-1
KAMATH S ET AL: "Multi-band spectral subtraction method for enhancing speech corrupted by colored noise", 2002 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING. PROCEEDINGS (CAT. NO.02CH37334) IEEE PISCATAWAY, NJ, USA, vol. 4, 2002, XP002649825, ISBN: 0-7803-7402-9 *
LUO JUN ET AL: "Spectral Subtraction Speech Enhancement Technology Based on Fast Noise Estimation", INFORMATION ENGINEERING AND COMPUTER SCIENCE, 2009. ICIECS 2009. INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 19 December 2009 (2009-12-19), pages 1 - 3, XP031587156, ISBN: 978-1-4244-4994-1 *
X. BING-YIN ET AL.: "A Modified Spectral Subtraction Method For Speech Enhancement Based on Masking Property of Human Auditory System", WIRELESS COMMUNICATIONS & SIGNAIS PROCESSING, INTERNATIONAL CONFERENCE ON IEEE, 13 November 2009 (2009-11-13), pages 1 - 5, XP031594664

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111854920A (zh) * 2020-07-24 2020-10-30 贵州电网有限责任公司 一种基于dvs振动监测信号的预处理方法及系统
CN113367676A (zh) * 2021-06-07 2021-09-10 广东电网有限责任公司 一种用于可穿戴式设备的ppg运动噪声滤波方法及装置
CN116495946A (zh) * 2023-06-27 2023-07-28 珠江水利委员会珠江水利科学研究院 一种用于城市雨污水截污调蓄和清淤综合处理方法及系统
CN116495946B (zh) * 2023-06-27 2023-10-13 珠江水利委员会珠江水利科学研究院 一种用于城市雨污水截污调蓄和清淤综合处理方法及系统

Also Published As

Publication number Publication date
EP2515300B1 (de) 2014-05-14
FR2974443A1 (fr) 2012-10-26
FR2974443B1 (fr) 2013-04-26
PT2515300E (pt) 2014-08-27
ES2492698T3 (es) 2014-09-10

Similar Documents

Publication Publication Date Title
EP1356461B1 (de) Rauschverminderungsverfahren und -einrichtung
EP1789956B1 (de) Verfahren zum verarbeiten eines rauschbehafteten tonsignals und einrichtung zur implementierung des verfahrens
EP2293594B1 (de) Verfahren zur Filterung von seitlichem nichtstationärem Rauschen für ein Multimikrofon-Audiogerät
EP2002428B1 (de) Verfahren zur trainierten diskrimination und dämpfung von echos eines digitalsignals in einem decoder und entsprechende einrichtung
US8010355B2 (en) Low complexity noise reduction method
EP2309499B1 (de) Verfahren zur optimierten Filterung nicht stationärer Geräusche, die von einem Audiogerät mit mehreren Mikrophonen eingefangen werden, insbesondere eine Freisprechtelefonanlage für Kraftfahrzeuge
EP2772916B1 (de) Verfahren zur Geräuschdämpfung eines Audiosignals mit Hilfe eines Algorithmus mit variabler Spektralverstärkung mit dynamisch modulierbarer Härte
FR3012928A1 (fr) Modificateurs reposant sur un snr estime exterieurement pour des calculs internes de mmse
EP0752181A1 (de) Echokompensation mit adaptivem filter im frequenzbereich
EP2936488B1 (de) Wirksame dämpfung von vorechos in einem digitalen audiosignal
EP0998166A1 (de) Anordnung zur Verarbeitung von Audiosignalen, Empfänger und Verfahren zum Filtern und Wiedergabe eines Nutzsignals in Gegenwart von Umgebungsgeräusche
EP1016071B1 (de) Verfahren und vorrichtung zur sprachdetektion
FR3012929A1 (fr) Modificateur de la presence de probabilite de la parole perfectionnant les performances de suppression du bruit reposant sur le log-mmse
EP2515300B1 (de) Verfahren und System für die Geräuschunterdrückung
EP3192073B1 (de) Unterscheidung und dämpfung von vorechos in einem digitalen audiosignal
EP2262216A1 (de) Verfahren zur Erkennung einer Doppelsprech-Situation für Telefonfreisprechgerät
EP1039736B1 (de) Verfahren und Vorrichtung zur adaptiven Identifikation und entsprechender adaptiver Echokompensator
FR2906070A1 (fr) Reduction de bruit multi-reference pour des applications vocales en environnement automobile
EP1021805B1 (de) Verfahren und vorrichtung zur verbesserung eines digitalen sprachsignals
FR2768546A1 (fr) Procede de debruitage d'un signal de parole numerique
FR2906071A1 (fr) Reduction de bruit multibande avec une reference de bruit non acoustique
EP4287648A1 (de) Elektronische vorrichtung und verarbeitungsverfahren, akustische vorrichtung und computerprogramm dafür
BE1020218A3 (fr) Procede pour ameliorer la resolution temporelle des informations fournies par un filtre compose et dispositif correspondant.
WO1999027523A1 (fr) Procede de reconstruction, apres debruitage, de signaux sonores
Malshikare et al. The Speech Enhancement through Elimination of Impulsive Disturbance Using Log MMSE Filtering

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17P Request for examination filed

Effective date: 20130308

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602012001676

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G10L0021020000

Ipc: G10L0021020800

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 25/18 20130101ALN20131128BHEP

Ipc: G10L 21/0208 20130101AFI20131128BHEP

INTG Intention to grant announced

Effective date: 20131216

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 668811

Country of ref document: AT

Kind code of ref document: T

Effective date: 20140615

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012001676

Country of ref document: DE

Effective date: 20140703

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: CABINET GERMAIN ET MAUREAU, FR

REG Reference to a national code

Ref country code: PT

Ref legal event code: SC4A

Free format text: AVAILABILITY OF NATIONAL TRANSLATION

Effective date: 20140811

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20140514

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140815

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140914

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140814

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012001676

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20150217

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012001676

Country of ref document: DE

Effective date: 20150217

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20150411

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20150411

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20160315

Year of fee payment: 5

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: PT

Payment date: 20160317

Year of fee payment: 5

Ref country code: FR

Payment date: 20160310

Year of fee payment: 5

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20160504

Year of fee payment: 5

Ref country code: DE

Payment date: 20160316

Year of fee payment: 5

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20160412

Year of fee payment: 5

Ref country code: BE

Payment date: 20160421

Year of fee payment: 5

Ref country code: SE

Payment date: 20160415

Year of fee payment: 5

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20160411

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160411

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20120411

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602012001676

Country of ref document: DE

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20171229

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171103

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170502

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170430

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170430

Ref country code: SE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170412

Ref country code: PT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20171011

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20170430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170411

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170430

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 668811

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170411

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20180629

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170412

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170411

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140514