EP2499843B1 - Verfahren zum abmischen von mikrofonsignalen einer tonaufnahme mit mehreren mikrofonen - Google Patents

Verfahren zum abmischen von mikrofonsignalen einer tonaufnahme mit mehreren mikrofonen Download PDF

Info

Publication number
EP2499843B1
EP2499843B1 EP10779267.3A EP10779267A EP2499843B1 EP 2499843 B1 EP2499843 B1 EP 2499843B1 EP 10779267 A EP10779267 A EP 10779267A EP 2499843 B1 EP2499843 B1 EP 2499843B1
Authority
EP
European Patent Office
Prior art keywords
spectral values
signal
microphone
imag
real
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP10779267.3A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP2499843A1 (de
Inventor
Jens Groh
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institut fuer Rundfunktechnik GmbH
Original Assignee
Institut fuer Rundfunktechnik GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institut fuer Rundfunktechnik GmbH filed Critical Institut fuer Rundfunktechnik GmbH
Publication of EP2499843A1 publication Critical patent/EP2499843A1/de
Application granted granted Critical
Publication of EP2499843B1 publication Critical patent/EP2499843B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/008—Systems employing more than two channels, e.g. quadraphonic in which the audio signals are in digital form, i.e. employing more than two discrete digital channels
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04H—BROADCAST COMMUNICATION
    • H04H60/00—Arrangements for broadcast applications with a direct linking to broadcast information or broadcast space-time; Broadcast-related systems
    • H04H60/02—Arrangements for generating broadcast information; Arrangements for generating broadcast-related information with a direct linking to broadcast information or to broadcast space-time; Arrangements for simultaneous generation of broadcast information and broadcast-related information
    • H04H60/04—Studio equipment; Interconnection of studios

Definitions

  • the invention relates to a method for mixing microphone signals of a sound exception with multiple microphones according to the preamble of claim 1. Such a method is known from WO 2004/084 185 A1 known.
  • An extended acoustic scene may be, for example, a concert hall with an orchestra of a variety of musical instruments.
  • each individual musical instrument is recorded with a single, closely positioned microphone and also positions further microphones at a greater distance to capture the overall acoustic image, including the reverberation in the concert hall and the audience noises (in particular applause).
  • Another example of an extended acoustic scene is a drum kit consisting of several percussion instruments recorded in the recording studio.
  • a microphone is positioned in close proximity in front of the individual percussion instruments and an additional microphone is mounted above the percussionist.
  • Such multimicrophone sound recordings make it possible to capture as many acoustic and sound properties of the details as possible as well as of the overall picture of the scenery in high quality and make them aesthetically satisfactorily shapable.
  • Each microphone signal of the plurality of microphones is usually recorded as multi-track recording. In the subsequent mixing of the microphone signals further creative work is done. In special cases it is also possible to mix "live" immediately and record only the result of the mixdown.
  • the design goals of the mix are usually a balanced ratio of the volumes of all sound sources, a natural sound and a realistic spatial impression of the overall acoustic image.
  • a method for mixing microphone signals of a multi-microphone sound recording it is known to form the spectral values of overlapping time windows of samples from a first microphone signal and a second microphone signal.
  • the spectral values of the first microphone signal are distributed to the spectral values of the second microphone signal in a first summation stage to form spectral values of a first sum signal, thereby dynamically correcting the spectral values of one of the two microphone signals.
  • spectral values of a result signal are formed, which are subjected to inverse Fourier transformation and block merging. For each block of samples, individual correction factors can be determined in this way.
  • the object of the invention is to largely compensate for the sound changes resulting from the mixing of multi-microphone sound recordings as a result of multipath propagation of sound components.
  • FIG. 3 shows a general block diagram of an arrangement for carrying out the method according to the invention.
  • a first microphone signal 100 and a second microphone signal 101 are each supplied to an associated blocking and spectral transformation unit 320.
  • the supplied microphone signals 100 and 101 are first divided into blocks of time-overlapping signal portions, whereupon the formed blocks undergo a Fourier transform. This results in the spectral values 300 of the first microphone signal 100 and the spectral values 301 of the second microphone signal 101 at the outputs of the blocks 320.
  • the spectral values 300 and 301 are then fed to a first summation stage 310, which generates the spectral values 311 of a first sum signal from the spectral values 300 and 301.
  • the spectral values 311 also form the spectral values 399 of a result signal, which are first subjected to an inverse Fourier transformation in a unit 330. The inverse spectral values thus formed are then combined to form blocks. The resulting blocks of time-overlapping signal portions are accumulated into the result signal 199.
  • the mentioned series connection of assemblies 700 should be understood as meaning that at the beginning of the series connection the spectral values 400 simultaneously form the spectral values of the first sum signal 311 and at the end of the series connection the spectral values 411 also form the spectral values of the result signal 399. In all other sections of the series connection, the spectral values 411 of a summation level 410 also form the spectral values 400 of the subsequent summation level 410.
  • Each block formation and spectral transformation unit 320 of an assembly 700 of FIG A n + 2 -th microphone signal 201 is fed in series, in which it is divided into blocks of time-overlapping signal sections.
  • the formed blocks of time-overlapping signal sections are Fourier-transformed, resulting in the spectral values 401 of the n + 2-th microphone signal.
  • the spectral values 400 of the n-th sum signal and the spectral values 401 of the n + 2-th microphone signal are then supplied to the n + 1-th summation stage 410, which generates from them the spectral values 411 of the n + 1-th sum signal.
  • the degree L is chosen so that experience has shown that no background noises are perceived. Typically, the degree L is on the order of 0.5. The greater the degree L, the lower the probability of the disturbances, but this also partially reduces the compensation of sound changes determined by the setting of D.
  • the spectral values A (k) of the signal 501 to be prioritized are additionally supplied to a multiplier 520, while the spectral values B (k) of the signal 502 which is not to be prioritized are additionally supplied to an adder 530.
  • the multiplier 520 receives the correction factor values m (k) of the output signal 511 from the calculation unit 510, where it is complex with the spectral values A (k) 501 (after real part and imaginary part). be multiplied.
  • the result values of the multiplier 520 are supplied to the adder 530 where they are added complexly (after real part and imaginary part) with the spectral values B (k) of the non-prioritizing signal 502. This results in the spectral values 311 of the first summation signal of the first summation stage 310.
  • the decisive factor for the prioritization is thus the multiplication of the correction factor m (k) with exactly one of the two summands of the addition performed in the adder 530.
  • the entire signal path of this summand is "prioritized" from the microphone signal input to the adder 530.
  • FIG. 6 represents the details of the n + 1-th summation stage 410.
  • the n + 1-th summation stage 410 is similar in construction to the first summation stage 310, but with the difference that here the allocation unit 500 the spectral values 400 of the n-th sum signal and the Spectral values 401 of the n + 2 nd microphone signal, and that the result values of the adder 530 form the spectral values 411 of the n + 1 th sum signal.
EP10779267.3A 2009-11-12 2010-11-02 Verfahren zum abmischen von mikrofonsignalen einer tonaufnahme mit mehreren mikrofonen Active EP2499843B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE200910052992 DE102009052992B3 (de) 2009-11-12 2009-11-12 Verfahren zum Abmischen von Mikrofonsignalen einer Tonaufnahme mit mehreren Mikrofonen
PCT/EP2010/066657 WO2011057922A1 (de) 2009-11-12 2010-11-02 Verfahren zum abmischen von mikrofonsignalen einer tonaufnahme mit mehreren mikrofonen

Publications (2)

Publication Number Publication Date
EP2499843A1 EP2499843A1 (de) 2012-09-19
EP2499843B1 true EP2499843B1 (de) 2016-07-13

Family

ID=43571276

Family Applications (1)

Application Number Title Priority Date Filing Date
EP10779267.3A Active EP2499843B1 (de) 2009-11-12 2010-11-02 Verfahren zum abmischen von mikrofonsignalen einer tonaufnahme mit mehreren mikrofonen

Country Status (8)

Country Link
US (1) US9049531B2 (zh)
EP (1) EP2499843B1 (zh)
JP (1) JP5812440B2 (zh)
KR (1) KR101759976B1 (zh)
CN (1) CN102687535B (zh)
DE (1) DE102009052992B3 (zh)
TW (1) TWI492640B (zh)
WO (1) WO2011057922A1 (zh)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ITTO20110890A1 (it) * 2011-10-05 2013-04-06 Inst Rundfunktechnik Gmbh Interpolationsschaltung zum interpolieren eines ersten und zweiten mikrofonsignals.
ITTO20120067A1 (it) 2012-01-26 2013-07-27 Inst Rundfunktechnik Gmbh Method and apparatus for conversion of a multi-channel audio signal into a two-channel audio signal.
ITTO20120274A1 (it) * 2012-03-27 2013-09-28 Inst Rundfunktechnik Gmbh Dispositivo per il missaggio di almeno due segnali audio.
ITTO20130028A1 (it) 2013-01-11 2014-07-12 Inst Rundfunktechnik Gmbh Mikrofonanordnung mit verbesserter richtcharakteristik
WO2015173422A1 (de) 2014-05-15 2015-11-19 Stormingswiss Sàrl Verfahren und vorrichtung zur residualfreien erzeugung eines upmix aus einem downmix
IT201700040732A1 (it) * 2017-04-12 2018-10-12 Inst Rundfunktechnik Gmbh Verfahren und vorrichtung zum mischen von n informationssignalen
EP3963902A4 (en) 2019-09-24 2022-07-13 Samsung Electronics Co., Ltd. METHODS AND SYSTEMS FOR MIXED AUDIO SIGNAL RECORDING AND DIRECTIONAL AUDIO CONTENT REPRODUCTION
CN114449434B (zh) * 2022-04-07 2022-08-16 北京荣耀终端有限公司 麦克风校准方法及电子设备

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5228093A (en) * 1991-10-24 1993-07-13 Agnello Anthony M Method for mixing source audio signals and an audio signal mixing system
US6154552A (en) * 1997-05-15 2000-11-28 Planning Systems Inc. Hybrid adaptive beamformer
JP4163294B2 (ja) * 1998-07-31 2008-10-08 株式会社東芝 雑音抑圧処理装置および雑音抑圧処理方法
DE69939272D1 (de) 1998-11-16 2008-09-18 Univ Illinois Binaural-signal-verarbeitungstechniken
EP1081985A3 (en) * 1999-09-01 2006-03-22 Northrop Grumman Corporation Microphone array processing system for noisy multipath environments
US6668062B1 (en) * 2000-05-09 2003-12-23 Gn Resound As FFT-based technique for adaptive directionality of dual microphones
AU2001294960A1 (en) * 2000-09-29 2002-04-08 Knowles Electronics, Llc. Second order microphone array
GB2375698A (en) * 2001-02-07 2002-11-20 Canon Kk Audio signal processing apparatus
US7315623B2 (en) * 2001-12-04 2008-01-01 Harman Becker Automotive Systems Gmbh Method for supressing surrounding noise in a hands-free device and hands-free device
JP4286637B2 (ja) * 2002-11-18 2009-07-01 パナソニック株式会社 マイクロホン装置および再生装置
ATE487213T1 (de) * 2003-03-17 2010-11-15 Koninkl Philips Electronics Nv Verarbeitung von mehrkanalsignalen
DE102004005998B3 (de) * 2004-02-06 2005-05-25 Ruwisch, Dietmar, Dr. Verfahren und Vorrichtung zur Separierung von Schallsignalen
ATE390683T1 (de) 2004-03-01 2008-04-15 Dolby Lab Licensing Corp Mehrkanalige audiocodierung
US8275147B2 (en) * 2004-05-05 2012-09-25 Deka Products Limited Partnership Selective shaping of communication signals
US20060147063A1 (en) * 2004-12-22 2006-07-06 Broadcom Corporation Echo cancellation in telephones with multiple microphones
JP4896449B2 (ja) * 2005-06-29 2012-03-14 株式会社東芝 音響信号処理方法、装置及びプログラム
DE102006027673A1 (de) 2006-06-14 2007-12-20 Friedrich-Alexander-Universität Erlangen-Nürnberg Signaltrenner, Verfahren zum Bestimmen von Ausgangssignalen basierend auf Mikrophonsignalen und Computerprogramm
JP4455614B2 (ja) * 2007-06-13 2010-04-21 株式会社東芝 音響信号処理方法及び装置
JP2009069181A (ja) * 2007-09-10 2009-04-02 Sharp Corp 音場補正装置
KR101434200B1 (ko) * 2007-10-01 2014-08-26 삼성전자주식회사 혼합 사운드로부터의 음원 판별 방법 및 장치
DE102008056704B4 (de) 2008-11-11 2010-11-04 Institut für Rundfunktechnik GmbH Verfahren zum Erzeugen eines abwärtskompatiblen Tonformates

Also Published As

Publication number Publication date
TWI492640B (zh) 2015-07-11
DE102009052992B3 (de) 2011-03-17
TW201129115A (en) 2011-08-16
CN102687535B (zh) 2015-09-23
WO2011057922A1 (de) 2011-05-19
EP2499843A1 (de) 2012-09-19
KR101759976B1 (ko) 2017-07-20
US9049531B2 (en) 2015-06-02
CN102687535A (zh) 2012-09-19
JP2013511178A (ja) 2013-03-28
KR20120095971A (ko) 2012-08-29
JP5812440B2 (ja) 2015-11-11
US20120237055A1 (en) 2012-09-20

Similar Documents

Publication Publication Date Title
EP2499843B1 (de) Verfahren zum abmischen von mikrofonsignalen einer tonaufnahme mit mehreren mikrofonen
EP2206113B1 (de) Vorrichtung und verfahren zum erzeugen eines multikanalsignals mit einer sprachsignalverarbeitung
DE69827775T2 (de) Tonkanalsmischung
EP1844627B1 (de) Vorrichtung und verfahren zum simulieren eines wellenfeldsynthese-systemes
DE4134130C2 (de) Vorrichtung zum Aufweiten und Ausbalancieren von Schallfeldern
EP2402942B1 (de) Vorrichtung und Verfahren zum Erzeugen eines Umgebungssignals
DE69827911T2 (de) Verfahren und einrichtung zur mehrkanaligen kompensation eines akustischen echos
DE102018127071B3 (de) Audiosignalverarbeitung mit akustischer Echounterdrückung
DE102010030534A1 (de) Vorrichtung zum Veränderung einer Audio-Szene und Vorrichtung zum Erzeugen einer Richtungsfunktion
WO2008125322A1 (de) Erzeugung dekorrelierter signale
DE19715498A1 (de) Stereobildverbesserungsvorrichtung und -verfahren unter Verwendung von Tabellen
DE69934069T2 (de) Schalleffekt Addiergerät
EP0808076B1 (de) Raumklangsystem
DE10148351B4 (de) Verfahren und Vorrichtung zur Auswahl eines Klangalgorithmus
DE102012025016B3 (de) Verfahren zur Ermittlung wenigstens zweier Einzelsignale aus wenigstens zwei Ausgangssignalen
DE102019102414B4 (de) Verfahren und System zur Detektion von Reibelauten in Sprachsignalen
DE19900961A1 (de) Verfahren und Vorrichtung zur Wiedergabe von Mehrkanaltonsignalen
EP1518441B1 (de) Vorrichtung und verfahren zum unterdrücken einer rückkopplung
DE10125229A1 (de) Verfahren und Vorrichtung zur Wiedergabe von Mehrkanaltonsignalen
DE102010015630B3 (de) Verfahren zum Erzeugen eines abwärtskompatiblen Tonformates
DE102006059764B4 (de) Signalverarbeitung
DE102019124285A1 (de) Dekorrelation von eingangssignalen
DE102020108958A1 (de) Verfahren zum Darbieten eines ersten Audiosignals während der Darbietung eines zweiten Audiosignals
EP1900250B1 (de) Elektroakustisches verfahren
DE102020100131A1 (de) Verfahren zur Verarbeitung eines Mehrkanal-Audiosignals

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20120607

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20130308

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20160216

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 813129

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160715

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502010012024

Country of ref document: DE

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: KATZAROV S.A., CH

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20160713

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161013

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161114

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161130

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161014

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502010012024

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161013

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

26N No opposition filed

Effective date: 20170418

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161130

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161130

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161130

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 8

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161102

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 813129

Country of ref document: AT

Kind code of ref document: T

Effective date: 20161102

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161102

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20161130

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 502010012024

Country of ref document: DE

Representative=s name: KOPLIN, MORITZ, DR., DE

Ref country code: DE

Ref legal event code: R082

Ref document number: 502010012024

Country of ref document: DE

Representative=s name: KOPLIN PATENTANWALTSGESELLSCHAFT MBH, DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20101102

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 502010012024

Country of ref document: DE

Representative=s name: KOPLIN PATENTANWALTSGESELLSCHAFT MBH, DE

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 502010012024

Country of ref document: DE

Representative=s name: KOPLIN PATENTANWALTSGESELLSCHAFT MBH, DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160713

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20221123

Year of fee payment: 13

Ref country code: FR

Payment date: 20221118

Year of fee payment: 13

Ref country code: DE

Payment date: 20221121

Year of fee payment: 13