EP2815589B1 - Procédé de synthèse transaurale pour la spatialisation sonore - Google Patents

Procédé de synthèse transaurale pour la spatialisation sonore Download PDF

Info

Publication number
EP2815589B1
EP2815589B1 EP13710449.3A EP13710449A EP2815589B1 EP 2815589 B1 EP2815589 B1 EP 2815589B1 EP 13710449 A EP13710449 A EP 13710449A EP 2815589 B1 EP2815589 B1 EP 2815589B1
Authority
EP
European Patent Office
Prior art keywords
signal
stereo
spatialized
producing
signals
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP13710449.3A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP2815589A1 (fr
Inventor
Franck Rosset
Jean-Luc HAURAIS
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Publication of EP2815589A1 publication Critical patent/EP2815589A1/fr
Application granted granted Critical
Publication of EP2815589B1 publication Critical patent/EP2815589B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00—Stereophonic arrangements
    • H04R5/033—Headphones for stereophonic communication
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/02—Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/01—Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]

Definitions

  • the present invention relates to the field of sound spatialization, so-called spatialised, of audio signals, integrating in particular a room effect, particularly in the field of transaural techniques.
  • binaural refers to the reproduction on a stereophonic headphones, or a pair of headphones or a pair of speakers, a sound signal with nevertheless spatialization effects.
  • the invention is however not limited to the aforementioned technique and applies, in particular, to techniques derived from “binaural” such as “transaural” (commercial name) restitution techniques, that is to say on remote loudspeakers, installed for example in a concert hall or cinema with a multi-point sound system.
  • a specific application of the invention is, for example, the enrichment of audio content broadcast by a pair of speakers to immerse a listener in a spatialized sound scene, including in particular a room effect or outdoor space.
  • the state of the art defines a transfer function, or filter, of a sound signal between a position of a sound source in the space and the two ears of a listener.
  • the acoustic transfer function of the aforementioned head is designated HRTF for "Head Related Transfer Function” in English in its form.
  • frequency and HRIR for "Head Related Impulse Response” in English in its temporal form.
  • the binaural technique consists in applying such acoustic transfer functions of the head to monophonic audio signals, in order to obtain a stereophonic signal which makes it possible, when listening to headphones, to have the feeling that the sources sounds come from a particular direction of space.
  • the signal from the right ear is obtained by filtering the monophonic signal by the HRTF of the right ear and the left ear signal is obtained by filtering the same monophonic signal by the HRTF of the left ear.
  • the method according to the invention is defined in claim 1 and comprises a first processing (1) of producing a database of pulse signals from the acquisition of acoustic signals in a plurality of physical spaces, by the recording signals produced by loudspeakers in response to a reference multifrequency signal.
  • This stereo signal can then be broadcast by a pair of standard loudspeakers, to restore a spatialized sound environment corresponding to the space that has been used to produce the impulse response signals or a combination of such spaces.
  • This step is replicated a plurality of times. It is illustrated by the figure 2 .
  • Each of the speakers (5 to 11) is then successively applied to an original multifrequency signal using the amplifier (14).
  • This original signal is for example a sequence of a duration of between 10 and 90 seconds, with a frequency variation in the sound spectrum.
  • This signal is for example a linear variation between 20 Hz and 20 kHz, or any signal covering the entire spectrum of the enclosure.
  • the sound signal produced by the active speaker is picked up by the microphone torque (12, 13) and produces a recorded stereo signal. From this signal is carried out at 96 Khz sampling in a known manner and at a fast Fourier transform deconvolution between the original signal and the recorded signal, to construct an impulse response for the chamber considered in space considered physical.
  • This step is repeated for each of the speakers (5 to 11) of the series, then for different physical spaces where a series of identical or different speakers is re-inserted with an identical or different amplifier and identical microphones.
  • This first step leads to the construction of a database of stereo impulse responses.
  • This step makes it possible to construct a spatial stereo audio signal from a multichannel signal N.i corresponding to a traditional digital recording.
  • This step consists in selecting in the database formed during the initial step N + i impulse responses.
  • the selection will consist in associating with each of the N + i signals one of the impulse responses of said database, by ensuring that the acquisition position in the space of the impulse response corresponds to the position in the space of the channel with which it is associated.
  • N + 1 pairs of spatialized signals S j sG and S j sD are thus produced , with J being between 1 and N + 1.
  • channel equalization is performed to improve the dynamics of the signals.
  • the final step is to recombine the signals to construct a spatialized right and left signal pair.
  • the signals S j sG corresponding to the space on the left are added to construct the left channel of the spatialized stereo signal.
  • the same procedure is followed for the signals S j sD corresponding to the space on the right, to construct the right channel of the spatialized stereo signal.
  • the channels are equalized to improve the dynamics of the two channels.
  • the signal to be spatialized is not of the Ni type but simply a stereo signal
  • a step is taken intermediate consisting of building a signal Ni by phase extraction treatments between the left and right track, to build different new signals.
  • This extraction by phase consists in producing a signal corresponding to a reconstructed central channel, by a processing consisting in adding the signal of the left channel with a signal of the straight line which is out of phase, for example in phase opposition.
  • the left and right tracks are phase-shifted, with different phase-shift angles, and the out-of-phase signal pairs are added, with empirically determined weights in order to restore a spacialized soundscape.
  • frequency filters are applied to right and left signals when creating "reconstructed" channels, in order to increase signal dynamics and maintain high fidelity sound quality.
  • the figure 3 represents a schematic view of the rendering installation, from a pair of actual speakers (17, 18).
  • This pair of speakers (17, 18) receives a signal for simulating calculated speakers (20 to 27 and 30 to 37).
  • the effective number of calculated speakers (20 to 27) corresponds to the number of physical speakers (5 to 11; 17) used for the production of the pulse signal database, or to the number of virtual speakers reconstructed by the method referred to above.
  • Virtual speakers (30 to 37) are also created which produce a perception in the sound space of a combination of real neighboring speakers, to fill the sound holes.
  • These virtual speakers are created by a modification of the signal supplying the neighboring real speakers.
  • This stereo signal is then applied to a conventional audio equipment, connected to a pair of speakers (18, 19), which will reproduce a spatialized sound environment corresponding to the sound environment of the installation that was used to build the base impulse signals, or a virtual sound environment corresponding to the combination of several original atmospheres, where appropriate enriched with virtual atmospheres.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Algebra (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Physics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Stereophonic System (AREA)
EP13710449.3A 2012-02-13 2013-02-11 Procédé de synthèse transaurale pour la spatialisation sonore Active EP2815589B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR1251328A FR2986932B1 (fr) 2012-02-13 2012-02-13 Procede de synthese transaurale pour la spatialisation sonore
PCT/FR2013/050278 WO2013121136A1 (fr) 2012-02-13 2013-02-11 Procédé de synthèse transaurale pour la spatialisation sonore

Publications (2)

Publication Number Publication Date
EP2815589A1 EP2815589A1 (fr) 2014-12-24
EP2815589B1 true EP2815589B1 (fr) 2017-04-05

Family

ID=47901163

Family Applications (1)

Application Number Title Priority Date Filing Date
EP13710449.3A Active EP2815589B1 (fr) 2012-02-13 2013-02-11 Procédé de synthèse transaurale pour la spatialisation sonore

Country Status (10)

Country Link
EP (1) EP2815589B1 (ru)
JP (1) JP6421385B2 (ru)
KR (1) KR20140128412A (ru)
CN (1) CN104160722B (ru)
BR (1) BR112014019926A2 (ru)
FR (1) FR2986932B1 (ru)
HK (1) HK1204188A1 (ru)
IN (1) IN2014DN06776A (ru)
RU (1) RU2639955C2 (ru)
WO (1) WO2013121136A1 (ru)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR3065137B1 (fr) 2017-04-07 2020-02-28 Axd Technologies, Llc Procede de spatialisation sonore

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020133327A1 (en) * 1998-03-31 2002-09-19 Mcgrath David Stanley Acoustic response simulation system

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030035553A1 (en) * 2001-08-10 2003-02-20 Frank Baumgarte Backwards-compatible perceptual coding of spatial cues
JP4062959B2 (ja) * 2002-04-26 2008-03-19 ヤマハ株式会社 残響付与装置、残響付与方法、インパルス応答生成装置、インパルス応答生成方法、残響付与プログラム、インパルス応答生成プログラムおよび記録媒体
US6937737B2 (en) * 2003-10-27 2005-08-30 Britannia Investment Corporation Multi-channel audio surround sound from front located loudspeakers
KR20050060789A (ko) * 2003-12-17 2005-06-22 삼성전자주식회사 가상 음향 재생 방법 및 그 장치
JP2005252332A (ja) * 2004-03-01 2005-09-15 Clarion Co Ltd 音場再生装置及びその制御方法
US8175286B2 (en) * 2005-05-26 2012-05-08 Bang & Olufsen A/S Recording, synthesis and reproduction of sound fields in an enclosure
JP2006339694A (ja) * 2005-05-31 2006-12-14 D & M Holdings Inc オーディオ信号出力装置
US7970626B2 (en) * 2005-07-08 2011-06-28 Oltine Acquistitions NY LLC Facilitating payments to health care providers
KR100619082B1 (ko) * 2005-07-20 2006-09-05 삼성전자주식회사 와이드 모노 사운드 재생 방법 및 시스템
TWI396188B (zh) * 2005-08-02 2013-05-11 Dolby Lab Licensing Corp 依聆聽事件之函數控制空間音訊編碼參數的技術
MX2008011994A (es) * 2006-03-24 2008-11-27 Dolby Sweden Ab Generacion de mezclas descendentes espaciales a partir de representaciones parametricas de señales de multicanal.
ATE532350T1 (de) * 2006-03-24 2011-11-15 Dolby Sweden Ab Erzeugung räumlicher heruntermischungen aus parametrischen darstellungen mehrkanaliger signale
JP2008301427A (ja) * 2007-06-04 2008-12-11 Onkyo Corp マルチチャンネル音声再生装置
RU2437247C1 (ru) * 2008-01-01 2011-12-20 ЭлДжи ЭЛЕКТРОНИКС ИНК. Способ и устройство для обработки звукового сигнала
RU2469497C2 (ru) * 2008-02-14 2012-12-10 Долби Лэборетериз Лайсенсинг Корпорейшн Стереофоническое расширение
UA101542C2 (ru) * 2008-12-15 2013-04-10 Долби Лабораторис Лайсензин Корпорейшн Виртуализатор окружающего звука с динамическим сжатием диапазона и способ
KR101764175B1 (ko) * 2010-05-04 2017-08-14 삼성전자주식회사 입체 음향 재생 방법 및 장치

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020133327A1 (en) * 1998-03-31 2002-09-19 Mcgrath David Stanley Acoustic response simulation system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
JEUB M ET AL: "A binaural room impulse response database for the evaluation of dereverberation algorithms", DIGITAL SIGNAL PROCESSING, 2009 16TH INTERNATIONAL CONFERENCE ON, IEEE, PISCATAWAY, NJ, USA, 5 July 2009 (2009-07-05), pages 1 - 5, XP031510342, ISBN: 978-1-4244-3297-4 *

Also Published As

Publication number Publication date
IN2014DN06776A (ru) 2015-05-22
EP2815589A1 (fr) 2014-12-24
WO2013121136A1 (fr) 2013-08-22
BR112014019926A2 (pt) 2017-07-04
CN104160722B (zh) 2018-01-12
JP6421385B2 (ja) 2018-11-14
RU2014133066A (ru) 2016-04-10
CN104160722A (zh) 2014-11-19
JP2015510348A (ja) 2015-04-02
FR2986932B1 (fr) 2014-03-07
KR20140128412A (ko) 2014-11-05
RU2639955C2 (ru) 2017-12-25
HK1204188A1 (en) 2015-11-06
FR2986932A1 (fr) 2013-08-16

Similar Documents

Publication Publication Date Title
KR100626233B1 (ko) 스테레오 확장 네트워크에서의 출력의 등화
US7231054B1 (en) Method and apparatus for three-dimensional audio display
WO2007110520A1 (fr) Procede de synthese binaurale prenant en compte un effet de salle
US20150131824A1 (en) Method for high quality efficient 3d sound reproduction
Rafaely et al. Spatial audio signal processing for binaural reproduction of recorded acoustic scenes–review and challenges
KR20080060640A (ko) 개인 청각 특성을 고려한 2채널 입체 음향 재생 방법 및장치
KR20110053600A (ko) 다채널 사운드 신호 생성 장치
US20190394596A1 (en) Transaural synthesis method for sound spatialization
JP6361000B2 (ja) 改良された復元のために音声信号を処理するための方法
EP2009891B1 (fr) Transmission de signal audio dans un système de conférence audio immersive
WO2000019415A2 (en) Method and apparatus for three-dimensional audio display
JP2022551873A (ja) スペクトル直交オーディオ成分を用いたサブバンド空間処理およびクロストーク処理
GB2605970A (en) Content based spatial remixing
US20200059750A1 (en) Sound spatialization method
EP2815589B1 (fr) Procédé de synthèse transaurale pour la spatialisation sonore
US20150036827A1 (en) Transaural Synthesis Method for Sound Spatialization
KR100275779B1 (ko) 5채널 오디오 데이터를 2채널로 변환하여 헤드폰으로 재생하는 장치 및 방법
US9609454B2 (en) Method for playing back the sound of a digital audio signal
Gamper et al. Spatialisation in audio augmented reality using finger snaps
Christensen et al. Stereo upmix design for shaping sound experiences
KR20050069859A (ko) 구체를 이용한 3차원 오디오 신호 처리 시스템 및 그 방법

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20140811

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAX Request for extension of the european patent (deleted)
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1204188

Country of ref document: HK

17Q First examination report despatched

Effective date: 20151125

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

INTG Intention to grant announced

Effective date: 20160823

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTC Intention to grant announced (deleted)
INTG Intention to grant announced

Effective date: 20161018

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 882820

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170415

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602013019417

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: ABREMA AGENCE BREVET ET MARQUES, GANGUILLET, CH

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 882820

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170706

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170705

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170805

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602013019417

Country of ref document: DE

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

26N No opposition filed

Effective date: 20180108

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602013019417

Country of ref document: DE

Representative=s name: KOENIG SZYNKA TILMANN VON RENESSE PATENTANWAEL, DE

Ref country code: DE

Ref legal event code: R081

Ref document number: 602013019417

Country of ref document: DE

Owner name: A3D TECHNOLOGIES LLC, LOS ANGELES, US

Free format text: FORMER OWNERS: HAURAIS, JEAN-LUC, PARIS, FR; ROSSET, FRANCK, BRUXELLES, BE

REG Reference to a national code

Ref country code: HK

Ref legal event code: GR

Ref document number: 1204188

Country of ref document: HK

REG Reference to a national code

Ref country code: FR

Ref legal event code: TP

Owner name: A3D TECHNOLOGIES LLC, US

Effective date: 20180412

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

REG Reference to a national code

Ref country code: GB

Ref legal event code: 732E

Free format text: REGISTERED BETWEEN 20180709 AND 20180711

REG Reference to a national code

Ref country code: NL

Ref legal event code: PD

Owner name: A3D TECHNOLOGIES LLC; US

Free format text: DETAILS ASSIGNMENT: CHANGE OF OWNER(S), ASSIGNMENT; FORMER OWNER NAME: ROSSET, FRANCK

Effective date: 20180703

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20180228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180211

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180211

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20130211

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170405

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170405

REG Reference to a national code

Ref country code: CH

Ref legal event code: PFUS

Owner name: A3D TECHNOLOGIES LLC, US

Free format text: FORMER OWNER: A3D TECHNOLOGIES LLC, US

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20240123

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240123

Year of fee payment: 12

Ref country code: CH

Payment date: 20240301

Year of fee payment: 12

Ref country code: GB

Payment date: 20240123

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20240123

Year of fee payment: 12

Ref country code: FR

Payment date: 20240123

Year of fee payment: 12