EP3308759A4 - Portable system that allows blind or visually impaired persons to interpret the surrounding environment by sound or touch - Google Patents

Portable system that allows blind or visually impaired persons to interpret the surrounding environment by sound or touch Download PDF

Info

Publication number
EP3308759A4
EP3308759A4 EP16806940.9A EP16806940A EP3308759A4 EP 3308759 A4 EP3308759 A4 EP 3308759A4 EP 16806940 A EP16806940 A EP 16806940A EP 3308759 A4 EP3308759 A4 EP 3308759A4
Authority
EP
European Patent Office
Prior art keywords
interpret
sound
touch
surrounding environment
visually impaired
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP16806940.9A
Other languages
German (de)
French (fr)
Other versions
EP3308759A1 (en
EP3308759B1 (en
Inventor
Antonio QUESADA HERVÁS
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Eyesynth S L
Eyesynth SL
Original Assignee
Eyesynth S L
Eyesynth SL
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Eyesynth S L, Eyesynth SL filed Critical Eyesynth S L
Publication of EP3308759A1 publication Critical patent/EP3308759A1/en
Publication of EP3308759A4 publication Critical patent/EP3308759A4/en
Application granted granted Critical
Publication of EP3308759B1 publication Critical patent/EP3308759B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61F—FILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F9/00—Methods or devices for treatment of the eyes; Devices for putting-in contact lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
    • A61F9/08—Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H3/00—Appliances for aiding patients or disabled persons to walk about
    • A61H3/06—Walking aids for blind persons
    • A61H3/061—Walking aids for blind persons with electronic detecting or guiding means
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/22—Measuring arrangements characterised by the use of optical techniques for measuring depth
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/593—Depth or shape recovery from multiple images from stereo images
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02—Alarms for ensuring the safety of persons
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B3/00—Audible signalling systems; Audible personal calling systems
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B6/00—Tactile signalling systems, e.g. personal calling systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H3/00—Appliances for aiding patients or disabled persons to walk about
    • A61H3/06—Walking aids for blind persons
    • A61H3/061—Walking aids for blind persons with electronic detecting or guiding means
    • A61H2003/063—Walking aids for blind persons with electronic detecting or guiding means with tactile perception
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G06T2207/10012—Stereo images

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Ophthalmology & Optometry (AREA)
  • Vascular Medicine (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biomedical Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Pain & Pain Management (AREA)
  • Epidemiology (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Rehabilitation Therapy (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Image Processing (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)
  • Traffic Control Systems (AREA)
  • Rehabilitation Tools (AREA)
  • Studio Devices (AREA)
  • Details Of Audible-Bandwidth Transducers (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Stereophonic System (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
EP16806940.9A 2015-06-12 2016-06-10 Portable system that allows blind or visually impaired persons to interpret the surrounding environment by sound or touch Active EP3308759B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
ES201530825A ES2597155B1 (en) 2015-06-12 2015-06-12 PORTABLE SOUND OR TOUCH INTERPRETATION SYSTEM OF THE ENVIRONMENT FOR AN INVIDENT
PCT/ES2016/070441 WO2016198721A1 (en) 2015-06-12 2016-06-10 Portable system that allows blind or visually impaired persons to interpret the surrounding environment by sound or touch

Publications (3)

Publication Number Publication Date
EP3308759A1 EP3308759A1 (en) 2018-04-18
EP3308759A4 true EP3308759A4 (en) 2019-02-27
EP3308759B1 EP3308759B1 (en) 2019-11-27

Family

ID=57504615

Family Applications (1)

Application Number Title Priority Date Filing Date
EP16806940.9A Active EP3308759B1 (en) 2015-06-12 2016-06-10 Portable system that allows blind or visually impaired persons to interpret the surrounding environment by sound or touch

Country Status (18)

Country Link
US (1) US11185445B2 (en)
EP (1) EP3308759B1 (en)
JP (1) JP6771548B2 (en)
KR (1) KR102615844B1 (en)
CN (1) CN107708624B (en)
AR (1) AR104959A1 (en)
AU (1) AU2016275789B2 (en)
BR (1) BR112017026545B1 (en)
CA (1) CA2986652A1 (en)
CO (1) CO2017012744A2 (en)
DK (1) DK3308759T3 (en)
ES (2) ES2597155B1 (en)
HK (1) HK1248093A1 (en)
IL (1) IL255624B (en)
MX (1) MX2017015146A (en)
PT (1) PT3308759T (en)
RU (1) RU2719025C2 (en)
WO (1) WO2016198721A1 (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11298266B2 (en) * 2016-12-07 2022-04-12 Second Sight Medical Products, Inc. Depth filter for visual prostheses
CN107320296A (en) * 2017-06-23 2017-11-07 重庆锦上医疗器械有限公司 The space three-dimensional acoustic expression system and method for visual signal
US10299982B2 (en) * 2017-07-21 2019-05-28 David M Frankel Systems and methods for blind and visually impaired person environment navigation assistance
CN108245385B (en) * 2018-01-16 2019-10-29 曹醒龙 A kind of device helping visually impaired people's trip
JP7415139B2 (en) * 2019-01-11 2024-01-17 ダイキン工業株式会社 Environmental condition storage device, environmental condition storage method, operating device, and environment recognition device
JP7549588B2 (en) * 2019-02-12 2024-09-11 キャン － ユー － シー リミテッド 3D sound equipment for the blind and visually impaired
WO2021219076A1 (en) 2020-04-30 2021-11-04 深圳市韶音科技有限公司 Bone conduction earphone
CN111862932B (en) * 2020-07-02 2022-07-19 北京科技大学 Wearable blind assisting system and method for converting image into sound
KR20220008659A (en) 2020-07-14 2022-01-21 김재현 Necklace for blinded
CN111862024A (en) * 2020-07-14 2020-10-30 四川深瑞视科技有限公司 Workpiece detection system and method based on depth information enhanced projection

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3172075A (en) * 1959-11-27 1965-03-02 Nat Res Dev Apparatus for furnishing information as to positioning of objects
US4292678A (en) * 1978-08-14 1981-09-29 Leslie Kay Method of and apparatus for echo location of objects
EP0410045A1 (en) * 1989-07-27 1991-01-30 Koninklijke Philips Electronics N.V. Image audio transformation system, particularly as a visual aid for the blind
US20070016425A1 (en) * 2005-07-12 2007-01-18 Koren Ward Device for providing perception of the physical environment
US20090122648A1 (en) * 2007-11-12 2009-05-14 Trustees Of Boston University Acoustic mobility aid for the visually impaired
US20120092460A1 (en) * 2009-06-19 2012-04-19 Andrew Mahoney System And Method For Alerting Visually Impaired Users Of Nearby Objects
WO2013018090A1 (en) * 2011-08-01 2013-02-07 Abir Eliahu System and method for non-visual sensory enhancement

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100586893B1 (en) * 2004-06-28 2006-06-08 삼성전자주식회사 System and method for estimating speaker localization in non-stationary noise environment
PT104120B (en) * 2008-06-30 2010-11-23 Metro Do Porto S A GUIDANCE, NAVIGATION AND INFORMATION SYSTEM SPECIFICALLY ADAPTED TO BLIND OR AMBITIOUS PEOPLE

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3172075A (en) * 1959-11-27 1965-03-02 Nat Res Dev Apparatus for furnishing information as to positioning of objects
US4292678A (en) * 1978-08-14 1981-09-29 Leslie Kay Method of and apparatus for echo location of objects
EP0410045A1 (en) * 1989-07-27 1991-01-30 Koninklijke Philips Electronics N.V. Image audio transformation system, particularly as a visual aid for the blind
US20070016425A1 (en) * 2005-07-12 2007-01-18 Koren Ward Device for providing perception of the physical environment
US20090122648A1 (en) * 2007-11-12 2009-05-14 Trustees Of Boston University Acoustic mobility aid for the visually impaired
US20120092460A1 (en) * 2009-06-19 2012-04-19 Andrew Mahoney System And Method For Alerting Visually Impaired Users Of Nearby Objects
WO2013018090A1 (en) * 2011-08-01 2013-02-07 Abir Eliahu System and method for non-visual sensory enhancement

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
BANKS J ET AL: "Non-parametric techniques for fast and robust stereo matching", TENCON '97. IEEE REGION 10 ANNUAL CONFERENCE. SPEECH AND IMAGE TECHNOL OGIES FOR COMPUTING AND TELECOMMUNICATIONS., PROCEEDINGS OF IEEE BRISBANE, QLD., AUSTRALIA 2-4 DEC. 1997, NEW YORK, NY, USA,IEEE, US, 1 January 1997 (1997-01-01), pages 365 - 368vol.1, XP031172476, ISBN: 978-0-7803-4365-8, DOI: 10.1109/TENCON.1997.647332 *
KAY LESLIE: "Auditory perception of objects by blind persons, using a bioacoustic high resolution air sonar", THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA, AMERICAN INSTITUTE OF PHYSICS FOR THE ACOUSTICAL SOCIETY OF AMERICA, NEW YORK, NY, US, vol. 107, no. 6, 1 June 2000 (2000-06-01), pages 3266 - 3275, XP012001735, ISSN: 0001-4966, DOI: 10.1121/1.429399 *
MEIJER P B L: "AN EXPERIMENTAL SYSTEM FOR AUDITORY IMAGE REPRESENTATIONS", IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, IEEE SERVICE CENTER, PISCATAWAY, NJ, USA, vol. 39, no. 2, 1 February 1992 (1992-02-01), pages 112 - 121, XP000246185, ISSN: 0018-9294, DOI: 10.1109/10.121642 *
PETER B L MEIJER: "THE VOICE AT TUCSON 2002", INTERNET CITATION, 4 May 2006 (2006-05-04), pages 1 - 4, XP009507758, Retrieved from the Internet <URL:http://WEB.ARCHIVE.ORG/WEB/20060504232231/HTTP://WWW.SEEINGWITHSOUND.COM/TUCSON2002.HTML> [retrieved on 20180831] *
See also references of WO2016198721A1 *

Also Published As

Publication number Publication date
RU2017144052A3 (en) 2019-10-29
EP3308759A1 (en) 2018-04-18
EP3308759B1 (en) 2019-11-27
BR112017026545A2 (en) 2018-08-14
JP6771548B2 (en) 2020-10-21
HK1248093A1 (en) 2018-10-12
ES2597155A1 (en) 2017-01-16
PT3308759T (en) 2020-04-01
JP2018524135A (en) 2018-08-30
AU2016275789B2 (en) 2021-03-11
CO2017012744A2 (en) 2018-02-20
MX2017015146A (en) 2018-03-28
BR112017026545B1 (en) 2022-07-12
RU2719025C2 (en) 2020-04-16
DK3308759T3 (en) 2020-03-02
AU2016275789A1 (en) 2018-01-25
ES2780725T3 (en) 2020-08-26
KR20180018587A (en) 2018-02-21
ES2597155B1 (en) 2017-09-18
CN107708624A (en) 2018-02-16
CA2986652A1 (en) 2016-12-15
IL255624A (en) 2018-01-31
RU2017144052A (en) 2019-07-12
AR104959A1 (en) 2017-08-30
IL255624B (en) 2021-04-29
KR102615844B1 (en) 2023-12-21
US20180177640A1 (en) 2018-06-28
US11185445B2 (en) 2021-11-30
WO2016198721A1 (en) 2016-12-15
CN107708624B (en) 2021-12-14

Similar Documents

Publication Publication Date Title
HK1248093A1 (en) Portable system that allows blind or visually impaired persons to interpret the surrounding environment by sound or touch
EP3494554A4 (en) Transducer system providing tactile sensations
EP3329466A4 (en) Mixed-reality architectural design environment
EP3509057A4 (en) Sound-proofing structure and sound-proofing system
EP3149611A4 (en) Learning deep face representation
EP3191108A4 (en) Neural microphysiological systems and methods of using the same
EP3126945A4 (en) Expandable application representation and taskbar
EP3514789A4 (en) Soundproofing structure and soundproofing system
EP3477628A4 (en) Acoustic panel
EP3423645A4 (en) Building panel system
EP3175896A4 (en) Building block and building block unit
EP3146124A4 (en) Panel systems and methods
EP3506254A4 (en) Soundproof structure
EP3550558A4 (en) Soundproofing structure
EP3092361A4 (en) Panel and panel assembly
EP3179832A4 (en) Planar light-emitting panel and elastic jacket
EP3030150A4 (en) Analyte sensor transceiver configured to provide tactile, visual, and / or aural feedback
EP3428462A4 (en) Different-material panel structure
EP3493036A4 (en) Ionic touch panel
EP3426823A4 (en) Portable electrolyzing system
EP3495927A4 (en) Touch panel
EP3114294A4 (en) Portable children&#39;s toilet system
EP3510209A4 (en) Acoustic panel
EP3531255A4 (en) Touch panel device
AU2016904125A0 (en) Personal location system

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20171227

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
A4 Supplementary search report drawn up and despatched

Effective date: 20190128

RIC1 Information provided on ipc code assigned before grant

Ipc: G01B 11/22 20060101ALI20190122BHEP

Ipc: G08B 3/00 20060101ALI20190122BHEP

Ipc: A61F 9/08 20060101AFI20190122BHEP

Ipc: G08B 6/00 20060101ALI20190122BHEP

RIC1 Information provided on ipc code assigned before grant

Ipc: G08B 3/00 20060101ALI20190306BHEP

Ipc: G01B 11/22 20060101ALI20190306BHEP

Ipc: A61F 9/08 20060101AFI20190306BHEP

Ipc: G08B 6/00 20060101ALI20190306BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20190813

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1205892

Country of ref document: AT

Kind code of ref document: T

Effective date: 20191215

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602016025238

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

Effective date: 20200225

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: PT

Ref legal event code: SC4A

Ref document number: 3308759

Country of ref document: PT

Date of ref document: 20200401

Kind code of ref document: T

Free format text: AVAILABILITY OF NATIONAL TRANSLATION

Effective date: 20200325

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200227

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200228

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200227

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200327

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2780725

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20200826

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602016025238

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1205892

Country of ref document: AT

Kind code of ref document: T

Effective date: 20191127

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20200828

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200610

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200610

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20191127

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230505

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20230626

Year of fee payment: 8

Ref country code: ES

Payment date: 20230809

Year of fee payment: 8

Ref country code: CH

Payment date: 20230702

Year of fee payment: 8

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20240607

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240617

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DK

Payment date: 20240612

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20240613

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240617

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: PT

Payment date: 20240607

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: BE

Payment date: 20240613

Year of fee payment: 9