EP2543187A4 - Systems and methods for spatially controlled scene illumination - Google Patents

Systems and methods for spatially controlled scene illumination Download PDF

Info

Publication number
EP2543187A4
EP2543187A4 EP11787052.7A EP11787052A EP2543187A4 EP 2543187 A4 EP2543187 A4 EP 2543187A4 EP 11787052 A EP11787052 A EP 11787052A EP 2543187 A4 EP2543187 A4 EP 2543187A4
Authority
EP
European Patent Office
Prior art keywords
systems
methods
scene illumination
spatially controlled
controlled scene
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP11787052.7A
Other languages
German (de)
French (fr)
Other versions
EP2543187A2 (en
EP2543187B1 (en
Inventor
Nelson G. Publicover
Jason Heffernan
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Google LLC
Original Assignee
Google LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Google LLC filed Critical Google LLC
Publication of EP2543187A2 publication Critical patent/EP2543187A2/en
Publication of EP2543187A4 publication Critical patent/EP2543187A4/en
Application granted granted Critical
Publication of EP2543187B1 publication Critical patent/EP2543187B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/74—Circuitry for compensating brightness variation in the scene by influencing the scene brightness using illuminating means
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • G—PHYSICS
    • G02—OPTICS
    • G02C—SPECTACLES; SUNGLASSES OR GOGGLES INSOFAR AS THEY HAVE THE SAME FEATURES AS SPECTACLES; CONTACT LENSES
    • G02C11/00—Non-optical adjuncts; Attachment thereof
    • G02C11/10—Electronic devices other than hearing aids
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/11—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths for generating image signals from visible and infrared light wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/61—Control of cameras or camera modules based on recognised objects
    • H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/71—Circuitry for evaluating the brightness variation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
EP11787052.7A 2010-03-01 2011-03-01 Systems and methods for spatially controlled scene illumination Active EP2543187B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US12/715,177 US8890946B2 (en) 2010-03-01 2010-03-01 Systems and methods for spatially controlled scene illumination
PCT/US2011/026736 WO2011149577A2 (en) 2010-03-01 2011-03-01 Systems and methods for spatially controlled scene illumination

Publications (3)

Publication Number Publication Date
EP2543187A2 EP2543187A2 (en) 2013-01-09
EP2543187A4 true EP2543187A4 (en) 2017-12-20
EP2543187B1 EP2543187B1 (en) 2022-11-16

Family

ID=44505076

Family Applications (1)

Application Number Title Priority Date Filing Date
EP11787052.7A Active EP2543187B1 (en) 2010-03-01 2011-03-01 Systems and methods for spatially controlled scene illumination

Country Status (8)

Country Link
US (2) US8890946B2 (en)
EP (1) EP2543187B1 (en)
JP (1) JP2013521710A (en)
KR (1) KR101829850B1 (en)
CN (1) CN102918834B (en)
AU (1) AU2011258826A1 (en)
RU (1) RU2012141574A (en)
WO (1) WO2011149577A2 (en)

Families Citing this family (142)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9250703B2 (en) 2006-03-06 2016-02-02 Sony Computer Entertainment Inc. Interface with gaze detection and voice input
US8730156B2 (en) 2010-03-05 2014-05-20 Sony Computer Entertainment America Llc Maintaining multiple views on a shared stable virtual space
US9158116B1 (en) 2014-04-25 2015-10-13 Osterhout Group, Inc. Temple and ear horn assembly for headworn computer
US9400390B2 (en) 2014-01-24 2016-07-26 Osterhout Group, Inc. Peripheral lighting for head worn computing
US9715112B2 (en) 2014-01-21 2017-07-25 Osterhout Group, Inc. Suppression of stray light in head worn computing
US9366867B2 (en) 2014-07-08 2016-06-14 Osterhout Group, Inc. Optical systems for see-through displays
KR101383235B1 (en) * 2010-06-17 2014-04-17 한국전자통신연구원 Apparatus for inputting coordinate using eye tracking and method thereof
WO2012011181A1 (en) * 2010-07-22 2012-01-26 富士通株式会社 Vein image capturing device
US20130154913A1 (en) * 2010-12-16 2013-06-20 Siemens Corporation Systems and methods for a gaze and gesture interface
US10120438B2 (en) * 2011-05-25 2018-11-06 Sony Interactive Entertainment Inc. Eye gaze to alter device behavior
KR101917630B1 (en) 2011-10-28 2018-11-13 매직 립, 인코포레이티드 System and method for augmented and virtual reality
US9854159B2 (en) * 2012-07-20 2017-12-26 Pixart Imaging Inc. Image system with eye protection
TWI471808B (en) * 2012-07-20 2015-02-01 Pixart Imaging Inc Pupil detection device
TW201416908A (en) 2012-10-23 2014-05-01 Pixart Imaging Inc Pupil tracking device
CN103793045B (en) * 2012-10-31 2016-12-28 原相科技股份有限公司 Pupil tracing device
US20140218281A1 (en) 2012-12-06 2014-08-07 Eyefluence, Inc. Systems and methods for eye gaze determination
US8761594B1 (en) 2013-02-28 2014-06-24 Apple Inc. Spatially dynamic illumination for camera systems
CA2902771C (en) 2013-03-15 2018-08-14 Synaptive Medical (Barbados) Inc. Context aware surgical systems
US10063782B2 (en) * 2013-06-18 2018-08-28 Motorola Solutions, Inc. Method and apparatus for displaying an image from a camera
KR101882594B1 (en) 2013-09-03 2018-07-26 토비 에이비 Portable eye tracking device
US10310597B2 (en) 2013-09-03 2019-06-04 Tobii Ab Portable eye tracking device
US10686972B2 (en) 2013-09-03 2020-06-16 Tobii Ab Gaze assisted field of view control
TWI532413B (en) * 2013-09-11 2016-05-01 晶睿通訊股份有限公司 Light compensating system and method thereof
US9736373B2 (en) * 2013-10-25 2017-08-15 Intel Corporation Dynamic optimization of light source power
US10649220B2 (en) 2014-06-09 2020-05-12 Mentor Acquisition One, Llc Content presentation in head worn computing
US9841599B2 (en) 2014-06-05 2017-12-12 Osterhout Group, Inc. Optical configurations for head-worn see-through displays
US9829707B2 (en) 2014-08-12 2017-11-28 Osterhout Group, Inc. Measuring content brightness in head worn computing
US9594246B2 (en) 2014-01-21 2017-03-14 Osterhout Group, Inc. See-through computer display systems
US10254856B2 (en) 2014-01-17 2019-04-09 Osterhout Group, Inc. External user interface for head worn computing
US20150228119A1 (en) 2014-02-11 2015-08-13 Osterhout Group, Inc. Spatial location presentation in head worn computing
US10684687B2 (en) 2014-12-03 2020-06-16 Mentor Acquisition One, Llc See-through computer display systems
US11669163B2 (en) 2014-01-21 2023-06-06 Mentor Acquisition One, Llc Eye glint imaging in see-through computer display systems
US11892644B2 (en) 2014-01-21 2024-02-06 Mentor Acquisition One, Llc See-through computer display systems
US9753288B2 (en) 2014-01-21 2017-09-05 Osterhout Group, Inc. See-through computer display systems
US11487110B2 (en) 2014-01-21 2022-11-01 Mentor Acquisition One, Llc Eye imaging in head worn computing
US11737666B2 (en) 2014-01-21 2023-08-29 Mentor Acquisition One, Llc Eye imaging in head worn computing
US20160018654A1 (en) 2014-01-24 2016-01-21 Osterhout Group, Inc. See-through computer display systems
US9846308B2 (en) 2014-01-24 2017-12-19 Osterhout Group, Inc. Haptic systems for head-worn computers
US20170090557A1 (en) * 2014-01-29 2017-03-30 Google Inc. Systems and Devices for Implementing a Side-Mounted Optical Sensor
US9396571B2 (en) * 2014-02-10 2016-07-19 International Business Machines Corporation Simplified lighting compositing
US9651787B2 (en) 2014-04-25 2017-05-16 Osterhout Group, Inc. Speaker assembly for headworn computer
US10853589B2 (en) 2014-04-25 2020-12-01 Mentor Acquisition One, Llc Language translation with head-worn computing
US20160137312A1 (en) 2014-05-06 2016-05-19 Osterhout Group, Inc. Unmanned aerial vehicle launch system
US10564714B2 (en) 2014-05-09 2020-02-18 Google Llc Systems and methods for biomechanically-based eye signals for interacting with real and virtual objects
KR102230172B1 (en) 2014-05-09 2021-03-19 아이플루언스, 인크. Systems and methods for biomechanically-based eye signals for interacting with real and virtual objects
US10663740B2 (en) 2014-06-09 2020-05-26 Mentor Acquisition One, Llc Content presentation in head worn computing
JP2016032257A (en) * 2014-07-30 2016-03-07 株式会社デンソー Driver monitoring device
WO2016045911A1 (en) * 2014-09-25 2016-03-31 Philips Lighting Holding B.V. Control of lighting.
WO2016084385A1 (en) * 2014-11-27 2016-06-02 京セラ株式会社 Imaging device and vehicle
US9635231B2 (en) 2014-12-22 2017-04-25 Google Inc. Time-of-flight camera system and method to improve measurement quality of weak field-of-view signal regions
US9674415B2 (en) 2014-12-22 2017-06-06 Google Inc. Time-of-flight camera system with scanning illuminator
US10878775B2 (en) 2015-02-17 2020-12-29 Mentor Acquisition One, Llc See-through computer display systems
CN107533632A (en) * 2015-04-02 2018-01-02 埃西勒国际通用光学公司 Method for being updated to the index of individual
EP3281056A1 (en) * 2015-04-10 2018-02-14 Essilor International Head mounted display device
US9891077B2 (en) 2015-05-19 2018-02-13 Magic Leap, Inc. Dual composite light field device
US10139966B2 (en) 2015-07-22 2018-11-27 Osterhout Group, Inc. External user interface for head worn computing
US11003246B2 (en) 2015-07-22 2021-05-11 Mentor Acquisition One, Llc External user interface for head worn computing
JP2018533099A (en) 2015-09-24 2018-11-08 カリフォルニア インスティチュート オブ テクノロジー Data visualization system and method using three-dimensional display
CN108369364B (en) * 2015-11-10 2021-07-16 亮锐控股有限公司 Adaptive light source
CN106814518A (en) * 2015-12-01 2017-06-09 深圳富泰宏精密工业有限公司 Auto-focusing camera system and electronic installation
US10480937B2 (en) * 2016-01-28 2019-11-19 Manatec Electronics Pvt. Ltd. System for simultaneous measurement of wheel alignment angles and wheel runout of multi-axle vehicles
US10850116B2 (en) 2016-12-30 2020-12-01 Mentor Acquisition One, Llc Head-worn therapy device
US10591728B2 (en) 2016-03-02 2020-03-17 Mentor Acquisition One, Llc Optical systems for head-worn computers
US10667981B2 (en) 2016-02-29 2020-06-02 Mentor Acquisition One, Llc Reading assistance system for visually impaired
US9826299B1 (en) 2016-08-22 2017-11-21 Osterhout Group, Inc. Speaker systems for head-worn computer systems
US9880441B1 (en) 2016-09-08 2018-01-30 Osterhout Group, Inc. Electrochromic systems for head-worn computer systems
US10466491B2 (en) 2016-06-01 2019-11-05 Mentor Acquisition One, Llc Modular systems for head-worn computers
US10084979B2 (en) * 2016-07-29 2018-09-25 International Business Machines Corporation Camera apparatus and system, method and recording medium for indicating camera field of view
KR102548199B1 (en) * 2016-08-03 2023-06-28 삼성전자주식회사 Electronic device and method for tracking gaze in the electronic device
CN106128406B (en) * 2016-09-08 2019-01-22 京东方科技集团股份有限公司 Eye map sheet value adjusting method, data transmission method, circuit and display device
US10925479B2 (en) * 2016-10-13 2021-02-23 Ronald Michael Kurtz Networked system of mobile communication platforms for nonpharmacologic constriction of a pupil
US10580234B2 (en) 2017-01-20 2020-03-03 Adesa, Inc. Vehicle documentation system
US10302764B2 (en) * 2017-02-03 2019-05-28 Microsoft Technology Licensing, Llc Active illumination management through contextual information
CN108507636B (en) * 2017-02-27 2020-11-20 深圳市朗驰欣创科技股份有限公司 Method for detecting water accumulation condition of channel, inspection device and water level detection equipment
EP3603108A4 (en) 2017-03-30 2021-04-14 Magic Leap, Inc. Non-blocking dual driver earphones
US10977858B2 (en) 2017-03-30 2021-04-13 Magic Leap, Inc. Centralized rendering
CN117389420A (en) 2017-04-14 2024-01-12 奇跃公司 Multimode eye tracking
US10295827B1 (en) * 2017-04-27 2019-05-21 Facebook Technologies, Llc Diffractive optics beam shaping for structured light generator
US11079522B1 (en) 2017-05-31 2021-08-03 Magic Leap, Inc. Fiducial design
US10578869B2 (en) 2017-07-24 2020-03-03 Mentor Acquisition One, Llc See-through computer display systems with adjustable zoom cameras
US10422995B2 (en) 2017-07-24 2019-09-24 Mentor Acquisition One, Llc See-through computer display systems with stray light management
US11409105B2 (en) 2017-07-24 2022-08-09 Mentor Acquisition One, Llc See-through computer display systems
CN110892443B (en) * 2017-08-01 2024-01-09 松下知识产权经营株式会社 Personal authentication device
US10969584B2 (en) 2017-08-04 2021-04-06 Mentor Acquisition One, Llc Image expansion optic for head-worn computer
IL297445B2 (en) 2017-10-17 2024-03-01 Magic Leap Inc Mixed reality spatial audio
US11194161B2 (en) 2018-02-09 2021-12-07 Pupil Labs Gmbh Devices, systems and methods for predicting gaze-related parameters
WO2019154511A1 (en) 2018-02-09 2019-08-15 Pupil Labs Gmbh Devices, systems and methods for predicting gaze-related parameters using a neural network
US11393251B2 (en) 2018-02-09 2022-07-19 Pupil Labs Gmbh Devices, systems and methods for predicting gaze-related parameters
IL307545A (en) 2018-02-15 2023-12-01 Magic Leap Inc Dual listener positions for mixed reality
US11403825B2 (en) 2018-02-15 2022-08-02 Magic Leap, Inc. Mixed reality musical instrument
IL276510B2 (en) 2018-02-15 2024-02-01 Magic Leap Inc Mixed reality virtual reverberation
CN112119340B (en) * 2018-04-18 2022-07-12 金泰克斯公司 Illumination of limited field of view in surgical microscopes
EP3782438A4 (en) 2018-04-19 2022-05-18 Seeing Machines Limited Infrared light source protective system
US10621762B2 (en) 2018-05-14 2020-04-14 Virtualitics, Inc. Systems and methods for high dimensional 3D data visualization
US10509467B1 (en) * 2018-06-01 2019-12-17 Facebook Technologies, Llc Determining fixation of a user's eyes from images of portions of the user's face enclosed by a head mounted display
US10667072B2 (en) 2018-06-12 2020-05-26 Magic Leap, Inc. Efficient rendering of virtual soundfields
US10602292B2 (en) 2018-06-14 2020-03-24 Magic Leap, Inc. Methods and systems for audio signal filtering
WO2019241754A1 (en) 2018-06-14 2019-12-19 Magic Leap, Inc. Reverberation gain normalization
WO2019246164A1 (en) 2018-06-18 2019-12-26 Magic Leap, Inc. Spatial audio for interactive audio environments
US11854566B2 (en) 2018-06-21 2023-12-26 Magic Leap, Inc. Wearable system speech processing
US11201953B2 (en) 2018-07-24 2021-12-14 Magic Leap, Inc. Application sharing
US10506681B1 (en) 2018-08-09 2019-12-10 Distech Controls Inc. Environment controller and method for proportionally adjusting the light intensity of several lighting devices
CN117572954A (en) 2018-09-25 2024-02-20 奇跃公司 System and method for augmented reality
CN116249053A (en) 2018-10-05 2023-06-09 奇跃公司 Inter-aural time difference crossfaders for binaural audio rendering
JP7194271B2 (en) 2018-10-05 2022-12-21 マジック リープ， インコーポレイテッド Near-field audio rendering
EP3864575A4 (en) 2018-10-09 2021-12-01 Magic Leap, Inc. Systems and methods for virtual and augmented reality
EP3871062A4 (en) 2018-10-24 2021-12-29 Magic Leap, Inc. Asynchronous asic
CN111127537A (en) * 2018-10-29 2020-05-08 托比股份公司 Method and apparatus for detecting shadows in a head mounted device
US11509870B2 (en) * 2018-11-07 2022-11-22 Sony Group Corporation Image acquisition system and image acquisition method
US11104279B2 (en) * 2018-11-26 2021-08-31 Magna Electronics Solutions Gmbh Vehicle vision system with adaptive reversing light
JP2022516074A (en) 2018-12-27 2022-02-24 マジック リープ， インコーポレイテッド Systems and methods for virtual and augmented reality
WO2020147948A1 (en) 2019-01-16 2020-07-23 Pupil Labs Gmbh Methods for generating calibration data for head-wearable devices and eye tracking system
EP3931827A4 (en) 2019-03-01 2022-11-02 Magic Leap, Inc. Determining input for speech processing engine
EP3979896A1 (en) 2019-06-05 2022-04-13 Pupil Labs GmbH Devices, systems and methods for predicting gaze-related parameters
JP7369212B2 (en) 2019-06-06 2023-10-25 マジック リープ， インコーポレイテッド Photorealistic character construction for spatial computing
CA3136749A1 (en) 2019-06-11 2020-12-17 Gentex Corporation Wearable medical device
US11704874B2 (en) 2019-08-07 2023-07-18 Magic Leap, Inc. Spatial instructions and guides in mixed reality
US11328740B2 (en) 2019-08-07 2022-05-10 Magic Leap, Inc. Voice onset detection
CN110490017B (en) * 2019-08-08 2022-02-22 广东斯玛特自动化科技有限公司 Illumination method and device for code scanner
US11328475B2 (en) 2019-10-18 2022-05-10 Magic Leap, Inc. Gravity estimation and bundle adjustment for visual-inertial odometry
US11304017B2 (en) 2019-10-25 2022-04-12 Magic Leap, Inc. Reverberation fingerprint estimation
CN114846434A (en) 2019-10-25 2022-08-02 奇跃公司 Non-uniform stereoscopic rendering
US11959997B2 (en) 2019-11-22 2024-04-16 Magic Leap, Inc. System and method for tracking a wearable device
WO2021113782A1 (en) 2019-12-04 2021-06-10 Magic Leap, Inc. Variable-pitch color emitting display
EP4070284A4 (en) 2019-12-06 2023-05-24 Magic Leap, Inc. Environment acoustics persistence
US11269181B2 (en) 2019-12-09 2022-03-08 Magic Leap, Inc. Systems and methods for operating a head-mounted display system based on user identity
US11337023B2 (en) 2019-12-20 2022-05-17 Magic Leap, Inc. Physics-based audio and haptic synthesis
EP4103999A4 (en) 2020-02-14 2023-08-02 Magic Leap, Inc. Session manager
US11778410B2 (en) 2020-02-14 2023-10-03 Magic Leap, Inc. Delayed audio following
EP4104000A4 (en) 2020-02-14 2023-07-12 Magic Leap, Inc. Tool bridge
CN115398316A (en) 2020-02-14 2022-11-25 奇跃公司 3D object annotation
EP4104456A4 (en) 2020-02-14 2023-07-19 Magic Leap, Inc. Multi-application audio rendering
US11205069B1 (en) * 2020-02-20 2021-12-21 Facebook Technologies, Llc Hybrid cornea and pupil tracking
CN116325808B (en) 2020-03-02 2023-12-22 奇跃公司 Immersive audio platform
US11917384B2 (en) 2020-03-27 2024-02-27 Magic Leap, Inc. Method of waking a device using spoken voice commands
EP4158908A4 (en) 2020-05-29 2023-11-29 Magic Leap, Inc. Surface appropriate collisions
WO2021243103A1 (en) 2020-05-29 2021-12-02 Magic Leap, Inc. Determining angular acceleration
KR20220169207A (en) * 2021-06-18 2022-12-27 삼성전자주식회사 Electronic device and biometric liveness authentication method
US11645844B2 (en) * 2021-10-05 2023-05-09 RS1 Worklete, LLC Computing devices programmed to detect slippery surfaces within enclosures and methods/systems of used thereof
US11592899B1 (en) 2021-10-28 2023-02-28 Tectus Corporation Button activation within an eye-controlled user interface
US11619994B1 (en) 2022-01-14 2023-04-04 Tectus Corporation Control of an electronic contact lens using pitch-based eye gestures
US11874961B2 (en) 2022-05-09 2024-01-16 Tectus Corporation Managing display of an icon in an eye tracking augmented reality device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050265014A1 (en) * 2004-05-25 2005-12-01 Nikon Corporation Illuminating device for photographing and camera
WO2007092512A2 (en) * 2006-02-07 2007-08-16 Attention Technologies, Inc. Driver drowsiness and distraction monitor
EP2005888A2 (en) * 2006-05-30 2008-12-24 Panasonic Corporation Imaging device and authentication device using the same
US20090018419A1 (en) * 2004-04-01 2009-01-15 Torch William C Biosensors, communicators, and controllers monitoring eye movement and methods for using them

Family Cites Families (125)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3689135A (en) 1971-01-26 1972-09-05 Laurence R Young Method for monitoring movement of a subject{40 s eyes
US3863243A (en) 1972-01-19 1975-01-28 Max Skolnick Sleep inhibiting alarm
US3798599A (en) 1972-02-24 1974-03-19 H Kafafian Single input controller for a communication system
US3966310A (en) 1974-02-15 1976-06-29 Larson Merlin D Pupillometer and method of use thereof
US4102564A (en) 1975-04-18 1978-07-25 Michael Henry L Portable device for the accurate measurement of eye movements both in light and obscurity
US4359724A (en) 1980-04-28 1982-11-16 Ronald R. Zimmerman Eyelid movement detector
EP0125808A3 (en) 1983-04-18 1986-01-29 Lee S. Weinblatt Eye movement monitoring technique
JPS6332624A (en) 1986-07-28 1988-02-12 Canon Inc Information processor
EP0280124A1 (en) 1987-02-12 1988-08-31 Omron Tateisi Electronics Co. Doze detector
US4850691A (en) 1987-03-18 1989-07-25 University Of Illinois Method and apparatus for determining pupillary response parameters
US4815839A (en) 1987-08-03 1989-03-28 Waldorf Ronald A Infrared/video electronystagmographic apparatus
JPH01312902A (en) 1988-06-13 1989-12-18 Konan Camera Kenkyusho:Kk Examination device of motion of eyeball
US4852988A (en) 1988-09-12 1989-08-01 Applied Science Laboratories Visor and camera providing a parallax-free field-of-view image for a head-mounted eye movement measurement system
JPH02164335A (en) 1988-12-16 1990-06-25 Konan Camera Kenkyusho:Kk Eyeball motion analyzing device
GB8916206D0 (en) 1989-07-14 1989-11-08 Marconi Gec Ltd Helmet systems
US4967186A (en) 1989-08-18 1990-10-30 Ariold Ludmirsky Method and apparatus for fatigue detection
US5778893A (en) 1991-04-01 1998-07-14 President And Fellows Of Harvard College Method of diagnosing and monitoring a treatment for Alzheimer's disease
US5189512A (en) 1991-07-01 1993-02-23 Camair Research, Inc. Helmet integrated display system
US5183512A (en) * 1991-08-29 1993-02-02 Aquotech, Inc. Method of cleaning semiconductor wafer and computer disks by purified water treated with electric AC signal
WO1995018565A1 (en) 1991-09-26 1995-07-13 Sam Technology, Inc. Non-invasive neurocognitive testing method and system
US5214456A (en) 1991-10-09 1993-05-25 Computed Anatomy Incorporated Mapping of corneal topography with display of pupil perimeter
US5583590A (en) 1992-05-04 1996-12-10 Wabash Scientific Corp. Alert monitoring system
US5341181A (en) 1992-11-20 1994-08-23 Godard Roger R Systems and methods for capturing and presentng visual information
US5345281A (en) 1992-12-17 1994-09-06 John Taboada Eye tracking system and method
US5461417A (en) * 1993-02-16 1995-10-24 Northeast Robotics, Inc. Continuous diffuse illumination method and apparatus
US5402109A (en) 1993-04-29 1995-03-28 Mannik; Kallis H. Sleep prevention device for automobile drivers
GB9323970D0 (en) 1993-11-22 1994-01-12 Toad Innovations Ltd Safety device
US5478239A (en) 1993-12-21 1995-12-26 Maximum Performance, Inc. Dynamic visual acuity training method and apparatus
US5481622A (en) 1994-03-01 1996-01-02 Rensselaer Polytechnic Institute Eye tracking apparatus and method employing grayscale threshold values
JP3293308B2 (en) 1994-03-10 2002-06-17 三菱電機株式会社 Person state detection device
JPH086708A (en) 1994-04-22 1996-01-12 Canon Inc Display device
US5617872A (en) 1994-07-25 1997-04-08 Beth Israel Hospitcal Assoc. Inc. Hypersensitive constriction velocity method for diagnosing Alzheimer's disease in a living human
US5469143A (en) 1995-01-10 1995-11-21 Cooper; David E. Sleep awakening device for drivers of motor vehicles
US5583795A (en) 1995-03-17 1996-12-10 The United States Of America As Represented By The Secretary Of The Army Apparatus for measuring eye gaze and fixation duration, and method therefor
US5566067A (en) 1995-03-23 1996-10-15 The President And Fellows Of Harvard College Eyelid vigilance detector system
US5689241A (en) 1995-04-24 1997-11-18 Clarke, Sr.; James Russell Sleep detection and driver alert apparatus
US5570698A (en) 1995-06-02 1996-11-05 Siemens Corporate Research, Inc. System for monitoring eyes for detecting sleep behavior
US7738678B2 (en) * 1995-06-07 2010-06-15 Automotive Technologies International, Inc. Light modulation techniques for imaging objects in or around a vehicle
US5682144A (en) 1995-11-20 1997-10-28 Mannik; Kallis Hans Eye actuated sleep prevention devices and other eye controlled devices
US6003991A (en) 1996-02-17 1999-12-21 Erik Scott Viirre Eye examination apparatus and method for remote examination of a patient by a health professional
US6373961B1 (en) 1996-03-26 2002-04-16 Eye Control Technologies, Inc. Eye controllable screen pointer
US5726916A (en) 1996-06-27 1998-03-10 The United States Of America As Represented By The Secretary Of The Army Method and apparatus for determining ocular gaze point of regard and fixation duration
US5861936A (en) 1996-07-26 1999-01-19 Gillan Holdings Limited Regulating focus in accordance with relationship of features of a person's eyes
US6163281A (en) 1996-08-19 2000-12-19 Torch; William C. System and method for communication using eye movement
US6246344B1 (en) 1996-08-19 2001-06-12 William C. Torch Method and apparatus for voluntary communication
US6542081B2 (en) 1996-08-19 2003-04-01 William C. Torch System and method for monitoring eye movement
US5748113A (en) 1996-08-19 1998-05-05 Torch; William C. Method and apparatus for communication
JP3651204B2 (en) 1996-12-18 2005-05-25 トヨタ自動車株式会社 Stereoscopic image display device, stereoscopic image display method, and recording medium
US6070098A (en) 1997-01-11 2000-05-30 Circadian Technologies, Inc. Method of and apparatus for evaluation and mitigation of microsleep events
WO1998049028A1 (en) 1997-04-25 1998-11-05 Applied Science Group, Inc. An alertness monitor
US5867587A (en) 1997-05-19 1999-02-02 Northrop Grumman Corporation Impaired operator detection and warning system employing eyeblink analysis
WO1998057579A1 (en) 1997-06-19 1998-12-23 Bioprobes, Inc. System and method for screening for dementia
US6007202A (en) 1997-10-23 1999-12-28 Lasersight Technologies, Inc. Eye illumination system and method
DE69840547D1 (en) 1997-10-30 2009-03-26 Myvu Corp INTERFACE SYSTEM FOR GLASSES
US6611618B1 (en) 1997-11-13 2003-08-26 Schepens Eye Research Institute, Inc. Wide-band image enhancement
US6055322A (en) 1997-12-01 2000-04-25 Sensor, Inc. Method and apparatus for illuminating and imaging eyes through eyeglasses using multiple sources of illumination
JP3361980B2 (en) 1997-12-12 2003-01-07 株式会社東芝 Eye gaze detecting apparatus and method
US6088470A (en) 1998-01-27 2000-07-11 Sensar, Inc. Method and apparatus for removal of bright or dark spots by the fusion of multiple images
DE19803158C1 (en) 1998-01-28 1999-05-06 Daimler Chrysler Ag Arrangement for determining the state of vigilance, esp. for machinery operator or vehicle driver
US6152563A (en) 1998-02-20 2000-11-28 Hutchinson; Thomas E. Eye gaze direction tracker
US6003992A (en) * 1998-06-02 1999-12-21 Northeast Robotics Llc Back lighting illumination system
US6091378A (en) 1998-06-17 2000-07-18 Eye Control Technologies, Inc. Video processing methods and apparatus for gaze point tracking
US6867752B1 (en) 1998-08-31 2005-03-15 Semiconductor Energy Laboratory Co., Ltd. Portable information processing system
US6087941A (en) 1998-09-01 2000-07-11 Ferraz; Mark Warning device for alerting a person falling asleep
JP2000137792A (en) 1998-10-30 2000-05-16 Toyota Motor Corp Eye part detecting device
US6433760B1 (en) 1999-01-14 2002-08-13 University Of Central Florida Head mounted display with eyetracking capability
US7120880B1 (en) 1999-02-25 2006-10-10 International Business Machines Corporation Method and system for real-time determination of a subject's interest level to media content
EP1157326B1 (en) 1999-03-01 2002-10-02 BAE Systems Electronics Ltd. Head tracker system
US6090051A (en) 1999-03-03 2000-07-18 Marshall; Sandra P. Method and apparatus for eye tracking and monitoring pupil dilation to evaluate cognitive activity
US6247813B1 (en) 1999-04-09 2001-06-19 Iritech, Inc. Iris identification system and method of identifying a person through iris recognition
US20020024633A1 (en) 1999-04-09 2002-02-28 Daehoon Kim Pupil evaluation system
US6116736A (en) 1999-04-23 2000-09-12 Neuroptics, Inc. Pupilometer with pupil irregularity detection capability
US6820979B1 (en) 1999-04-23 2004-11-23 Neuroptics, Inc. Pupilometer with pupil irregularity detection, pupil tracking, and pupil response detection capability, glaucoma screening capability, intracranial pressure detection capability, and ocular aberration measurement capability
US6346887B1 (en) 1999-09-14 2002-02-12 The United States Of America As Represented By The Secretary Of The Navy Eye activity monitor
GB0023172D0 (en) 2000-09-20 2000-11-01 Minter Kemp Martin J Wakeup vizor
JP2002143094A (en) 2000-11-07 2002-05-21 Nac Image Technology Inc Visual axis detector
US6864473B2 (en) 2000-12-07 2005-03-08 The United States Of America As Represented By The United States National Aeronautics And Space Administration Dynamic optical filtration
US6634553B2 (en) 2001-03-21 2003-10-21 Hewlett-Packard Development Company, L.P. Method and apparatus for even illumination of a target for image capture
JP3812355B2 (en) * 2001-03-22 2006-08-23 日産自動車株式会社 Driving attention detection device and drowsy driving detection device
US6578962B1 (en) * 2001-04-27 2003-06-17 International Business Machines Corporation Calibration-free eye gaze tracking
US6959102B2 (en) 2001-05-29 2005-10-25 International Business Machines Corporation Method for increasing the signal-to-noise in IR-based eye gaze trackers
US6775060B2 (en) 2001-07-26 2004-08-10 The Schepens Eye Research Institute, Inc. Bioptic telescope system embedded into a spectacle lens
GB0119859D0 (en) 2001-08-15 2001-10-10 Qinetiq Ltd Eye tracking system
US6927694B1 (en) 2001-08-20 2005-08-09 Research Foundation Of The University Of Central Florida Algorithm for monitoring head/eye motion for driver alertness with one camera
US6997556B2 (en) 2001-10-01 2006-02-14 Ernst Pfleger Method for detecting, evaluating, and analyzing look sequences
AUPR872301A0 (en) 2001-11-08 2001-11-29 Sleep Diagnostics Pty Ltd Alertness monitor
US7206435B2 (en) * 2002-03-26 2007-04-17 Honda Giken Kogyo Kabushiki Kaisha Real-time eye detection and tracking under various light conditions
JP2004034938A (en) * 2002-07-08 2004-02-05 Yazaki Corp Dozing-off drive prevention device and vehicle operation information collection device
US20040061680A1 (en) 2002-07-10 2004-04-01 John Taboada Method and apparatus for computer control
NL1021496C2 (en) 2002-09-19 2004-03-22 Joannes Hermanus Heiligers Monitoring device for alertness of person, especially vehicle driver, generates warning signal if eye is detected as being closed too long
US6637883B1 (en) 2003-01-23 2003-10-28 Vishwas V. Tengshe Gaze tracking system and method
US7347551B2 (en) 2003-02-13 2008-03-25 Fergason Patent Properties, Llc Optical system for monitoring eye movement
EP1482768B1 (en) 2003-05-30 2009-01-07 Continental Automotive GmbH Method and driver for driving electroluminescent lamps
US7391888B2 (en) 2003-05-30 2008-06-24 Microsoft Corporation Head pose assessment methods and systems
US7731360B2 (en) 2003-11-07 2010-06-08 Neuro Kinetics Portable video oculography system
US20050100191A1 (en) * 2003-11-11 2005-05-12 Harbach Andrew P. Imaging system and method for monitoring an eye
US7963652B2 (en) * 2003-11-14 2011-06-21 Queen's University At Kingston Method and apparatus for calibration-free eye tracking
US7374284B2 (en) 2003-12-17 2008-05-20 The Schepens Eye Research Institute, Inc. Peripheral field expansion device
US7331671B2 (en) 2004-03-29 2008-02-19 Delphi Technologies, Inc. Eye tracking method based on correlation and detected eye movement
US7775679B2 (en) * 2004-08-18 2010-08-17 Advanced Illumination, Inc. High intensity light source for a machine vision system and method of making same
US7438418B2 (en) 2005-02-23 2008-10-21 Eyetracking, Inc. Mental alertness and mental proficiency level determination
WO2006091893A2 (en) 2005-02-23 2006-08-31 Eyetracking, Inc. Mental alertness level determination
WO2006092022A1 (en) 2005-03-04 2006-09-08 Sleep Diagnostics Pty. Ltd Measuring alertness
US7430365B2 (en) * 2005-03-31 2008-09-30 Avago Technologies Ecbu (Singapore) Pte Ltd. Safe eye detection
JP2006307611A (en) 2005-04-28 2006-11-09 Shingo Kizai Kk Opposed reflection diffusion type high output led internal illumination system road sign
DE602006016961D1 (en) 2005-08-11 2010-10-28 Sleep Diagnostics Pty Ltd WATCHMAKING GLASSES
WO2007023798A1 (en) * 2005-08-23 2007-03-01 National University Corporation Shizuoka University Pupil detection device and pupil detection method
JP4921486B2 (en) 2005-11-30 2012-04-25 シーイング・マシーンズ・プロプライエタリー・リミテッド Visual tracking of glasses in a system that visually tracks head and eyes
ES2605367T3 (en) 2006-01-26 2017-03-14 Nokia Technologies Oy Eye tracking device
JP2007195775A (en) 2006-01-27 2007-08-09 Keio Gijuku Pupil detection method, pupil position detector, and transmission type eye-direction detector
JP5850600B2 (en) 2006-06-28 2016-02-03 コーニンクレッカ フィリップス エヌ ヴェＫｏｎｉｎｋｌｉｊｋｅ Ｐｈｉｌｉｐｓ Ｎ．Ｖ． Lighting system control method based on target light distribution
US7646422B2 (en) 2006-10-04 2010-01-12 Branislav Kisacanin Illumination and imaging system with glare reduction and method therefor
JP2008091840A (en) 2006-10-05 2008-04-17 Sony Corp Solid state imaging device and electronic device
US8225229B2 (en) 2006-11-09 2012-07-17 Sony Mobile Communications Ab Adjusting display brightness and/or refresh rates based on eye tracking
SE0602545L (en) 2006-11-29 2008-05-30 Tobii Technology Ab Eye tracking illumination
US7783077B2 (en) 2006-12-01 2010-08-24 The Boeing Company Eye gaze tracker system and method
US20080128498A1 (en) * 2006-12-05 2008-06-05 Fausak Andrew T System, method, and apparatus for high value product management and tracking
KR100850357B1 (en) 2006-12-06 2008-08-04 한국전자통신연구원 System and method for tracking gaze
US20080166052A1 (en) 2007-01-10 2008-07-10 Toshinobu Hatano Face condition determining device and imaging device
WO2008136067A1 (en) * 2007-04-20 2008-11-13 Meinan Machinery Works, Inc. Lumber inspection method, device and program
US7633244B2 (en) 2007-09-19 2009-12-15 Zippy Technology Corp. Method for dimming multiple lamps
CN101945612B (en) 2008-02-14 2013-09-25 诺基亚公司 Device and method for determining gaze direction
WO2010003410A1 (en) 2008-07-08 2010-01-14 It-University Of Copenhagen Eye gaze tracking
ATE527934T1 (en) 2009-04-01 2011-10-15 Tobii Technology Ab ADAPTIVE CAMERA AND ILLUMINATOR EYE TRACKER
ES2880475T3 (en) 2009-04-01 2021-11-24 Tobii Ab Visual representation system with illuminators for gaze tracking

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090018419A1 (en) * 2004-04-01 2009-01-15 Torch William C Biosensors, communicators, and controllers monitoring eye movement and methods for using them
US20050265014A1 (en) * 2004-05-25 2005-12-01 Nikon Corporation Illuminating device for photographing and camera
WO2007092512A2 (en) * 2006-02-07 2007-08-16 Attention Technologies, Inc. Driver drowsiness and distraction monitor
EP2005888A2 (en) * 2006-05-30 2008-12-24 Panasonic Corporation Imaging device and authentication device using the same

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of WO2011149577A2 *

Also Published As

Publication number Publication date
KR101829850B1 (en) 2018-03-29
EP2543187A2 (en) 2013-01-09
EP2543187B1 (en) 2022-11-16
US8890946B2 (en) 2014-11-18
JP2013521710A (en) 2013-06-10
AU2011258826A1 (en) 2012-10-11
WO2011149577A3 (en) 2012-02-02
CN102918834A (en) 2013-02-06
US20150181100A1 (en) 2015-06-25
CN102918834B (en) 2016-08-03
US20110211056A1 (en) 2011-09-01
WO2011149577A2 (en) 2011-12-01
RU2012141574A (en) 2014-04-10
KR20130058665A (en) 2013-06-04

Similar Documents

Publication Publication Date Title
EP2543187A4 (en) Systems and methods for spatially controlled scene illumination
EP2801243B8 (en) Systems and methods for providing high-mast lighting
EP2654576B8 (en) Adjustable anchor systems
EP2636035A4 (en) Systems and methods for controlling solid state lighting devices and lighting apparatus incorporating such systems and/or methods
EP2444716B8 (en) Lighting apparatus
EP2559937A4 (en) Illumination device and illumination system
EP2619742A4 (en) Systems and methods for vslam optimization
EP2630314A4 (en) Motorizable shade system and method
EP2716137A4 (en) Systems and methods for controlling solid state lighting devices and lighting apparatus incorporating such systems and/or methods
EP2547175A4 (en) Illumination system
EP2578924B8 (en) Lighting apparatus
ZA201207964B (en) Illumination apparatus methods and systems
EP2583214A4 (en) Systems and methods for wellbore optimization
EP2616278B8 (en) Lighting system
EP2523570A4 (en) Carrying system
EP2633450A4 (en) Systems and methods for scheduling changes
PL2375870T3 (en) Light control method and system
EP2649492A4 (en) 3d conference system
EP2758245A4 (en) Image forming system and methods thereof
AU2010904287A0 (en) Lighting apparatus and systems
AU2010901746A0 (en) Illumination Apparatus Methods and Systems
AU2010904257A0 (en) Downlight systems and methods
AU2010902425A0 (en) Downlight Systems and Methods
AU2011903943A0 (en) Lighting apparatus and systems
AU2011902322A0 (en) Downlight systems and methods

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20120927

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: EYEFLUENCE, INC.

A4 Supplementary search report drawn up and despatched

Effective date: 20171120

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: GOOGLE LLC

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 5/235 20060101ALI20171114BHEP

Ipc: H04N 7/18 20060101AFI20171114BHEP

Ipc: H04N 5/232 20060101ALI20171114BHEP

Ipc: H04N 5/225 20060101ALI20171114BHEP

Ipc: H04N 5/33 20060101ALI20171114BHEP

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20190516

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20211026

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

INTC Intention to grant announced (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20220614

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602011073451

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1532488

Country of ref document: AT

Kind code of ref document: T

Effective date: 20221215

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20221116

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1532488

Country of ref document: AT

Kind code of ref document: T

Effective date: 20221116

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230316

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230216

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230316

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230217

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230327

Year of fee payment: 13

Ref country code: DE

Payment date: 20230329

Year of fee payment: 13

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230505

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602011073451

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20230817

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221116

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20230331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230301

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230331

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230301

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230331

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230331