CA2943426C - Handling glare in eye tracking - Google Patents

Handling glare in eye tracking Download PDF

Info

Publication number
CA2943426C
CA2943426C CA2943426A CA2943426A CA2943426C CA 2943426 C CA2943426 C CA 2943426C CA 2943426 A CA2943426 A CA 2943426A CA 2943426 A CA2943426 A CA 2943426A CA 2943426 C CA2943426 C CA 2943426C
Authority
CA
Canada
Prior art keywords
light sources
image
eye tracking
light
saturated
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CA2943426A
Other languages
English (en)
French (fr)
Other versions
CA2943426A1 (en
Inventor
Mudit Agrawal
Vaibhav Thukral
Ibrahim Eden
David Nister
Shivkumar SWAMINATHAN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Technology Licensing LLC
Original Assignee
Microsoft Technology Licensing LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Technology Licensing LLC filed Critical Microsoft Technology Licensing LLC
Publication of CA2943426A1 publication Critical patent/CA2943426A1/en
Application granted granted Critical
Publication of CA2943426C publication Critical patent/CA2943426C/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/002—Specific input/output arrangements not covered by G06F3/01 - G06F3/16
    • G06F3/005—Input arrangements through a video camera
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/60—Extraction of image or video features relating to illumination properties, e.g. using a reflectance or lighting model
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/19—Sensors therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/11—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths for generating image signals from visible and infrared light wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/20—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from infrared radiation only
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/30—Transforming light or analogous information into electric information
    • H04N5/33—Transforming infrared radiation
    • F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
    • F21—LIGHTING
    • F21Y—INDEXING SCHEME ASSOCIATED WITH SUBCLASSES F21K, F21L, F21S and F21V, RELATING TO THE FORM OR THE KIND OF THE LIGHT SOURCES OR OF THE COLOUR OF THE LIGHT EMITTED
    • F21Y2115/00—Light-generating elements of semiconductor light sources
    • F21Y2115/10—Light-emitting diodes [LED]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • General Health & Medical Sciences (AREA)
  • Ophthalmology & Optometry (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Veterinary Medicine (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Image Analysis (AREA)
  • Eye Examination Apparatus (AREA)
  • User Interface Of Digital Computer (AREA)
  • Studio Devices (AREA)
  • Image Input (AREA)
  • Position Input By Displaying (AREA)
CA2943426A 2014-04-29 2015-04-23 Handling glare in eye tracking Active CA2943426C (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US14/264,952 US9454699B2 (en) 2014-04-29 2014-04-29 Handling glare in eye tracking
US14/264,952 2014-04-29
PCT/US2015/027185 WO2015167906A1 (en) 2014-04-29 2015-04-23 Handling glare in eye tracking

Publications (2)

Publication Number Publication Date
CA2943426A1 CA2943426A1 (en) 2015-11-05
CA2943426C true CA2943426C (en) 2021-11-09

Family

ID=53059471

Family Applications (1)

Application Number Title Priority Date Filing Date
CA2943426A Active CA2943426C (en) 2014-04-29 2015-04-23 Handling glare in eye tracking

Country Status (18)

Country Link
US (2) US9454699B2 (enExample)
EP (1) EP3137973B1 (enExample)
JP (2) JP6592000B2 (enExample)
KR (1) KR102358936B1 (enExample)
CN (2) CN106462236B (enExample)
AU (1) AU2015253554B2 (enExample)
BR (1) BR112016024889B1 (enExample)
CA (1) CA2943426C (enExample)
CL (1) CL2016002668A1 (enExample)
IL (1) IL247738B (enExample)
MX (1) MX366780B (enExample)
MY (1) MY184112A (enExample)
NZ (1) NZ724321A (enExample)
PH (1) PH12016501886A1 (enExample)
RU (1) RU2678478C2 (enExample)
SG (1) SG11201608289YA (enExample)
TW (1) TW201541284A (enExample)
WO (1) WO2015167906A1 (enExample)

Families Citing this family (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9454699B2 (en) * 2014-04-29 2016-09-27 Microsoft Technology Licensing, Llc Handling glare in eye tracking
US9946339B2 (en) * 2014-10-08 2018-04-17 Microsoft Technology Licensing, Llc Gaze tracking through eyewear
US10444972B2 (en) 2015-11-28 2019-10-15 International Business Machines Corporation Assisting a user with efficient navigation between a selection of entries with elements of interest to the user within a stream of entries
US10878237B2 (en) * 2016-06-29 2020-12-29 Seeing Machines Limited Systems and methods for performing eye gaze tracking
CN110023946A (zh) * 2016-07-05 2019-07-16 吴业成 在现场图像捕捉期间的欺骗攻击检测
US20180081434A1 (en) * 2016-09-22 2018-03-22 Apple Inc. Eye and Head Tracking
US10120442B2 (en) * 2016-12-21 2018-11-06 Oculus Vr, Llc Eye tracking using a light field camera on a head-mounted display
CN106874895B (zh) 2017-03-31 2019-02-05 北京七鑫易维信息技术有限公司 一种视线追踪装置及头戴式显示设备
US10657401B2 (en) * 2017-06-06 2020-05-19 Microsoft Technology Licensing, Llc Biometric object spoof detection based on image intensity variations
WO2019014861A1 (en) * 2017-07-18 2019-01-24 Hangzhou Taruo Information Technology Co., Ltd. INTELLIGENT FOLLOWING OF OBJECTS
US10474231B2 (en) 2017-08-16 2019-11-12 Industrial Technology Research Institute Eye tracking apparatus and method thereof
US11567318B1 (en) * 2017-09-25 2023-01-31 Meta Platforms Technologies, Llc Determining features of a user's eye from depth mapping of the user's eye via indirect time of flight
KR102410834B1 (ko) * 2017-10-27 2022-06-20 삼성전자주식회사 반사 영역을 제거하는 방법, 사용자의 눈을 추적하는 방법 및 그 장치
DE112018005191T5 (de) * 2017-10-31 2020-06-18 Seeing Machines Limited System und Verfahren zur Verbesserung des Signal-Rausch-Verhältnisses bei der Objektverfolgung unter schlechten Lichtbedingungen
US20190129174A1 (en) * 2017-10-31 2019-05-02 Google Llc Multi-perspective eye-tracking for vr/ar systems
KR102476757B1 (ko) 2017-12-21 2022-12-09 삼성전자주식회사 반사를 검출하는 장치 및 방법
TWI647472B (zh) * 2018-01-22 2019-01-11 國立臺灣大學 雙模式視線追蹤方法與系統
CN108510542B (zh) * 2018-02-12 2020-09-11 北京七鑫易维信息技术有限公司 匹配光源与光斑的方法和装置
US10863812B2 (en) * 2018-07-18 2020-12-15 L'oreal Makeup compact with eye tracking for guidance of makeup application
US12318164B2 (en) 2018-08-01 2025-06-03 Intuitive Surgical Operations, Inc. Systems and methods for controlling a robotic manipulator or associated tool
US11483545B2 (en) 2018-08-29 2022-10-25 Intuitive Surgical Operations, Inc. Dynamic illumination for eye-tracking
CN113316411B (zh) * 2018-09-19 2025-01-17 艾维德洛公司 用于眼部治疗期间的眼部追踪的系统及方法
US11455716B2 (en) * 2018-11-13 2022-09-27 Rivian Ip Holdings, Llc Image analysis of applied adhesive with fluorescence enhancement
SE543240C2 (en) * 2018-12-21 2020-10-27 Tobii Ab Classification of glints using an eye tracking system
DE102018010099B4 (de) * 2018-12-21 2021-12-02 Diehl Aerospace Gmbh Vermeiden der Blendung von Personen durch eine Lichtquelle
US10948729B2 (en) 2019-04-16 2021-03-16 Facebook Technologies, Llc Keep-out zone for in-field light sources of a head mounted display
US10877268B2 (en) * 2019-04-16 2020-12-29 Facebook Technologies, Llc Active control of in-field light sources of a head mounted display
US11516374B2 (en) 2019-06-05 2022-11-29 Synaptics Incorporated Under-display image sensor
IT201900011403A1 (it) * 2019-07-10 2021-01-10 Ambarella Int Lp Detecting illegal use of phone to prevent the driver from getting a fine
TWI751647B (zh) * 2019-08-13 2022-01-01 宏達國際電子股份有限公司 頭戴式顯示器
US11153513B2 (en) 2019-08-19 2021-10-19 Synaptics Incorporated Light source for camera
CN112578556B (zh) * 2019-09-27 2023-02-21 托比股份公司 用于减少来自光学装置的不合需的反射的眼睛跟踪系统
US11076080B2 (en) 2019-12-05 2021-07-27 Synaptics Incorporated Under-display image sensor for eye tracking
WO2021125992A1 (ru) * 2019-12-16 2021-06-24 федеральное государственное автономное образовательное учреждение высшего образования "Московский физико-технический институт (национальный исследовательский университет)" Устройство определения направления взгляда
US11340701B2 (en) * 2019-12-16 2022-05-24 Nvidia Corporation Gaze determination using glare as input
WO2021125993A1 (ru) * 2019-12-16 2021-06-24 Федеральное Государственное Автономное Образовательное Учреждение Высшего Образования "Московский Физико-Технический Инстиут (Национальный Исследовательский Университет)" Способ определения направления взгляда
CN113515975B (zh) * 2020-04-10 2022-11-08 北京眼神科技有限公司 人脸和虹膜图像采集方法、装置、可读存储介质及设备
CN111772575A (zh) * 2020-06-02 2020-10-16 岭南师范学院 基于led光学无损特殊儿童检测仪及检测方法
CN112198965B (zh) * 2020-12-04 2021-03-09 宁波圻亿科技有限公司 一种ar眼镜护目自动控制方法及装置
JP6956985B1 (ja) * 2020-12-22 2021-11-02 株式会社スワローインキュベート 目検出方法、目検出装置及び目検出プログラム
US11972042B2 (en) * 2021-06-15 2024-04-30 Microsoft Technology Licensing, Llc Variable intensity distributions for gaze detection assembly
JP2023041097A (ja) * 2021-09-13 2023-03-24 矢崎総業株式会社 検出システム
CN116301301A (zh) * 2021-12-20 2023-06-23 华为技术有限公司 眼动追踪装置和眼动追踪方法
GB2618078A (en) * 2022-04-25 2023-11-01 Continental Automotive Tech Gmbh An image processing method for removal of glare spots and system for the same

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6714665B1 (en) 1994-09-02 2004-03-30 Sarnoff Corporation Fully automated iris recognition system utilizing wide and narrow fields of view
US6152563A (en) * 1998-02-20 2000-11-28 Hutchinson; Thomas E. Eye gaze direction tracker
US7336821B2 (en) 2006-02-14 2008-02-26 Fotonation Vision Limited Automatic detection and correction of non-red eye flash defects
JP4250506B2 (ja) * 2003-10-31 2009-04-08 キヤノン株式会社 画像処理方法、画像処理装置、画像処理プログラムおよび撮像システム
US7777778B2 (en) 2004-10-27 2010-08-17 Delphi Technologies, Inc. Illumination and imaging system and method
JP2006318374A (ja) 2005-05-16 2006-11-24 Matsushita Electric Ind Co Ltd 眼鏡判別装置および認証装置ならびに眼鏡判別方法
JP4921486B2 (ja) * 2005-11-30 2012-04-25 シーイング・マシーンズ・プロプライエタリー・リミテッド 視覚的に頭と目を追跡するシステムにおける眼鏡の視覚的追跡
US7630002B2 (en) 2007-01-05 2009-12-08 Microsoft Corporation Specular reflection reduction using multiple cameras
CN101336089A (zh) * 2006-01-26 2008-12-31 诺基亚公司 眼睛跟踪器设备
CN101008982A (zh) * 2006-01-27 2007-08-01 侯雨石 消除眼镜干扰的虹膜采集方法和装置
CN101499127B (zh) * 2008-02-03 2012-07-18 上海银晨智能识别科技有限公司 防止眼镜镜片反光干扰而导致无法识别人脸的方法
US20100079508A1 (en) 2008-09-30 2010-04-01 Andrew Hodge Electronic devices with gaze detection capabilities
US20120105486A1 (en) 2009-04-09 2012-05-03 Dynavox Systems Llc Calibration free, motion tolerent eye-gaze direction detector with contextually aware computer interaction and communication methods
US8324602B2 (en) 2009-04-14 2012-12-04 Intersil Americas Inc. Optical sensors that reduce specular reflections
CN101545588A (zh) * 2009-04-30 2009-09-30 清华大学 一种防止眼镜反光的机器视觉照明系统
US8333475B2 (en) * 2009-11-12 2012-12-18 Panasonic Corporation Electro-oculography measuring device, ophthalmological diagnosis device, eye-gaze tracking device, wearable camera, head-mounted display, electronic eyeglasses, electro-oculography measuring method, and recording medium
EP2339534A1 (en) 2009-11-18 2011-06-29 Panasonic Corporation Specular reflection compensation
RU93553U1 (ru) * 2009-11-19 2010-04-27 Общество с ограниченной ответственностью "Системы позиционирования - оптические технологии" Компьютерный манипулятор, использующий углы поворота глаз и направление взгляда человека для позиционирования курсора на экране компьютера
US20110170060A1 (en) 2010-01-08 2011-07-14 Gordon Gary B Gaze Tracking Using Polarized Light
RU2576344C2 (ru) 2010-05-29 2016-02-27 Вэньюй ЦЗЯН Системы, способы и аппараты для создания и использования очков с адаптивной линзой на основе определения расстояния наблюдения и отслеживания взгляда в условиях низкого энергопотребления
JP5336665B2 (ja) 2010-10-07 2013-11-06 株式会社ソニー・コンピュータエンタテインメント ３−ｄメガネと、カメラベースの頭部トラッキング
US8730356B2 (en) 2011-03-07 2014-05-20 Sony Corporation System and method for automatic flash removal from images
EP2724292A4 (en) 2011-06-18 2015-05-06 Identix Inc SYSTEMS AND METHOD FOR RECOGNIZING A MIRROR REFLECTION MODEL FOR BIOMETRIC ANALYZES
US8929589B2 (en) 2011-11-07 2015-01-06 Eyefluence, Inc. Systems and methods for high-resolution gaze tracking
US8913789B1 (en) * 2012-01-06 2014-12-16 Google Inc. Input methods and systems for eye positioning using plural glints
KR20130121303A (ko) * 2012-04-27 2013-11-06 한국전자통신연구원 원거리 시선 추적 시스템 및 방법
US9192305B2 (en) * 2012-09-28 2015-11-24 Align Technology, Inc. Estimating a surface texture of a tooth
US9829971B2 (en) * 2013-01-21 2017-11-28 Facebook, Inc. Systems and methods of eye tracking control
US20140375541A1 (en) * 2013-06-25 2014-12-25 David Nister Eye tracking via depth camera
KR101847756B1 (ko) * 2013-11-09 2018-04-10 선전 구딕스 테크놀로지 컴퍼니, 리미티드 광학 눈 추적
US9454699B2 (en) * 2014-04-29 2016-09-27 Microsoft Technology Licensing, Llc Handling glare in eye tracking

Also Published As

Publication number Publication date
AU2015253554B2 (en) 2019-12-19
EP3137973A1 (en) 2017-03-08
KR20160146858A (ko) 2016-12-21
TW201541284A (zh) 2015-11-01
SG11201608289YA (en) 2016-11-29
IL247738A0 (en) 2016-11-30
PH12016501886B1 (en) 2016-12-19
BR112016024889A2 (pt) 2017-08-15
BR112016024889A8 (pt) 2021-06-08
KR102358936B1 (ko) 2022-02-04
RU2016142286A3 (enExample) 2018-11-19
US9916502B2 (en) 2018-03-13
MX366780B (es) 2019-07-24
CA2943426A1 (en) 2015-11-05
RU2678478C2 (ru) 2019-01-29
CN106462236B (zh) 2019-06-04
WO2015167906A1 (en) 2015-11-05
CN106462236A (zh) 2017-02-22
JP6592000B2 (ja) 2019-10-16
CL2016002668A1 (es) 2017-04-21
MX2016014094A (es) 2017-02-09
JP6845295B2 (ja) 2021-03-17
AU2015253554A1 (en) 2016-10-20
RU2016142286A (ru) 2018-04-27
NZ724321A (en) 2020-03-27
IL247738B (en) 2019-06-30
BR112016024889B1 (pt) 2022-09-27
US20160358009A1 (en) 2016-12-08
JP2020013601A (ja) 2020-01-23
PH12016501886A1 (en) 2016-12-19
US20150310253A1 (en) 2015-10-29
US9454699B2 (en) 2016-09-27
JP2017521738A (ja) 2017-08-03
CN110266916A (zh) 2019-09-20
CN110266916B (zh) 2021-09-21
EP3137973B1 (en) 2020-03-25
MY184112A (en) 2021-03-18

Similar Documents

Publication Publication Date Title
CA2943426C (en) Handling glare in eye tracking
US10650226B2 (en) False face representation identification
KR102456046B1 (ko) 동공 검출
EP3284011B1 (en) Two-dimensional infrared depth sensing
EP3243161A1 (en) Eye tracking
US20180197047A1 (en) Stereoscopic object detection leveraging expected object distance
HK1234172A1 (en) Handling glare in eye tracking
HK1234172B (zh) 在眼睛追踪中处理眩光

Legal Events

Date Code Title Description
EEER Examination request

Effective date: 20200422