JP7423659B2 - 眼姿勢を推定するためのシステムおよび技法 - Google Patents

眼姿勢を推定するためのシステムおよび技法 Download PDF

Info

Publication number
JP7423659B2
JP7423659B2 JP2021568889A JP2021568889A JP7423659B2 JP 7423659 B2 JP7423659 B2 JP 7423659B2 JP 2021568889 A JP2021568889 A JP 2021568889A JP 2021568889 A JP2021568889 A JP 2021568889A JP 7423659 B2 JP7423659 B2 JP 7423659B2
Authority
JP
Japan
Prior art keywords
user
eye
flash
image
locations
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2021568889A
Other languages
English (en)
Japanese (ja)
Other versions
JP2022533205A (ja
Inventor
ダニエル ファーマー，
デイビッド コーエン，
ブラッドリー ヴィンセント スチュアート，
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Magic Leap Inc
Original Assignee
Magic Leap Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Magic Leap Inc filed Critical Magic Leap Inc
Publication of JP2022533205A publication Critical patent/JP2022533205A/ja
Priority to JP2023188490A priority Critical patent/JP7570487B2/ja
Application granted granted Critical
Publication of JP7423659B2 publication Critical patent/JP7423659B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/77—Determining position or orientation of objects or cameras using statistical methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10141—Special mode during image acquisition
    • G06T2207/10152—Varying illumination
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30041—Eye; Retina; Ophthalmic
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • User Interface Of Digital Computer (AREA)
  • Eye Examination Apparatus (AREA)
  • Image Analysis (AREA)
JP2021568889A 2019-05-20 2020-05-19 眼姿勢を推定するためのシステムおよび技法 Active JP7423659B2 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2023188490A JP7570487B2 (ja) 2019-05-20 2023-11-02 眼姿勢を推定するためのシステムおよび技法

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201962850539P 2019-05-20 2019-05-20
US62/850,539 2019-05-20
PCT/US2020/033620 WO2020236827A1 (en) 2019-05-20 2020-05-19 Systems and techniques for estimating eye pose

Related Child Applications (1)

Application Number Title Priority Date Filing Date
JP2023188490A Division JP7570487B2 (ja) 2019-05-20 2023-11-02 眼姿勢を推定するためのシステムおよび技法

Publications (2)

Publication Number Publication Date
JP2022533205A JP2022533205A (ja) 2022-07-21
JP7423659B2 true JP7423659B2 (ja) 2024-01-29

Family

ID=73457064

Family Applications (2)

Application Number Title Priority Date Filing Date
JP2021568889A Active JP7423659B2 (ja) 2019-05-20 2020-05-19 眼姿勢を推定するためのシステムおよび技法
JP2023188490A Active JP7570487B2 (ja) 2019-05-20 2023-11-02 眼姿勢を推定するためのシステムおよび技法

Family Applications After (1)

Application Number Title Priority Date Filing Date
JP2023188490A Active JP7570487B2 (ja) 2019-05-20 2023-11-02 眼姿勢を推定するためのシステムおよび技法

Country Status (5)

Country Link
US (5) US11315288B2 (https=)
EP (1) EP3973347B1 (https=)
JP (2) JP7423659B2 (https=)
CN (1) CN114127610A (https=)
WO (1) WO2020236827A1 (https=)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11944481B1 (en) * 2018-02-23 2024-04-02 Robert Edwin Douglas Method and apparatus for predicting an image analysis tool based on user behavior
EP3973347B1 (en) 2019-05-20 2025-03-26 Magic Leap, Inc. Systems and techniques for estimating eye pose
US11315326B2 (en) * 2019-10-15 2022-04-26 At&T Intellectual Property I, L.P. Extended reality anchor caching based on viewport prediction
US11205069B1 (en) * 2020-02-20 2021-12-21 Facebook Technologies, Llc Hybrid cornea and pupil tracking
GB2594966B (en) * 2020-05-13 2024-07-31 Sony Interactive Entertainment Inc Gaze tracking apparatus and systems
WO2021247435A1 (en) * 2020-06-05 2021-12-09 Magic Leap, Inc. Enhanced eye tracking techniques based on neural network analysis of images
WO2022015847A1 (en) 2020-07-15 2022-01-20 Magic Leap, Inc. Eye tracking using aspheric cornea model
US12073018B2 (en) * 2020-09-22 2024-08-27 Apple Inc. Multiple gaze dependent illumination sources for retinal eye tracking
US20220198831A1 (en) * 2020-12-17 2022-06-23 Delphinium Clinic Ltd. System for determining one or more characteristics of a user based on an image of their eye using an ar/vr headset
CN113177434A (zh) * 2021-03-30 2021-07-27 青岛小鸟看看科技有限公司 基于单眼球追踪的虚拟现实系统注视渲染方法、系统
ES3058363T3 (es) * 2021-06-30 2026-03-10 Tobii Ab Sistema de seguimiento ocular
EP4113252A1 (en) 2021-07-02 2023-01-04 IniVation AG Eye tracking device, eye tracking method, and computer-readable medium
GB2614326B (en) * 2021-12-31 2024-10-02 Sony Interactive Entertainment Europe Ltd Apparatus and method for virtual reality
WO2023187780A1 (en) * 2022-03-30 2023-10-05 Jerusalem College Of Technology Eye tracking device
CN115171217B (zh) * 2022-07-27 2023-03-03 北京拙河科技有限公司 一种动态背景下的动作识别方法及系统
JP2024034143A (ja) * 2022-08-31 2024-03-13 株式会社ソニー・インタラクティブエンタテインメント 情報処理装置および情報処理方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013535022A (ja) 2010-05-29 2013-09-09 ウェンユー・ジアーン 凝視距離および低電力凝視追跡によって動かされる適応レンズを有する眼鏡を製作し使用するためのシステム、方法および装置
JP2014093704A (ja) 2012-11-06 2014-05-19 Sony Computer Entertainment Inc ヘッドマウントディスプレイ、および動き検出方法
CN103870796A (zh) 2012-12-13 2014-06-18 汉王科技股份有限公司 一种人眼视线估计的方法与装置
WO2016098406A1 (ja) 2014-12-17 2016-06-23 ソニー株式会社 情報処理装置、情報処理方法及びプログラム
JP2017211891A (ja) 2016-05-27 2017-11-30 ソニー株式会社 情報処理装置、情報処理方法および記録媒体
US20190042842A1 (en) 2017-08-04 2019-02-07 Facebook Technologies, Llc Eye tracking using time multiplexing
US20190076015A1 (en) 2017-09-08 2019-03-14 Tobii Ab Eye tracking using eyeball center position

Family Cites Families (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6222525B1 (en) 1992-03-05 2001-04-24 Brad A. Armstrong Image controllers with sheet connected sensors
US5670988A (en) 1995-09-05 1997-09-23 Interlink Electronics, Inc. Trigger operated electronic device
US11428937B2 (en) 2005-10-07 2022-08-30 Percept Technologies Enhanced optical and perceptual digital eyewear
US20070081123A1 (en) 2005-10-07 2007-04-12 Lewis Scott W Digital eyewear
US8696113B2 (en) 2005-10-07 2014-04-15 Percept Technologies Inc. Enhanced optical and perceptual digital eyewear
US9304319B2 (en) 2010-11-18 2016-04-05 Microsoft Technology Licensing, Llc Automatic focus improvement for augmented reality displays
AU2011348122A1 (en) 2010-12-24 2013-07-11 Magic Leap Inc. An ergonomic head mounted display device and optical system
US10156722B2 (en) 2010-12-24 2018-12-18 Magic Leap, Inc. Methods and systems for displaying stereoscopy with a freeform optical system with addressable focus for virtual and augmented reality
JP6316186B2 (ja) 2011-05-06 2018-04-25 マジック リープ， インコーポレイテッドＭａｇｉｃ Ｌｅａｐ，Ｉｎｃ． 広範囲同時遠隔ディジタル提示世界
US9213163B2 (en) * 2011-08-30 2015-12-15 Microsoft Technology Licensing, Llc Aligning inter-pupillary distance in a near-eye display system
US9354445B1 (en) * 2011-09-16 2016-05-31 Google Inc. Information processing on a head-mountable device
EP2760363A4 (en) 2011-09-29 2015-06-24 Magic Leap Inc TACTILE GLOVE FOR HUMAN COMPUTER INTERACTION
US9215293B2 (en) 2011-10-28 2015-12-15 Magic Leap, Inc. System and method for augmented and virtual reality
WO2013152205A1 (en) 2012-04-05 2013-10-10 Augmented Vision Inc. Wide-field of view (fov) imaging devices with active foveation capability
US9671566B2 (en) 2012-06-11 2017-06-06 Magic Leap, Inc. Planar waveguide apparatus with diffraction element(s) and system employing same
KR20150054967A (ko) 2012-09-11 2015-05-20 매직 립, 인코포레이티드 인체공학적 헤드 마운티드 디스플레이 디바이스 및 광학 시스템
US9417452B2 (en) 2013-03-15 2016-08-16 Magic Leap, Inc. Display system and method
US9874749B2 (en) 2013-11-27 2018-01-23 Magic Leap, Inc. Virtual and augmented reality systems and methods
JP6479785B2 (ja) 2013-10-16 2019-03-06 マジック リープ， インコーポレイテッドＭａｇｉｃ Ｌｅａｐ，Ｉｎｃ． 調節可能な瞳孔間距離を有する仮想または拡張現実ヘッドセット
US9857591B2 (en) 2014-05-30 2018-01-02 Magic Leap, Inc. Methods and system for creating focal planes in virtual and augmented reality
CN107272199B (zh) 2013-11-27 2023-04-07 奇跃公司 虚拟和增强现实系统与方法
US10203762B2 (en) 2014-03-11 2019-02-12 Magic Leap, Inc. Methods and systems for creating virtual and augmented reality
EP3140780B1 (en) 2014-05-09 2020-11-04 Google LLC Systems and methods for discerning eye signals and continuous biometric identification
JP2016106668A (ja) * 2014-12-02 2016-06-20 ソニー株式会社 情報処理装置、情報処理方法、およびプログラム
NZ773818A (en) 2015-03-16 2022-07-29 Magic Leap Inc Methods and systems for diagnosing and treating health ailments
CN108141575B (zh) 2015-05-19 2020-10-30 奇跃公司 半全局快门成像器
KR102591552B1 (ko) 2015-08-21 2023-10-18 매직 립, 인코포레이티드 눈 포즈 측정을 사용한 눈꺼풀 형상 추정
EP3337383B1 (en) 2015-08-21 2024-10-16 Magic Leap, Inc. Eyelid shape estimation
EP3353750B1 (en) 2015-09-25 2021-09-01 Magic Leap, Inc. Methods and systems for detecting and combining structural features in 3d reconstruction
KR102701209B1 (ko) 2015-10-20 2024-08-29 매직 립, 인코포레이티드 3차원 공간에서 가상 객체들 선택
JP6631951B2 (ja) * 2015-12-18 2020-01-15 国立大学法人静岡大学 視線検出装置及び視線検出方法
US10296792B2 (en) 2016-07-14 2019-05-21 Magic Leap, Inc. Iris boundary estimation using cornea curvature
KR102269065B1 (ko) 2016-10-05 2021-06-24 매직 립, 인코포레이티드 혼합 현실 교정을 위한 안구주위 테스트
AU2017356847B2 (en) 2016-11-10 2022-11-17 Magic Leap, Inc. Method and system for multiple F-number lens
CN116886835A (zh) 2016-12-23 2023-10-13 奇跃公司 用于确定内容捕获设备的设置的技术
JP6948688B2 (ja) * 2017-02-06 2021-10-13 国立大学法人神戸大学 視線計測装置、視線計測方法および視線計測プログラム
CN110352033B (zh) * 2017-02-27 2023-04-28 托比股份公司 用眼睛追踪装置确定眼睛睁开度
AU2018237067B2 (en) 2017-03-21 2022-11-10 Magic Leap, Inc. Depth sensing techniques for virtual, augmented, and mixed reality systems
IL300511A (en) 2017-03-22 2023-04-01 Magic Leap Inc Depth-based rendering for display systems
AU2018237688B2 (en) 2017-03-24 2022-08-25 Magic Leap, Inc. Accumulation and confidence assignment of iris codes
US9940518B1 (en) 2017-09-11 2018-04-10 Tobii Ab Reliability of gaze tracking data for left and right eye
EP4095913B1 (en) 2017-12-13 2024-09-25 Magic Leap, Inc. Global shutter pixel circuit
IL275824B2 (en) 2018-01-17 2024-08-01 Magic Leap Inc Display systems and methods for determining registration between a display and a user's eyes
KR102902962B1 (ko) 2018-01-17 2025-12-22 매직 립, 인코포레이티드 디스플레이 시스템들에서의 눈 회전 중심 결정, 깊이 평면 선택, 및 렌더 카메라 포지셔닝
US11067805B2 (en) 2018-04-19 2021-07-20 Magic Leap, Inc. Systems and methods for operating a display system based on user perceptibility
EP3914997A4 (en) 2019-01-25 2022-10-12 Magic Leap, Inc. Eye-tracking using images having different exposure times
EP3973347B1 (en) 2019-05-20 2025-03-26 Magic Leap, Inc. Systems and techniques for estimating eye pose

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013535022A (ja) 2010-05-29 2013-09-09 ウェンユー・ジアーン 凝視距離および低電力凝視追跡によって動かされる適応レンズを有する眼鏡を製作し使用するためのシステム、方法および装置
JP2014093704A (ja) 2012-11-06 2014-05-19 Sony Computer Entertainment Inc ヘッドマウントディスプレイ、および動き検出方法
CN103870796A (zh) 2012-12-13 2014-06-18 汉王科技股份有限公司 一种人眼视线估计的方法与装置
WO2016098406A1 (ja) 2014-12-17 2016-06-23 ソニー株式会社 情報処理装置、情報処理方法及びプログラム
JP2017211891A (ja) 2016-05-27 2017-11-30 ソニー株式会社 情報処理装置、情報処理方法および記録媒体
US20190042842A1 (en) 2017-08-04 2019-02-07 Facebook Technologies, Llc Eye tracking using time multiplexing
US20190076015A1 (en) 2017-09-08 2019-03-14 Tobii Ab Eye tracking using eyeball center position

Also Published As

Publication number Publication date
EP3973347B1 (en) 2025-03-26
EP3973347A1 (en) 2022-03-30
JP7570487B2 (ja) 2024-10-21
CN114127610A (zh) 2022-03-01
US11675432B2 (en) 2023-06-13
EP3973347A4 (en) 2023-05-31
US20230305627A1 (en) 2023-09-28
JP2022533205A (ja) 2022-07-21
US12474775B2 (en) 2025-11-18
JP2023181496A (ja) 2023-12-21
US20250371736A1 (en) 2025-12-04
US20240329737A1 (en) 2024-10-03
US20220237823A1 (en) 2022-07-28
US12050727B2 (en) 2024-07-30
US20200372678A1 (en) 2020-11-26
US11315288B2 (en) 2022-04-26
WO2020236827A1 (en) 2020-11-26

Similar Documents

Publication Publication Date Title
JP7431246B2 (ja) 異なる露光時間を有する画像を使用した眼追跡
JP7423659B2 (ja) 眼姿勢を推定するためのシステムおよび技法
JP7710586B2 (ja) ディスプレイとユーザの眼との間の位置合わせを決定するためのディスプレイシステムおよび方法
JP7415077B2 (ja) ディスプレイとユーザの眼との間の位置合わせを決定するためのディスプレイシステムおよび方法
JP7443332B2 (ja) ユーザカテゴリ化による多深度平面ディスプレイシステムのための深度平面選択
JP7838038B2 (ja) １つ以上の眼追跡カメラを用いた眼回転中心の決定
JP7456995B2 (ja) 左および右ディスプレイとユーザの眼との間の垂直整合を決定するためのディスプレイシステムおよび方法

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20230518

A871 Explanation of circumstances concerning accelerated examination

Free format text: JAPANESE INTERMEDIATE CODE: A871

Effective date: 20230518

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20230802

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20231102

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20240116

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20240117

R150 Certificate of patent or registration of utility model

Ref document number: 7423659

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150