US11867900B2 - Bright pupil eye-tracking system - Google Patents
Bright pupil eye-tracking system Download PDFInfo
- Publication number
- US11867900B2 US11867900B2 US16/844,890 US202016844890A US11867900B2 US 11867900 B2 US11867900 B2 US 11867900B2 US 202016844890 A US202016844890 A US 202016844890A US 11867900 B2 US11867900 B2 US 11867900B2
- Authority
- US
- United States
- Prior art keywords
- eye
- image
- light sources
- light source
- light
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
- 210000001747 pupil Anatomy 0.000 title claims abstract description 110
- 238000000034 method Methods 0.000 claims abstract description 51
- 230000004044 response Effects 0.000 claims abstract description 15
- 230000003287 optical effect Effects 0.000 claims description 45
- 238000007493 shaping process Methods 0.000 claims description 16
- 238000001514 detection method Methods 0.000 claims description 13
- 239000000758 substrate Substances 0.000 claims description 5
- 230000008569 process Effects 0.000 description 36
- 230000015654 memory Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 4
- 230000003190 augmentative effect Effects 0.000 description 3
- 210000003128 head Anatomy 0.000 description 3
- 230000010287 polarization Effects 0.000 description 3
- 238000013500 data storage Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 239000012528 membrane Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 230000010339 dilation Effects 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001179 pupillary effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
- G06T7/0014—Biomedical image inspection using an image reference approach
- G06T7/0016—Biomedical image inspection using an image reference approach involving temporal comparison
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/19—Sensors therefor
-
- H—ELECTRICITY
- H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
- H05B—ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL
- H05B47/00—Circuit arrangements for operating light sources in general, i.e. where the type of light source is not relevant
- H05B47/10—Controlling the light source
- H05B47/105—Controlling the light source in response to determined parameters
- H05B47/115—Controlling the light source in response to determined parameters by determining the presence or movement of objects or living beings
- H05B47/13—Controlling the light source in response to determined parameters by determining the presence or movement of objects or living beings by using passive infrared detectors
-
- H—ELECTRICITY
- H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
- H05B—ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL
- H05B47/00—Circuit arrangements for operating light sources in general, i.e. where the type of light source is not relevant
- H05B47/10—Controlling the light source
- H05B47/155—Coordinated control of two or more light sources
-
- H—ELECTRICITY
- H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
- H05B—ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL
- H05B47/00—Circuit arrangements for operating light sources in general, i.e. where the type of light source is not relevant
- H05B47/10—Controlling the light source
- H05B47/165—Controlling the light source following a pre-assigned programmed sequence; Logic control [LC]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30041—Eye; Retina; Ophthalmic
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02B—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS
- Y02B20/00—Energy efficient lighting technologies, e.g. halogen lamps or gas discharge lamps
- Y02B20/40—Control techniques providing energy savings, e.g. smart controller or presence detection
Abstract
Description
Claims (20)
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/844,890 US11867900B2 (en) | 2020-02-28 | 2020-04-09 | Bright pupil eye-tracking system |
JP2022534692A JP2023514474A (en) | 2020-02-28 | 2021-01-25 | Bright pupil eye tracking system |
PCT/US2021/014970 WO2021173276A1 (en) | 2020-02-28 | 2021-01-25 | Bright pupil eye-tracking system |
CN202180010061.4A CN115053164A (en) | 2020-02-28 | 2021-01-25 | Bright pupil eye tracking system |
EP21707455.8A EP4111251A1 (en) | 2020-02-28 | 2021-01-25 | Bright pupil eye-tracking system |
KR1020227022207A KR20220143640A (en) | 2020-02-28 | 2021-01-25 | Bright pupil eye tracking system |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US202062983352P | 2020-02-28 | 2020-02-28 | |
US16/844,890 US11867900B2 (en) | 2020-02-28 | 2020-04-09 | Bright pupil eye-tracking system |
Publications (2)
Publication Number | Publication Date |
---|---|
US20210271317A1 US20210271317A1 (en) | 2021-09-02 |
US11867900B2 true US11867900B2 (en) | 2024-01-09 |
Family
ID=77463762
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/844,890 Active 2040-08-16 US11867900B2 (en) | 2020-02-28 | 2020-04-09 | Bright pupil eye-tracking system |
Country Status (6)
Country | Link |
---|---|
US (1) | US11867900B2 (en) |
EP (1) | EP4111251A1 (en) |
JP (1) | JP2023514474A (en) |
KR (1) | KR20220143640A (en) |
CN (1) | CN115053164A (en) |
WO (1) | WO2021173276A1 (en) |
Citations (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5815741A (en) | 1994-12-21 | 1998-09-29 | Canon Kabushiki Kaisha | Image observing apparatus and imaging apparatus utilizing the image observing apparatus |
US6027216A (en) * | 1997-10-21 | 2000-02-22 | The Johns University School Of Medicine | Eye fixation monitor and tracker |
US20080100803A1 (en) | 1999-12-03 | 2008-05-01 | Manfred Dick | Method for determining vision defects and for collecting data for correcting vision defects of the eye by interaction of a patient with an examiner and apparatus therefor |
US20130286053A1 (en) * | 2012-04-25 | 2013-10-31 | Rod G. Fleck | Direct view augmented reality eyeglass-type display |
US8596786B2 (en) | 2008-09-30 | 2013-12-03 | Carl Zeiss Meditec Ag | Arrangements and method for measuring an eye movement, particularly a movement of the fundus of the eye |
WO2016162822A1 (en) | 2015-04-08 | 2016-10-13 | Controlrad Systems Inc. | Devices and methods for monitoring gaze |
EP3079560A1 (en) | 2013-12-09 | 2016-10-19 | SensoMotoric Instruments Gesellschaft für innovative Sensorik mbH | Method for operating an eye tracking device and eye tracking device for providing an active illumination control for improved eye tracking robustness |
US20170038836A1 (en) * | 2015-08-03 | 2017-02-09 | Oculus Vr, Llc | Display with an Embedded Eye Tracker |
US20180070819A1 (en) | 2015-11-05 | 2018-03-15 | Panasonic Intellectual Property Management Co., Ltd. | Camera system, feeding system, imaging method, and imaging device |
US20180129167A1 (en) | 2016-11-04 | 2018-05-10 | Microsoft Technology Licensing, Llc | Adjustable scanned beam projector |
US20190019023A1 (en) | 2016-12-01 | 2019-01-17 | Varjo Technologies Oy | Gaze-tracking system and method |
US20190046031A1 (en) | 2017-08-14 | 2019-02-14 | Verily Life Sciences Llc | Dynamic illumination during retinal burst imaging |
US20190101767A1 (en) | 2017-10-03 | 2019-04-04 | Facebook Technologies, Llc | Fresnel assembly for light redirection in eye tracking systems |
US10268268B1 (en) | 2016-09-02 | 2019-04-23 | Facebook Technologies, Llc | Waveguide integrated eye tracking |
US20190250707A1 (en) * | 2018-02-09 | 2019-08-15 | Ricoh Company, Ltd. | Eye-gaze tracker, eye-gaze tracking method, and recording medium |
US20190282089A1 (en) | 2018-03-13 | 2019-09-19 | Welch Allyn, Inc. | Selective Illumination Fundus Imaging |
US20190361250A1 (en) | 2017-12-18 | 2019-11-28 | Facebook Technologies, Llc | Eye tracking for pupil steering in head-mounted displays using eye tracking sensors |
US20210165207A1 (en) | 2011-06-21 | 2021-06-03 | Gholam A. Peyman | Fluidic Glasses For Correcting Refractive Errors Of A Human Or Animal |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9361519B2 (en) * | 2014-03-28 | 2016-06-07 | Intel Corporation | Computational array camera with dynamic illumination for eye tracking |
-
2020
- 2020-04-09 US US16/844,890 patent/US11867900B2/en active Active
-
2021
- 2021-01-25 KR KR1020227022207A patent/KR20220143640A/en active Search and Examination
- 2021-01-25 JP JP2022534692A patent/JP2023514474A/en active Pending
- 2021-01-25 WO PCT/US2021/014970 patent/WO2021173276A1/en unknown
- 2021-01-25 CN CN202180010061.4A patent/CN115053164A/en active Pending
- 2021-01-25 EP EP21707455.8A patent/EP4111251A1/en active Pending
Patent Citations (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5815741A (en) | 1994-12-21 | 1998-09-29 | Canon Kabushiki Kaisha | Image observing apparatus and imaging apparatus utilizing the image observing apparatus |
US6027216A (en) * | 1997-10-21 | 2000-02-22 | The Johns University School Of Medicine | Eye fixation monitor and tracker |
US20080100803A1 (en) | 1999-12-03 | 2008-05-01 | Manfred Dick | Method for determining vision defects and for collecting data for correcting vision defects of the eye by interaction of a patient with an examiner and apparatus therefor |
US8596786B2 (en) | 2008-09-30 | 2013-12-03 | Carl Zeiss Meditec Ag | Arrangements and method for measuring an eye movement, particularly a movement of the fundus of the eye |
US20210165207A1 (en) | 2011-06-21 | 2021-06-03 | Gholam A. Peyman | Fluidic Glasses For Correcting Refractive Errors Of A Human Or Animal |
US20130286053A1 (en) * | 2012-04-25 | 2013-10-31 | Rod G. Fleck | Direct view augmented reality eyeglass-type display |
EP3079560A1 (en) | 2013-12-09 | 2016-10-19 | SensoMotoric Instruments Gesellschaft für innovative Sensorik mbH | Method for operating an eye tracking device and eye tracking device for providing an active illumination control for improved eye tracking robustness |
WO2016162822A1 (en) | 2015-04-08 | 2016-10-13 | Controlrad Systems Inc. | Devices and methods for monitoring gaze |
US10274730B2 (en) | 2015-08-03 | 2019-04-30 | Facebook Technologies, Llc | Display with an embedded eye tracker |
US20170038836A1 (en) * | 2015-08-03 | 2017-02-09 | Oculus Vr, Llc | Display with an Embedded Eye Tracker |
US20180070819A1 (en) | 2015-11-05 | 2018-03-15 | Panasonic Intellectual Property Management Co., Ltd. | Camera system, feeding system, imaging method, and imaging device |
US10268268B1 (en) | 2016-09-02 | 2019-04-23 | Facebook Technologies, Llc | Waveguide integrated eye tracking |
US20180129167A1 (en) | 2016-11-04 | 2018-05-10 | Microsoft Technology Licensing, Llc | Adjustable scanned beam projector |
US20190019023A1 (en) | 2016-12-01 | 2019-01-17 | Varjo Technologies Oy | Gaze-tracking system and method |
US20190046031A1 (en) | 2017-08-14 | 2019-02-14 | Verily Life Sciences Llc | Dynamic illumination during retinal burst imaging |
US20190101767A1 (en) | 2017-10-03 | 2019-04-04 | Facebook Technologies, Llc | Fresnel assembly for light redirection in eye tracking systems |
US20190361250A1 (en) | 2017-12-18 | 2019-11-28 | Facebook Technologies, Llc | Eye tracking for pupil steering in head-mounted displays using eye tracking sensors |
US20190250707A1 (en) * | 2018-02-09 | 2019-08-15 | Ricoh Company, Ltd. | Eye-gaze tracker, eye-gaze tracking method, and recording medium |
US20190282089A1 (en) | 2018-03-13 | 2019-09-19 | Welch Allyn, Inc. | Selective Illumination Fundus Imaging |
Non-Patent Citations (8)
Title |
---|
International Preliminary Report on Patentability for International Application No. PCT/US2020/063513, dated Aug. 18, 2022, 8 pages. |
International Preliminary Report on Patentability for International Application No. PCT/US2021/014970, dated Sep. 9, 2022, 7 pages. |
International Searching Authority, Patent Cooperation Treaty, European Application No. PCT/US2021/014970, dated Apr. 15, 2021, 3 pages. |
International Searching Authority, Patent Cooperation Treaty, European Application No. PCT/US2021/019912, dated Jun. 18, 2021, 9 pages. |
International Searching Authority, Patent Cooperation Treaty, Written Opinion of the International Searching Authority, European Application No. PCT/US2020/063513, dated Feb. 18, 2021, 9 pages. |
International Searching Authority, Patent Cooperation Treaty, Written Opinion of the International Searching Authority, European Application No. PCT/US2021/014693, dated Mar. 12, 2021, 12 pages. |
International Searching Authority, Patent Cooperation Treaty, Written Opinion of the International Searching Authority, European Application No. PCT/US2021/014970, dated Apr. 15, 2021, 5 pages. |
Non-Final Office Action dated Apr. 14, 2022 for U.S. Appl. No. 16/844,877, filed Apr. 9, 2020, 13 pages. |
Also Published As
Publication number | Publication date |
---|---|
KR20220143640A (en) | 2022-10-25 |
EP4111251A1 (en) | 2023-01-04 |
US20210271317A1 (en) | 2021-09-02 |
CN115053164A (en) | 2022-09-13 |
JP2023514474A (en) | 2023-04-06 |
WO2021173276A1 (en) | 2021-09-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10606071B1 (en) | Lightfield waveguide integrated eye tracking | |
US10948729B2 (en) | Keep-out zone for in-field light sources of a head mounted display | |
US20210208397A1 (en) | Foveated display system | |
US10877268B2 (en) | Active control of in-field light sources of a head mounted display | |
US20210208389A1 (en) | Switchable pancharatnam-berry phase grating stack | |
US20230210367A1 (en) | Eye-tracking fundus imaging system | |
US20230087535A1 (en) | Wavefront sensing from retina-reflected light | |
US20240061246A1 (en) | Light field directional backlighting based three-dimensional (3d) pupil steering | |
US20230280468A1 (en) | Addressable projector for dot based direct time of flight depth sensing | |
US11867900B2 (en) | Bright pupil eye-tracking system | |
US11205069B1 (en) | Hybrid cornea and pupil tracking | |
US11307415B1 (en) | Head mounted display with active optics feedback and calibration | |
US20210244273A1 (en) | Eye-tracking fundus illumination system | |
US20230168506A1 (en) | High efficiency optical assembly with folded optical path | |
US11709364B1 (en) | Addressable crossed line projector for depth camera assembly | |
US11848542B1 (en) | Active optics feedback and calibration | |
US20230314596A1 (en) | Ear-region imaging | |
US10991343B2 (en) | Automatic image alignment with head mounted display optics | |
US20240069347A1 (en) | System and method using eye tracking illumination | |
WO2023167888A1 (en) | Addressable projector for dot based direct time of flight depth sensing |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
AS | Assignment |
Owner name: FACEBOOK TECHNOLOGIES, LLC, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:SHARMA, ROBIN;EL-HADDAD, MOHAMED TAREK;SIGNING DATES FROM 20200424 TO 20200501;REEL/FRAME:052585/0566 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
AS | Assignment |
Owner name: META PLATFORMS TECHNOLOGIES, LLC, CALIFORNIA Free format text: CHANGE OF NAME;ASSIGNOR:FACEBOOK TECHNOLOGIES, LLC;REEL/FRAME:060246/0845 Effective date: 20220318 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: AWAITING TC RESP, ISSUE FEE PAYMENT VERIFIED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |