CN114270294B - 使用眩光作为输入的注视确定 - Google Patents

使用眩光作为输入的注视确定 Download PDF

Info

Publication number
CN114270294B
CN114270294B CN202080036232.6A CN202080036232A CN114270294B CN 114270294 B CN114270294 B CN 114270294B CN 202080036232 A CN202080036232 A CN 202080036232A CN 114270294 B CN114270294 B CN 114270294B
Authority
CN
China
Prior art keywords
face
machine learning
input
glare
vehicle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202080036232.6A
Other languages
English (en)
Chinese (zh)
Other versions
CN114270294A (zh
Inventor
N·M·阿拉尔
江海荣
N·普里
N·阿瓦达汗纳姆
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nvidia Corp
Original Assignee
Nvidia Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nvidia Corp filed Critical Nvidia Corp
Publication of CN114270294A publication Critical patent/CN114270294A/zh
Application granted granted Critical
Publication of CN114270294B publication Critical patent/CN114270294B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/19—Sensors therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Ophthalmology & Optometry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
CN202080036232.6A 2019-12-16 2020-12-16 使用眩光作为输入的注视确定 Active CN114270294B (zh)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201962948793P 2019-12-16 2019-12-16
US62/948,793 2019-12-16
US16/902,737 2020-06-16
US16/902,737 US11340701B2 (en) 2019-12-16 2020-06-16 Gaze determination using glare as input
PCT/US2020/065350 WO2021127005A1 (en) 2019-12-16 2020-12-16 Gaze determination using glare as input

Publications (2)

Publication Number Publication Date
CN114270294A CN114270294A (zh) 2022-04-01
CN114270294B true CN114270294B (zh) 2025-05-06

Family

ID=76317901

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202080036232.6A Active CN114270294B (zh) 2019-12-16 2020-12-16 使用眩光作为输入的注视确定

Country Status (5)

Country Link
US (2) US11340701B2 (cg-RX-API-DMAC7.html)
JP (1) JP7696764B2 (cg-RX-API-DMAC7.html)
CN (1) CN114270294B (cg-RX-API-DMAC7.html)
DE (1) DE112020006181T5 (cg-RX-API-DMAC7.html)
WO (1) WO2021127005A1 (cg-RX-API-DMAC7.html)

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11340701B2 (en) 2019-12-16 2022-05-24 Nvidia Corporation Gaze determination using glare as input
US11460857B1 (en) * 2020-02-21 2022-10-04 Zoox, Inc. Object or person attribute characterization
JP7160867B2 (ja) * 2020-07-17 2022-10-25 京セラ株式会社 電子機器、情報処理装置、推定方法、および推定プログラム
CN112711984B (zh) * 2020-12-09 2022-04-12 北京航空航天大学 注视点定位方法、装置和电子设备
CN112766163B (zh) * 2021-01-13 2022-05-31 北京航空航天大学 一种基于对抗优化的视线方向确定方法
DE112022001065T5 (de) * 2021-02-12 2023-12-28 Sony Semiconductor Solutions Corporation Informationsverarbeitungseinrichtung, informationsverarbeitungsverfahren und informationsverarbeitungsprogramm
US11687155B2 (en) * 2021-05-13 2023-06-27 Toyota Research Institute, Inc. Method for vehicle eye tracking system
CN115503609B (zh) * 2021-06-07 2024-11-22 英业达科技有限公司 侦测系统
US20230080085A1 (en) * 2021-09-10 2023-03-16 Aptiv Technologies Limited Driver vision assistance systems and methods
JP7766432B2 (ja) * 2021-09-10 2025-11-10 キヤノン株式会社 個人認証装置及び個人認証方法、プログラム
DE102021125945A1 (de) * 2021-10-06 2023-04-06 Bayerische Motoren Werke Aktiengesellschaft Fahrzeugführungssystem und Verfahren zur Steuerung einer Nothalteassistenzfunktion eines Kraftfahrzeugs
US11972614B2 (en) * 2021-11-09 2024-04-30 Zoox, Inc. Machine-learned architecture for efficient object attribute and/or intention classification
CN116229558A (zh) * 2021-12-02 2023-06-06 北京三星通信技术研究有限公司 视线估计方法、装置、计算机设备、存储介质及程序产品
EP4191545B1 (en) * 2021-12-02 2025-12-31 Samsung Electronics Co., Ltd. DEVICE AND METHOD FOR ESTIMATING THE EYES
GB2618078A (en) 2022-04-25 2023-11-01 Continental Automotive Tech Gmbh An image processing method for removal of glare spots and system for the same
CN115214492B (zh) * 2022-07-12 2023-02-03 深圳市城市交通规划设计研究中心股份有限公司 基于视频辅助的车载酒驾检测方法、电子设备及存储介质
EP4345774A1 (en) * 2022-09-27 2024-04-03 Aptiv Technologies Limited Diminished driver control detection system, method, and software
EP4427995A1 (en) * 2023-03-06 2024-09-11 Aptiv Technologies AG Method of assisting a driver of a vehicle, and a driver assistance system thereof
CN118052718A (zh) * 2024-01-15 2024-05-17 南京理工大学 一种基于双目估计和偏振调制的无监督水下成像方法

Family Cites Families (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1032872A1 (en) 1997-11-17 2000-09-06 BRITISH TELECOMMUNICATIONS public limited company User interface
US8457352B2 (en) * 2007-05-23 2013-06-04 The University Of British Columbia Methods and apparatus for estimating point-of-gaze in three dimensions
US8581647B2 (en) 2011-11-10 2013-11-12 Qualcomm Incorporated System and method of stabilizing charge pump node voltage levels
KR101995958B1 (ko) * 2012-11-28 2019-07-03 한국전자통신연구원 스마트 글래스 기반 영상 처리 장치 및 방법
CN103873840B (zh) * 2012-12-12 2018-08-31 联想(北京)有限公司 显示方法及显示设备
US11137832B2 (en) * 2012-12-13 2021-10-05 Eyesight Mobile Technologies, LTD. Systems and methods to predict a user action within a vehicle
US9454699B2 (en) * 2014-04-29 2016-09-27 Microsoft Technology Licensing, Llc Handling glare in eye tracking
JP2016201609A (ja) 2015-04-08 2016-12-01 日本電気通信システム株式会社 加入者端末装置、通信サービス提供システム、通信制御方法、及び、通信制御プログラム
JP2017129973A (ja) * 2016-01-19 2017-07-27 富士通テン株式会社 運転支援装置および運転支援方法
US20180081434A1 (en) * 2016-09-22 2018-03-22 Apple Inc. Eye and Head Tracking
US10397498B2 (en) * 2017-01-11 2019-08-27 Sony Corporation Compressive sensing capturing device and method
JP6665819B2 (ja) * 2017-03-17 2020-03-13 トヨタ自動車株式会社 車載表示装置
WO2018227597A1 (en) * 2017-06-16 2018-12-20 Boe Technology Group Co., Ltd. Vision-based interactive control apparatus and method of controlling rear-view mirror for vehicle
JP6946831B2 (ja) * 2017-08-01 2021-10-13 オムロン株式会社 人物の視線方向を推定するための情報処理装置及び推定方法、並びに学習装置及び学習方法
JP6693489B2 (ja) * 2017-11-15 2020-05-13 オムロン株式会社 情報処理装置、運転者モニタリングシステム、情報処理方法、及び情報処理プログラム
CN108875524B (zh) * 2018-01-02 2021-03-02 北京旷视科技有限公司 视线估计方法、装置、系统和存储介质
CN111868737B (zh) * 2018-01-24 2025-01-24 苹果公司 使用神经网络的基于事件相机的注视跟踪
US10725292B2 (en) * 2018-02-01 2020-07-28 Varjo Technologies Oy Gaze-tracking system and aperture device
JP7040539B2 (ja) * 2018-02-08 2022-03-23 日本電気株式会社 視線推定装置、視線推定方法、およびプログラム
WO2019154511A1 (en) 2018-02-09 2019-08-15 Pupil Labs Gmbh Devices, systems and methods for predicting gaze-related parameters using a neural network
US10664999B2 (en) * 2018-02-15 2020-05-26 Adobe Inc. Saliency prediction for a mobile user interface
JP2021530069A (ja) * 2018-06-26 2021-11-04 カッツ,イテイ 状況的ドライバ監視システム
US10861225B2 (en) * 2018-08-10 2020-12-08 Intel Corporation Neural network processing for multi-object 3D modeling
JP7180228B2 (ja) * 2018-09-20 2022-11-30 いすゞ自動車株式会社 車両用監視装置
CN109635761B (zh) * 2018-12-18 2021-09-10 北京七鑫易维信息技术有限公司 一种虹膜识别图像确定方法、装置、终端设备及存储介质
US10996751B2 (en) * 2018-12-21 2021-05-04 Tobii Ab Training of a gaze tracking model
US10936059B2 (en) * 2019-07-26 2021-03-02 Cajal Corporation Systems and methods for gaze tracking
US11340701B2 (en) * 2019-12-16 2022-05-24 Nvidia Corporation Gaze determination using glare as input
JP7219787B2 (ja) * 2021-04-09 2023-02-08 本田技研工業株式会社 情報処理装置、情報処理方法、学習方法、およびプログラム

Also Published As

Publication number Publication date
JP7696764B2 (ja) 2025-06-23
US20220283638A1 (en) 2022-09-08
DE112020006181T5 (de) 2022-11-10
US20210181837A1 (en) 2021-06-17
JP2021195124A (ja) 2021-12-27
WO2021127005A1 (en) 2021-06-24
US11841987B2 (en) 2023-12-12
US11340701B2 (en) 2022-05-24
CN114270294A (zh) 2022-04-01

Similar Documents

Publication Publication Date Title
US11841458B2 (en) Domain restriction of neural networks through synthetic data pre-training
US11934955B2 (en) Neural network based facial analysis using facial landmarks and associated confidence values
JP7742240B2 (ja) 自律及び半自律運転アプリケーションのための乗員注意力及び認知負荷監視
CN113767389B (zh) 从用于自主机器应用的经变换的真实世界传感器数据模拟逼真的测试数据
JP7680897B2 (ja) 歩行者横断危険性評価及び指向性警報のためのシステム及び方法
US11841987B2 (en) Gaze determination using glare as input
CN115218888B (zh) 用于更新高清地图的系统和方法
CN115136148B (zh) 投影使用鱼眼镜头捕获的图像用于自主机器应用中的特征检测
CN112989914A (zh) 具有自适应加权输入的注视确定机器学习系统
US12492913B2 (en) Automatic graphical content recognition for vehicle applications
CN113632095A (zh) 使用适合停车位检测的倾斜多边形进行对象检测
JP2021195124A5 (cg-RX-API-DMAC7.html)
CN111133448A (zh) 使用安全到达时间控制自动驾驶车辆
CN113609888B (zh) 利用平面单应性和自监督的场景结构理解进行对象检测
CN117011329B (zh) 用于自主系统和应用的对象追踪和碰撞时间估计
CN119559596A (zh) 用于自主和半自主系统及应用的路径标记检测和分类
JP2023082647A (ja) 自律システム及びアプリケーションのためのカメラベースの入力を使用した深層学習ベースの動作領域検証
CN117581117A (zh) 自主机器系统和应用中使用LiDAR数据的动态对象检测
US20240010196A1 (en) Learning autonomous vehicle safety concepts from demonstrations
CN117516565A (zh) 用于在自主系统和应用中导航的车道偏置
CN120070881A (zh) 用于自主系统和应用的检测图像中的被遮挡对象
CN120655873A (zh) 使用生成式ai的基于深度的车辆环境可视化
CN120655872A (zh) 使用生成式ai的基于深度的车辆环境可视化
CN121500321A (zh) 用于自主系统与应用的使用超声波传感器的对象检测
CN120833365A (zh) 计算特征相关性以估计立体图像的深度信息

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant