JP7696764B2 - 入力としてグレアを使用する注視判定 - Google Patents

入力としてグレアを使用する注視判定 Download PDF

Info

Publication number
JP7696764B2
JP7696764B2 JP2021100474A JP2021100474A JP7696764B2 JP 7696764 B2 JP7696764 B2 JP 7696764B2 JP 2021100474 A JP2021100474 A JP 2021100474A JP 2021100474 A JP2021100474 A JP 2021100474A JP 7696764 B2 JP7696764 B2 JP 7696764B2
Authority
JP
Japan
Prior art keywords
subject
representation
face
machine learning
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2021100474A
Other languages
English (en)
Japanese (ja)
Other versions
JP2021195124A5 (cg-RX-API-DMAC7.html
JP2021195124A (ja
Inventor
ジャン ハイロン
プーリー ニシャント
アヴァダナム ニランジャン
ムラート アラル ヌーリー
Original Assignee
エヌビディア コーポレーション
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by エヌビディア コーポレーション filed Critical エヌビディア コーポレーション
Publication of JP2021195124A publication Critical patent/JP2021195124A/ja
Publication of JP2021195124A5 publication Critical patent/JP2021195124A5/ja
Application granted granted Critical
Publication of JP7696764B2 publication Critical patent/JP7696764B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/19—Sensors therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Ophthalmology & Optometry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
JP2021100474A 2019-12-16 2021-06-16 入力としてグレアを使用する注視判定 Active JP7696764B2 (ja)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201962948793P 2019-12-16 2019-12-16
US16/902,737 2020-06-16
US16/902,737 US11340701B2 (en) 2019-12-16 2020-06-16 Gaze determination using glare as input

Publications (3)

Publication Number Publication Date
JP2021195124A JP2021195124A (ja) 2021-12-27
JP2021195124A5 JP2021195124A5 (cg-RX-API-DMAC7.html) 2025-01-10
JP7696764B2 true JP7696764B2 (ja) 2025-06-23

Family

ID=76317901

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2021100474A Active JP7696764B2 (ja) 2019-12-16 2021-06-16 入力としてグレアを使用する注視判定

Country Status (5)

Country Link
US (2) US11340701B2 (cg-RX-API-DMAC7.html)
JP (1) JP7696764B2 (cg-RX-API-DMAC7.html)
CN (1) CN114270294B (cg-RX-API-DMAC7.html)
DE (1) DE112020006181T5 (cg-RX-API-DMAC7.html)
WO (1) WO2021127005A1 (cg-RX-API-DMAC7.html)

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11340701B2 (en) 2019-12-16 2022-05-24 Nvidia Corporation Gaze determination using glare as input
US11460857B1 (en) * 2020-02-21 2022-10-04 Zoox, Inc. Object or person attribute characterization
JP7160867B2 (ja) * 2020-07-17 2022-10-25 京セラ株式会社 電子機器、情報処理装置、推定方法、および推定プログラム
CN112711984B (zh) * 2020-12-09 2022-04-12 北京航空航天大学 注视点定位方法、装置和电子设备
CN112766163B (zh) * 2021-01-13 2022-05-31 北京航空航天大学 一种基于对抗优化的视线方向确定方法
DE112022001065T5 (de) * 2021-02-12 2023-12-28 Sony Semiconductor Solutions Corporation Informationsverarbeitungseinrichtung, informationsverarbeitungsverfahren und informationsverarbeitungsprogramm
US11687155B2 (en) * 2021-05-13 2023-06-27 Toyota Research Institute, Inc. Method for vehicle eye tracking system
CN115503609B (zh) * 2021-06-07 2024-11-22 英业达科技有限公司 侦测系统
US20230080085A1 (en) * 2021-09-10 2023-03-16 Aptiv Technologies Limited Driver vision assistance systems and methods
JP7766432B2 (ja) * 2021-09-10 2025-11-10 キヤノン株式会社 個人認証装置及び個人認証方法、プログラム
DE102021125945A1 (de) * 2021-10-06 2023-04-06 Bayerische Motoren Werke Aktiengesellschaft Fahrzeugführungssystem und Verfahren zur Steuerung einer Nothalteassistenzfunktion eines Kraftfahrzeugs
US11972614B2 (en) * 2021-11-09 2024-04-30 Zoox, Inc. Machine-learned architecture for efficient object attribute and/or intention classification
CN116229558A (zh) * 2021-12-02 2023-06-06 北京三星通信技术研究有限公司 视线估计方法、装置、计算机设备、存储介质及程序产品
EP4191545B1 (en) * 2021-12-02 2025-12-31 Samsung Electronics Co., Ltd. DEVICE AND METHOD FOR ESTIMATING THE EYES
GB2618078A (en) 2022-04-25 2023-11-01 Continental Automotive Tech Gmbh An image processing method for removal of glare spots and system for the same
CN115214492B (zh) * 2022-07-12 2023-02-03 深圳市城市交通规划设计研究中心股份有限公司 基于视频辅助的车载酒驾检测方法、电子设备及存储介质
EP4345774A1 (en) * 2022-09-27 2024-04-03 Aptiv Technologies Limited Diminished driver control detection system, method, and software
EP4427995A1 (en) * 2023-03-06 2024-09-11 Aptiv Technologies AG Method of assisting a driver of a vehicle, and a driver assistance system thereof
CN118052718A (zh) * 2024-01-15 2024-05-17 南京理工大学 一种基于双目估计和偏振调制的无监督水下成像方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017129973A (ja) 2016-01-19 2017-07-27 富士通テン株式会社 運転支援装置および運転支援方法
JP2017521738A (ja) 2014-04-29 2017-08-03 マイクロソフト テクノロジー ライセンシング,エルエルシー 目追跡でのグレアに対処すること
JP2019028843A (ja) 2017-08-01 2019-02-21 オムロン株式会社 人物の視線方向を推定するための情報処理装置及び推定方法、並びに学習装置及び学習方法
JP2019091255A (ja) 2017-11-15 2019-06-13 オムロン株式会社 情報処理装置、運転者モニタリングシステム、情報処理方法、及び情報処理プログラム
WO2019155570A1 (ja) 2018-02-08 2019-08-15 日本電気株式会社 視線推定装置、視線推定方法、および記録媒体
JP2020047087A (ja) 2018-09-20 2020-03-26 いすゞ自動車株式会社 車両用監視装置
JP2022161689A (ja) 2021-04-09 2022-10-21 本田技研工業株式会社 情報処理装置、情報処理方法、学習方法、およびプログラム

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1032872A1 (en) 1997-11-17 2000-09-06 BRITISH TELECOMMUNICATIONS public limited company User interface
US8457352B2 (en) * 2007-05-23 2013-06-04 The University Of British Columbia Methods and apparatus for estimating point-of-gaze in three dimensions
US8581647B2 (en) 2011-11-10 2013-11-12 Qualcomm Incorporated System and method of stabilizing charge pump node voltage levels
KR101995958B1 (ko) * 2012-11-28 2019-07-03 한국전자통신연구원 스마트 글래스 기반 영상 처리 장치 및 방법
CN103873840B (zh) * 2012-12-12 2018-08-31 联想(北京)有限公司 显示方法及显示设备
US11137832B2 (en) * 2012-12-13 2021-10-05 Eyesight Mobile Technologies, LTD. Systems and methods to predict a user action within a vehicle
JP2016201609A (ja) 2015-04-08 2016-12-01 日本電気通信システム株式会社 加入者端末装置、通信サービス提供システム、通信制御方法、及び、通信制御プログラム
US20180081434A1 (en) * 2016-09-22 2018-03-22 Apple Inc. Eye and Head Tracking
US10397498B2 (en) * 2017-01-11 2019-08-27 Sony Corporation Compressive sensing capturing device and method
JP6665819B2 (ja) * 2017-03-17 2020-03-13 トヨタ自動車株式会社 車載表示装置
WO2018227597A1 (en) * 2017-06-16 2018-12-20 Boe Technology Group Co., Ltd. Vision-based interactive control apparatus and method of controlling rear-view mirror for vehicle
CN108875524B (zh) * 2018-01-02 2021-03-02 北京旷视科技有限公司 视线估计方法、装置、系统和存储介质
CN111868737B (zh) * 2018-01-24 2025-01-24 苹果公司 使用神经网络的基于事件相机的注视跟踪
US10725292B2 (en) * 2018-02-01 2020-07-28 Varjo Technologies Oy Gaze-tracking system and aperture device
WO2019154511A1 (en) 2018-02-09 2019-08-15 Pupil Labs Gmbh Devices, systems and methods for predicting gaze-related parameters using a neural network
US10664999B2 (en) * 2018-02-15 2020-05-26 Adobe Inc. Saliency prediction for a mobile user interface
JP2021530069A (ja) * 2018-06-26 2021-11-04 カッツ,イテイ 状況的ドライバ監視システム
US10861225B2 (en) * 2018-08-10 2020-12-08 Intel Corporation Neural network processing for multi-object 3D modeling
CN109635761B (zh) * 2018-12-18 2021-09-10 北京七鑫易维信息技术有限公司 一种虹膜识别图像确定方法、装置、终端设备及存储介质
US10996751B2 (en) * 2018-12-21 2021-05-04 Tobii Ab Training of a gaze tracking model
US10936059B2 (en) * 2019-07-26 2021-03-02 Cajal Corporation Systems and methods for gaze tracking
US11340701B2 (en) * 2019-12-16 2022-05-24 Nvidia Corporation Gaze determination using glare as input

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017521738A (ja) 2014-04-29 2017-08-03 マイクロソフト テクノロジー ライセンシング,エルエルシー 目追跡でのグレアに対処すること
JP2017129973A (ja) 2016-01-19 2017-07-27 富士通テン株式会社 運転支援装置および運転支援方法
JP2019028843A (ja) 2017-08-01 2019-02-21 オムロン株式会社 人物の視線方向を推定するための情報処理装置及び推定方法、並びに学習装置及び学習方法
JP2019091255A (ja) 2017-11-15 2019-06-13 オムロン株式会社 情報処理装置、運転者モニタリングシステム、情報処理方法、及び情報処理プログラム
WO2019155570A1 (ja) 2018-02-08 2019-08-15 日本電気株式会社 視線推定装置、視線推定方法、および記録媒体
JP2020047087A (ja) 2018-09-20 2020-03-26 いすゞ自動車株式会社 車両用監視装置
JP2022161689A (ja) 2021-04-09 2022-10-21 本田技研工業株式会社 情報処理装置、情報処理方法、学習方法、およびプログラム

Also Published As

Publication number Publication date
US20220283638A1 (en) 2022-09-08
DE112020006181T5 (de) 2022-11-10
CN114270294B (zh) 2025-05-06
US20210181837A1 (en) 2021-06-17
JP2021195124A (ja) 2021-12-27
WO2021127005A1 (en) 2021-06-24
US11841987B2 (en) 2023-12-12
US11340701B2 (en) 2022-05-24
CN114270294A (zh) 2022-04-01

Similar Documents

Publication Publication Date Title
US11934955B2 (en) Neural network based facial analysis using facial landmarks and associated confidence values
US11841987B2 (en) Gaze determination using glare as input
US11841458B2 (en) Domain restriction of neural networks through synthetic data pre-training
JP7680897B2 (ja) 歩行者横断危険性評価及び指向性警報のためのシステム及び方法
US20230012645A1 (en) Deep neural network processing for sensor blindness detection in autonomous machine applications
JP7644716B2 (ja) 自律マシン・アプリケーションのためのマップ情報で拡張されたグラウンド・トゥルース・データを使用するニューラル・ネットワーク・トレーニング
JP7399164B2 (ja) 駐車スペース検出に適したスキューされたポリゴンを使用した物体検出
US11636609B2 (en) Gaze determination machine learning system having adaptive weighting of inputs
JP2021195124A5 (cg-RX-API-DMAC7.html)
JP7820039B2 (ja) 自律システム及びアプリケーションのための画像コントラスト分析を使用した眩しさ軽減
US12492913B2 (en) Automatic graphical content recognition for vehicle applications
US12260017B2 (en) Gaze determination machine learning system having adaptive weighting of inputs
JP7649706B2 (ja) 自律マシン・アプリケーションのための配備されたディープ・ニューラル・ネットワークのパッチ
JP2023071168A (ja) 自律マシン・アプリケーションのための粒子ベース危険検出
JP7652587B2 (ja) 拘束デバイスのローカリゼーション
US20250314502A1 (en) Determining wait condition information associated with traffic features for autonomous systems and applications

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20240308

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20240926

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20241001

A524 Written submission of copy of amendment under article 19 pct

Free format text: JAPANESE INTERMEDIATE CODE: A524

Effective date: 20241226

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20250212

A601 Written request for extension of time

Free format text: JAPANESE INTERMEDIATE CODE: A601

Effective date: 20250512

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20250520

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20250603

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20250611

R150 Certificate of patent or registration of utility model

Ref document number: 7696764

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150