CN104685541B - 用于确定三维对象上注视点的方法和装置 - Google Patents

用于确定三维对象上注视点的方法和装置 Download PDF

Info

Publication number
CN104685541B
CN104685541B CN201380048140.XA CN201380048140A CN104685541B CN 104685541 B CN104685541 B CN 104685541B CN 201380048140 A CN201380048140 A CN 201380048140A CN 104685541 B CN104685541 B CN 104685541B
Authority
CN
China
Prior art keywords
scene
gaze
attentively
camera
endpoint
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201380048140.XA
Other languages
English (en)
Chinese (zh)
Other versions
CN104685541A (zh
Inventor
简·霍夫曼
汤姆·辛格洛布
丹尼斯·威廉姆斯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Apple Inc
Original Assignee
Smi Innovative Sensor Technology Co ltd
SensoMotoric Instruments Gesellschaft fuer Innovative Sensorik mbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Smi Innovative Sensor Technology Co ltd, SensoMotoric Instruments Gesellschaft fuer Innovative Sensorik mbH filed Critical Smi Innovative Sensor Technology Co ltd
Publication of CN104685541A publication Critical patent/CN104685541A/zh
Application granted granted Critical
Publication of CN104685541B publication Critical patent/CN104685541B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/19—Sensors therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • General Health & Medical Sciences (AREA)
  • Ophthalmology & Optometry (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biophysics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Veterinary Medicine (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Position Input By Displaying (AREA)
  • User Interface Of Digital Computer (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Eye Examination Apparatus (AREA)
  • Studio Devices (AREA)
  • Image Analysis (AREA)
CN201380048140.XA 2012-09-17 2013-09-17 用于确定三维对象上注视点的方法和装置 Active CN104685541B (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
EP12184725.5 2012-09-17
EP12184725.5A EP2709060B1 (en) 2012-09-17 2012-09-17 Method and an apparatus for determining a gaze point on a three-dimensional object
PCT/EP2013/069236 WO2014041188A1 (en) 2012-09-17 2013-09-17 Method and an apparatus for determining a gaze point on a three-dimensional object

Publications (2)

Publication Number Publication Date
CN104685541A CN104685541A (zh) 2015-06-03
CN104685541B true CN104685541B (zh) 2019-03-05

Family

ID=47010238

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201380048140.XA Active CN104685541B (zh) 2012-09-17 2013-09-17 用于确定三维对象上注视点的方法和装置

Country Status (6)

Country Link
US (5) US10223808B2 (enExample)
EP (1) EP2709060B1 (enExample)
JP (2) JP6808320B2 (enExample)
CN (1) CN104685541B (enExample)
IN (1) IN2015DN01984A (enExample)
WO (1) WO2014041188A1 (enExample)

Families Citing this family (53)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9811731B2 (en) * 2013-10-04 2017-11-07 Qualcomm Incorporated Dynamic extension of map data for object detection and tracking
EP3968288A2 (de) * 2014-02-04 2022-03-16 FRAUNHOFER-GESELLSCHAFT zur Förderung der angewandten Forschung e.V. 3d-bildanalysator zur blickrichtungsbestimmung
JP2015219892A (ja) * 2014-05-21 2015-12-07 大日本印刷株式会社 視線分析システムおよび視線分析装置
KR20170041720A (ko) * 2014-08-07 2017-04-17 포브, 아이엔씨. 3차원상의 주시점의 위치 특정 알고리즘
US10228766B2 (en) * 2014-09-12 2019-03-12 Microsoft Technology Licensing, Llc Enhanced Display Rotation
JP6547268B2 (ja) * 2014-10-02 2019-07-24 富士通株式会社 視線位置検出装置、視線位置検出方法及び視線位置検出プログラム
JP6663441B2 (ja) * 2015-03-01 2020-03-11 ノバサイト リミテッド 眼球運動を測定するためのシステム
CN107533634A (zh) * 2015-03-23 2018-01-02 控制辐射系统有限公司 眼睛跟踪系统
WO2016179825A1 (zh) 2015-05-14 2016-11-17 中国科学院深圳先进技术研究院 基于三维场景的导航方法
GB2539009A (en) 2015-06-03 2016-12-07 Tobii Ab Gaze detection method and apparatus
US9829976B2 (en) * 2015-08-07 2017-11-28 Tobii Ab Gaze direction mapping
US10444972B2 (en) 2015-11-28 2019-10-15 International Business Machines Corporation Assisting a user with efficient navigation between a selection of entries with elements of interest to the user within a stream of entries
JP2017129898A (ja) * 2016-01-18 2017-07-27 ソニー株式会社 情報処理装置、情報処理方法、およびプログラム
US11579686B2 (en) * 2016-03-07 2023-02-14 Apple Inc. Method and device for carrying out eye gaze mapping
US10063560B2 (en) * 2016-04-29 2018-08-28 Microsoft Technology Licensing, Llc Gaze-based authentication
WO2017208227A1 (en) 2016-05-29 2017-12-07 Nova-Sight Ltd. Display system and method
WO2017216273A1 (en) * 2016-06-16 2017-12-21 SensoMotoric Instruments Gesellschaft für innovative Sensorik mbH Method and system for providing eye tracking based information about a user behavior, client device, server and computer program product
JP2020509790A (ja) 2016-09-23 2020-04-02 ノバサイト リミテッド スクリーニング装置及び方法
WO2018076202A1 (zh) * 2016-10-26 2018-05-03 中国科学院深圳先进技术研究院 能够进行人眼追踪的头戴式可视设备及人眼追踪方法
CN108338798B (zh) * 2017-01-25 2020-11-06 北京数字新思科技有限公司 一种注意力测量方法和系统
CN106840167B (zh) * 2017-02-23 2019-12-20 周良辰 基于街景地图的目标物地理位置的二维量测算方法
WO2018191970A1 (zh) 2017-04-21 2018-10-25 深圳前海达闼云端智能科技有限公司 一种机器人控制方法、机器人装置及机器人设备
CN108965687B (zh) * 2017-05-22 2021-01-29 阿里巴巴集团控股有限公司 拍摄方向识别方法、服务器及监控方法、系统及摄像设备
CN107369205B (zh) * 2017-07-04 2020-10-16 东南大学 一种移动终端城市二三维联动显示方法
CN107392853B (zh) * 2017-07-13 2020-05-26 河北中科恒运软件科技股份有限公司 双摄像头视频融合畸变校正与视点微调整方法及系统
US10853965B2 (en) 2017-08-07 2020-12-01 Standard Cognition, Corp Directional impression analysis using deep learning
CN107504917B (zh) * 2017-08-17 2020-02-21 深圳市异方科技有限公司 一种三维尺寸测量方法及装置
KR102495359B1 (ko) 2017-10-27 2023-02-02 삼성전자주식회사 객체 트래킹 방법 및 장치
US11310472B2 (en) * 2017-11-16 2022-04-19 Sony Interactive Entertainment Inc. Information processing device and image generation method for projecting a subject image onto a virtual screen
US11017779B2 (en) 2018-02-15 2021-05-25 DMAI, Inc. System and method for speech understanding via integrated audio and visual based speech recognition
US11308312B2 (en) 2018-02-15 2022-04-19 DMAI, Inc. System and method for reconstructing unoccupied 3D space
US11455986B2 (en) 2018-02-15 2022-09-27 DMAI, Inc. System and method for conversational agent via adaptive caching of dialogue tree
WO2019161241A1 (en) * 2018-02-15 2019-08-22 DMAI, Inc. System and method for identifying a point of interest based on intersecting visual trajectories
CN114935974B (zh) 2018-03-30 2025-04-25 托比股份公司 用于确定注视关注目标的对物体映射的多线迹注视
TWI719343B (zh) * 2018-08-28 2021-02-21 財團法人工業技術研究院 資訊顯示方法及其顯示系統
CN112753037A (zh) * 2018-09-28 2021-05-04 苹果公司 传感器融合眼睛跟踪
CN109491508B (zh) * 2018-11-27 2022-08-26 北京七鑫易维信息技术有限公司 一种确定注视对象的方法和装置
SE543121C2 (en) * 2019-02-04 2020-10-13 Tobii Ab Method and system for determining a current gaze direction
SE1950580A1 (en) * 2019-05-15 2020-10-13 Tobii Ab Method and system for dwell-less, hands-free interaction with a selectable object
CN112013844B (zh) * 2019-05-31 2022-02-11 北京小米智能科技有限公司 建立室内环境地图的方法及装置
US11181978B2 (en) 2019-06-17 2021-11-23 Hemy8 Sa System and method for gaze estimation
US11487968B2 (en) * 2019-12-16 2022-11-01 Nvidia Corporation Neural network based facial analysis using facial landmarks and associated confidence values
EP3839411B1 (fr) * 2019-12-17 2023-08-23 John Cockerill Defense SA Système intelligent pour le contrôle de fonctions dans une tourelle de véhicule de combat
CN112364765B (zh) * 2020-11-11 2024-12-31 京东方科技集团股份有限公司 一种人眼运动注视点检测方法及装置
CN112509007B (zh) * 2020-12-14 2024-06-04 科大讯飞股份有限公司 真实注视点定位方法以及头戴式视线跟踪系统
GB2603947A (en) * 2021-02-22 2022-08-24 Nokia Technologies Oy Method and apparatus for depth estimation relative to a mobile terminal
CN113553920B (zh) * 2021-07-02 2024-06-04 黑芝麻智能科技(上海)有限公司 注视方向特征采集方法、装置、计算机设备和存储介质
JP7640427B2 (ja) * 2021-10-01 2025-03-05 株式会社日立製作所 注視位置分析システム及び注視位置分析方法
US20250168486A1 (en) * 2022-04-01 2025-05-22 Sony Group Corporation Information processing device, information processing method, and recording medium
JP2023155799A (ja) * 2022-04-11 2023-10-23 株式会社日立ソリューションズ・クリエイト 作業状態検出システムおよび方法
SE547500C2 (en) * 2022-06-03 2025-10-07 Tobii Ab Method of estimating a three-dimensional position of an object
US12056276B1 (en) 2023-04-27 2024-08-06 Meta Platforms Technologies, Llc Eye tracking based on vergence
SE546818C2 (en) * 2023-07-05 2025-02-25 Tobii Ab A method and a system for determining gaze convergence distance for a user of an eye tracking system

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101943982A (zh) * 2009-07-10 2011-01-12 北京大学 基于被跟踪的眼睛运动的图像操作

Family Cites Families (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6130957A (en) * 1996-12-06 2000-10-10 Nippon Telegraph And Telephone Corporation Method and system for producing computer generated holograms realizing real time holographic video production and display
JPH10326071A (ja) * 1997-03-27 1998-12-08 Nippon Telegr & Teleph Corp <Ntt> 注視点高解像度化による計算機ホログラム表示方法並びに装置並びにこの方法を記録した記録媒体
CN1126823C (zh) 1998-09-14 2003-11-05 库利克及索发投资有限公司 金属丝接合的合金复合材料
US7046924B2 (en) * 2002-11-25 2006-05-16 Eastman Kodak Company Method and computer program product for determining an area of importance in an image using eye monitoring information
JP2005071041A (ja) * 2003-08-22 2005-03-17 Nissan Motor Co Ltd 運転者視対象検出装置及び運転者視対象検出システム
CN102670163B (zh) * 2004-04-01 2016-04-13 威廉·C·托奇 控制计算装置的系统及方法
JP4875843B2 (ja) 2004-10-20 2012-02-15 オリンパス株式会社 情報端末装置
JP5044237B2 (ja) * 2006-03-27 2012-10-10 富士フイルム株式会社 画像記録装置、画像記録方法、および画像記録プログラム
JP4876687B2 (ja) 2006-04-19 2012-02-15 株式会社日立製作所 注目度計測装置及び注目度計測システム
JP2007319187A (ja) * 2006-05-30 2007-12-13 Toyota Motor Corp 動作補助装置及びその制御方法
US8781162B2 (en) * 2011-01-05 2014-07-15 Ailive Inc. Method and system for head tracking and pose estimation
KR100820639B1 (ko) * 2006-07-25 2008-04-10 한국과학기술연구원 시선 기반 3차원 인터랙션 시스템 및 방법 그리고 3차원시선 추적 시스템 및 방법
US8077914B1 (en) * 2006-08-07 2011-12-13 Arkady Kaplan Optical tracking apparatus using six degrees of freedom
CN101945612B (zh) * 2008-02-14 2013-09-25 诺基亚公司 用于确定注视方向的设备和方法
CN201307266Y (zh) * 2008-06-25 2009-09-09 韩旭 双目视线跟踪装置
CN101419672B (zh) * 2008-12-03 2010-09-08 中国科学院计算技术研究所 一种同步采集人脸图像和注视视角的装置及方法
WO2010083853A1 (en) 2009-01-26 2010-07-29 Tobii Technology Ab Detection of gaze point assisted by optical reference signals
US8553936B2 (en) * 2009-03-03 2013-10-08 The Ohio State University Gaze tracking measurement and training system and method
JP5590487B2 (ja) * 2010-07-30 2014-09-17 公立大学法人広島市立大学 視線計測方法及び視線計測装置
US8408706B2 (en) * 2010-12-13 2013-04-02 Microsoft Corporation 3D gaze tracker
JP5924862B2 (ja) * 2010-12-15 2016-05-25 キヤノン株式会社 情報処理装置、情報処理方法及びプログラム
JP5522799B2 (ja) * 2011-02-18 2014-06-18 日本放送協会 視線位置推定システム及び視線位置推定プログラム
JP2013008365A (ja) * 2011-06-23 2013-01-10 Ailive Inc 運動感知型機器を用いる遠隔制御
US8885882B1 (en) * 2011-07-14 2014-11-11 The Research Foundation For The State University Of New York Real time eye tracking for human computer interaction
US9116926B2 (en) * 2012-12-20 2015-08-25 Google Inc. Sharing photos

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101943982A (zh) * 2009-07-10 2011-01-12 北京大学 基于被跟踪的眼睛运动的图像操作

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
视线跟踪技术及其应用研究;徐兴民;《中国优秀硕士学位论文全文数据库 信息科技辑》;20080115(第1期);正文第4-5、12、21、40-41、49、59、62、65页 *

Also Published As

Publication number Publication date
US10699439B2 (en) 2020-06-30
US20200334858A1 (en) 2020-10-22
US20190272650A1 (en) 2019-09-05
US12423865B2 (en) 2025-09-23
CN104685541A (zh) 2015-06-03
US11676302B2 (en) 2023-06-13
US11948332B2 (en) 2024-04-02
US20240233178A1 (en) 2024-07-11
US20230206494A1 (en) 2023-06-29
US20150243036A1 (en) 2015-08-27
EP2709060B1 (en) 2020-02-26
WO2014041188A1 (en) 2014-03-20
EP2709060A1 (en) 2014-03-19
JP2015528359A (ja) 2015-09-28
JP6808320B2 (ja) 2021-01-06
US10223808B2 (en) 2019-03-05
JP2019141620A (ja) 2019-08-29
IN2015DN01984A (enExample) 2015-08-14
JP6690041B2 (ja) 2020-04-28

Similar Documents

Publication Publication Date Title
CN104685541B (zh) 用于确定三维对象上注视点的方法和装置
CN111194416B (zh) 用于生成场景的表示的装置和方法
JP5858433B2 (ja) 注視点検出方法及び注視点検出装置
JP6548171B2 (ja) 瞳孔検出システム、視線検出システム、瞳孔検出方法、および瞳孔検出プログラム
JP2020034919A (ja) 構造化光を用いた視線追跡
TWI520576B (zh) 將二維影像轉換爲三維影像的方法與系統及電腦可讀媒體
CN108259887B (zh) 注视点校准方法及装置、注视点标定方法及装置
JP2014211404A (ja) モーションキャプチャー方法
JP2011160421A (ja) 立体画像の作成方法、作成装置及びプログラム
CN108537103B (zh) 基于瞳孔轴测量的活体人脸检测方法及其设备
Hua et al. A testbed for precise registration, natural occlusion and interaction in an augmented environment using a head-mounted projective display (HMPD)
JP2022528202A (ja) 位置合わせ方法およびセットアップ
US20250391051A1 (en) Method and an Apparatus for Determining a Gaze Point on a Three-Dimensional Object
JP5337409B2 (ja) 情報提示装置
JP2016057906A (ja) 視点位置の計測方法及び計測システム
Paletta et al. 3D recovery of human gaze in natural environments
JP2003223654A (ja) 仮想表示物体観測装置

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information
CB02 Change of applicant information

Address after: Germany / Berlin Teltow

Applicant after: Sensomotoric Instruments GmbH

Address before: Germany / Berlin Teltow

Applicant before: Sense organ motion instrument innovation sensor company limited

GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20190604

Address after: American California

Patentee after: Apple Computer, Inc.

Address before: Germany / Berlin Teltow

Patentee before: Sensomotoric Instruments GmbH