SG11201902856QA - Eyeball movement analysis method and device, and storage medium - Google Patents

Eyeball movement analysis method and device, and storage medium

Info

Publication number
SG11201902856QA
SG11201902856QA SG11201902856QA SG11201902856QA SG11201902856QA SG 11201902856Q A SG11201902856Q A SG 11201902856QA SG 11201902856Q A SG11201902856Q A SG 11201902856QA SG 11201902856Q A SG11201902856Q A SG 11201902856QA SG 11201902856Q A SG11201902856Q A SG 11201902856QA
Authority
SG
Singapore
Prior art keywords
real
eyeball
feature points
facial image
time
Prior art date
Application number
SG11201902856QA
Inventor
Lin Chen
Guohui Zhang
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Publication of SG11201902856QA publication Critical patent/SG11201902856QA/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/19—Sensors therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/14—Arrangements specially adapted for eye photography
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/197—Matching; Classification
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Ophthalmology & Optometry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Geometry (AREA)
  • Medical Informatics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Surgery (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Molecular Biology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Image Analysis (AREA)
  • Eye Examination Apparatus (AREA)
  • Image Processing (AREA)

Abstract

EYEBALL MOVEMENT ANALYSIS METHOD AND DEVICE, AND STORAGE MEDIUM The application discloses an eyeball movement analysis method and device and a storage medium. The method includes: acquiring a real-time image shot by a photographic device and extracting a real-time facial image from the real-time image; inputting the real-time facial image into a pretrained eye mean shape and recognizing n1 orbit feature points and n2 eyeball feature points representative of an eye position in the real-time facial image; determining an eye region according to the (n1+n2) feature points and judging whether the eye region is a human eye region or not by use of a human eye classification model; and if YES, calculating a movement direction and movement distance of an eyeball in the real-time facial image according to x and y coordinates of the n1 orbit feature points and n2 eyeball feature points in the real-time facial image. According to the application, movement information of the eyeball in the real-time facial image is calculated according to the coordinates of eye feature points to implement analysis about the eye region and real-time capturing of an eyeball movement. Fig. 3
SG11201902856QA 2017-08-17 2017-10-31 Eyeball movement analysis method and device, and storage medium SG11201902856QA (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710709112.8A CN107679448B (en) 2017-08-17 2017-08-17 Eyeball action-analysing method, device and storage medium
PCT/CN2017/108748 WO2019033569A1 (en) 2017-08-17 2017-10-31 Eyeball movement analysis method, device and storage medium

Publications (1)

Publication Number Publication Date
SG11201902856QA true SG11201902856QA (en) 2019-05-30

Family

ID=61135004

Family Applications (1)

Application Number Title Priority Date Filing Date
SG11201902856QA SG11201902856QA (en) 2017-08-17 2017-10-31 Eyeball movement analysis method and device, and storage medium

Country Status (4)

Country Link
US (1) US10534957B2 (en)
CN (1) CN107679448B (en)
SG (1) SG11201902856QA (en)
WO (1) WO2019033569A1 (en)

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10521661B2 (en) * 2017-09-01 2019-12-31 Magic Leap, Inc. Detailed eye shape model for robust biometric applications
CN108564531B (en) * 2018-05-08 2022-07-08 麒麟合盛网络技术股份有限公司 Image processing method and device
CN108563238B (en) * 2018-06-15 2021-08-24 歌尔科技有限公司 Method, device, equipment and system for remotely controlling unmanned aerial vehicle
JP7290930B2 (en) * 2018-09-27 2023-06-14 株式会社アイシン Occupant modeling device, occupant modeling method and occupant modeling program
CN109389069B (en) 2018-09-28 2021-01-05 北京市商汤科技开发有限公司 Gaze point determination method and apparatus, electronic device, and computer storage medium
CN109389105B (en) * 2018-12-20 2022-02-08 北京万里红科技有限公司 Multitask-based iris detection and visual angle classification method
CN109901716B (en) * 2019-03-04 2022-08-26 厦门美图之家科技有限公司 Sight point prediction model establishing method and device and sight point prediction method
CN110297539A (en) * 2019-06-19 2019-10-01 重庆工商职业学院 A kind of eye movement recognition methods and device based on artificial intelligence
CN111062786B (en) * 2019-12-25 2023-05-23 创新奇智(青岛)科技有限公司 Model updating method based on establishment of commodity appearance characteristic mapping table
US10860098B1 (en) * 2019-12-30 2020-12-08 Hulu, LLC Gesture-based eye tracking
CN111160303B (en) * 2019-12-31 2023-05-02 深圳大学 Eye movement response information detection method and device, mobile terminal and storage medium
CN111353429A (en) * 2020-02-28 2020-06-30 深圳壹账通智能科技有限公司 Interest degree method and system based on eyeball turning
CN111563633A (en) * 2020-05-15 2020-08-21 上海乂学教育科技有限公司 Reading training system and method based on eye tracker
CN112022642B (en) * 2020-09-16 2023-01-10 杭州集视智能科技有限公司 Edge visual field training equipment and training method based on visual field center damage
CN112329629B (en) * 2020-11-05 2023-11-14 平安科技（深圳）有限公司 Evaluation method and device for online training, computer equipment and storage medium
CN112446322B (en) * 2020-11-24 2024-01-23 杭州网易云音乐科技有限公司 Eyeball characteristic detection method, device, equipment and computer readable storage medium
CN113963799A (en) * 2021-10-22 2022-01-21 向子伦 Eyeball training effect evaluation method and device, electronic equipment and storage medium

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7206435B2 (en) * 2002-03-26 2007-04-17 Honda Giken Kogyo Kabushiki Kaisha Real-time eye detection and tracking under various light conditions
JP4876687B2 (en) * 2006-04-19 2012-02-15 株式会社日立製作所 Attention level measuring device and attention level measuring system
JP4946730B2 (en) * 2007-08-27 2012-06-06 ソニー株式会社 Face image processing apparatus, face image processing method, and computer program
CN102073435A (en) 2009-11-23 2011-05-25 英业达股份有限公司 Picture operating method and electronic device using same
CN101706872A (en) * 2009-11-26 2010-05-12 上海交通大学 Universal open type face identification system
CN101984453B (en) * 2010-11-02 2013-05-08 中国科学技术大学 Human eye recognition system and method
EP2877909B1 (en) * 2012-07-27 2018-12-26 Nokia Technologies Oy Multimodal interaction with near-to-eye display
US9176581B2 (en) * 2012-09-28 2015-11-03 Intel Corporation System and method for inferring user intent based on eye movement during observation of a display screen
CN104123543B (en) * 2014-07-23 2018-11-27 泰亿格电子（上海）有限公司 A kind of eye movement recognition methods based on recognition of face
US20160378079A1 (en) * 2015-06-24 2016-12-29 Hon Hai Precision Industry Co., Ltd. Computing device and electrical device controlling method
CN105069745A (en) * 2015-08-14 2015-11-18 济南中景电子科技有限公司 face-changing system based on common image sensor and enhanced augmented reality technology and method
KR102648770B1 (en) * 2016-07-14 2024-03-15 매직 립, 인코포레이티드 Deep neural network for iris identification
CN106295566B (en) * 2016-08-10 2019-07-09 北京小米移动软件有限公司 Facial expression recognizing method and device
EP3535646A4 (en) * 2016-11-07 2020-08-12 Nauto, Inc. System and method for driver distraction determination
CN106845425A (en) * 2017-01-25 2017-06-13 迈吉客科技（北京）有限公司 A kind of visual tracking method and tracks of device
US20180271364A1 (en) * 2017-03-24 2018-09-27 Osaka University Method and apparatus for detecting ocular movement disorders

Also Published As

Publication number Publication date
CN107679448A (en) 2018-02-09
WO2019033569A8 (en) 2019-11-21
CN107679448B (en) 2018-09-25
US20190362144A1 (en) 2019-11-28
WO2019033569A1 (en) 2019-02-21
US10534957B2 (en) 2020-01-14

Similar Documents

Publication Publication Date Title
SG11201902856QA (en) Eyeball movement analysis method and device, and storage medium
Jalal et al. Human activity recognition via the features of labeled depth body parts
CN108062546B (en) Computer face emotion recognition system
Megavannan et al. Human action recognition using depth maps
CN107766819B (en) Video monitoring system and real-time gait recognition method thereof
US9723979B2 (en) Method, apparatus and computer program product for positioning pupil
EP3396591A3 (en) Recognition, reidentification and security enhancements using autonomous machines
WO2020108834A9 (en) System and method for human gait analysis
Sohail et al. Detection of facial feature points using anthropometric face model
CN105426827A (en) Living body verification method, device and system
Ding et al. STFC: Spatio-temporal feature chain for skeleton-based human action recognition
US20040028260A1 (en) Posture recognition apparatus and autonomous robot
CN105809144A (en) Gesture recognition system and method adopting action segmentation
CN106960473B (en) behavior perception system and method
SG11201900381PA (en) Method for tracking pinched fingertips based on video
CN102831408A (en) Human face recognition method
US20220189131A1 (en) Modification of projected structured light based on identified points within captured image
CN106599873A (en) Figure identity identification method based on three-dimensional attitude information
KR20100011280A (en) Method and apparatus of recognizing detailed facial expression using facial expression information amplification
CN113807287A (en) 3D structured light face recognition method
Rakshita Communication through real-time video oculography using face landmark detection
KR20140146840A (en) Visual speech recognition system using multiple lip movement features extracted from lip image
Das et al. Human gait analysis and recognition using support vector machines
KR101514242B1 (en) Method and apparatus for tracking facial features
Sharma et al. Comparative analysis of edge detection operators for better glint detection