WO2014169441A1 - Procédé et système d'oculométrie utilisant une combinaison de détection et d'estimation du mouvement - Google Patents

Procédé et système d'oculométrie utilisant une combinaison de détection et d'estimation du mouvement Download PDF

Info

Publication number
WO2014169441A1
WO2014169441A1 PCT/CN2013/074273 CN2013074273W WO2014169441A1 WO 2014169441 A1 WO2014169441 A1 WO 2014169441A1 CN 2013074273 W CN2013074273 W CN 2013074273W WO 2014169441 A1 WO2014169441 A1 WO 2014169441A1
Authority
WO
WIPO (PCT)
Prior art keywords
eye
detection
information
frame
algorithm
Prior art date
Application number
PCT/CN2013/074273
Other languages
English (en)
Inventor
Wenjuan Song
Wei Zhou
Jianping Song
Original Assignee
Thomson Licensing
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thomson Licensing filed Critical Thomson Licensing
Priority to PCT/CN2013/074273 priority Critical patent/WO2014169441A1/fr
Publication of WO2014169441A1 publication Critical patent/WO2014169441A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/19—Sensors therefor

Definitions

  • the present invention relates to a method and a system for eye tracking using a combination of detection and motion estimation. More particularly, the present invention relates to eye tracking using face detection and eye detection combined with motion estimation.
  • Eye tracking is the process of measuring either the point of gaze ("where we are looking") or the motion of eyes relative to the head of a user.
  • An eye tracker is a device or system for measuring eye positions and eye movements. The eye tracker is used in research on the visual system, in psychology, in cognitive linguistics, and in product design. There are a number of methods for measuring eye movement.
  • we implement an eye tracker by detecting image data. The detection is divided into steps. The first step is detecting a face. Then the next step is performing eye detection on the basis of the result of the face detection as input and obtaining the eye's position. Both face detection and eye detection algorithms can use Haar features and Adaboost. Haar features are simples features based on basic computation on sub image areas.
  • Adaboost is a simple and iterative algorithm that helps building an accurate "strong” detector by combining layers of "weak” detectors in a cascade in an initial learning step.
  • Haar features and Adaboost have a much higher detected rate and faster detection result than most other detection algorithms, computation is very expensive.
  • Motion estimation is the process of determining motion vectors that describe the transformation from one 2D image to another, usually from adjacent frames in a video sequence. It is an ill-posed problem as the motion is in three dimensions but the images are a projection of the 3D scene onto a 2D plane.
  • the motion vectors may relate to the whole image (global motion estimation) or specific parts, such as rectangular blocks, arbitrary shaped patches or even per pixel.
  • the motion vectors may be represented by a translational model or many other models that can approximate the motion of a real video camera, such as rotation and translation in all three dimensions and zoom.
  • a Block Matching Algorithm (BMA) is a way of locating matching blocks in a sequence of digital video frames for the purpose of motion estimation.
  • the purpose of a block Matching Algorithm is to find a matching block from a frame i in some other frame j.
  • the Block Matching Algorithm makes use of criteria to determine whether a given block in frame j matches the search block in frame .
  • This invention combines detection algorithms with motion estimation algorithms to implement a one eye tracker system and makes the eye tracker system run on one limited resource (CPU, memory %) device.
  • US7708407B2 discloses a method and device for eye tracking compensation to reduce motion blur arising from eye tracking characteristics.
  • face detection when the next frame data arrives, face detection can be skipped; and the forecasted face position using motion estimation is directly used to detect eye position.
  • a method for detecting eye positions comprising the steps of: receiving image data of a frame;
  • determining at least one identified block in the frame checking whether there is information on the identified block; if there is the information, calculating face position on the basis of the information using move estimation algorithm; and detecting eye positions using eye detection algorithm.
  • a system for detecting eye positions comprising: means for receiving image data of a frame; means for determining at least one identified block in the frame; means for checking whether there is information on the identified block; means for
  • Figure 1A is an exemplary diagram illustrating a basic environment according to one embodiment of the present invention.
  • Figure IB is an exemplary diagram illustrating a detection result identified area according to one embodiment of the present invention.
  • Figure 2 is an exemplary diagram illustrating an eye detection process
  • Figure 3 is an exemplary flow chart illustrating an eye tracker system work process
  • Figure 4 is an exemplary block diagram illustrating a system according to an embodiment of the present invention .
  • the present invention is related to a method of eye tracking using face detection and eye detection combined with motion estimation to accelerate eye tracking.
  • An eye tracker system can be run on one limited resource (CPU, memory ...) devices.
  • CPU central processing unit
  • memory volatile memory
  • face detection for a whole frame always consumes a lot of time.
  • the present invention relates to the field of eye tracking. It is important for many applications that obtain real-life person's eye positions. For example, when a user is reading an eBook, the system can automatically adjust the content according to the user' s gaze positions.
  • the present invention relates to the processing of images showing the face of a user as the user moves his/her head, to track the movement of the user's eyes in an accurate and reliable manner. According to the present invention, eye tracking is carried out using face detection and eye detection algorithms and combined with motion estimation algorithm to accelerate tracking performance.
  • Figure 1A illustrates a basic environment according to one embodiment of the present invention.
  • the user 10 is in front of a display device 12 which has a camera 11 to capture image data.
  • Image 13 is the captured data by the camera 11 and displayed on a display device 12.
  • Figure IB is the same as Figure 1A except for the detection result of identified area.
  • the rectangle 14 shows a face detection result.
  • Two crosses 15 shows detected eye's positions.
  • Figure 2 illustrates eye detection process. Images (a), (b) , and (c) illustrate detecting phases in one frame image data.
  • the image (a) is new input frame data; the image (b) shows obtaining face detection result, i.e.
  • image (c) shows obtaining eye's positions according to the detected face rectangle.
  • the images (a), (b) , and (c) comprise the entire process of one frame for the eye tracker system.
  • Image (d) illustrates current frame face position information to forecast the next frame face position (the dashed rectangle) using moving estimate algorithm (such as a block-match algorithm and the like) .
  • moving estimate algorithm such as a block-match algorithm and the like
  • the forecast face position may deviate somewhat from the frame real face area. Some mechanism is necessary to guarantee the detection result.
  • Figure 3 is the flow chart of eye tracker system work process .
  • the system determines if a new frame of image data comes. If YES, then the processing proceeds to the step 303; if NO, the processing proceeds to the end.
  • the system checks whether there is identified moving block information. When previous image data is stored, the system determines the result of the check to be "YES". When the current frame is the first one and there is no previous data, the system determines the result of the check to be "NO”. If YES, then the processing proceeds to step 305.
  • the system can skip face detection and forecast face position using moving block for motion estimation. If NO at step 303, the system cannot skip face detection.
  • the system detects face position by face detection algorithm.
  • the system detects eye positions by eye detection algorithm.
  • the system certificates the eye positions according to threshold.
  • the threshold is set by a certify algorithm using statistic method.
  • step 313 If the system certificates eye positions at step 311, then the processing proceeds to step 313. If the system does not certificate eye positions at step 311, i.e. the detect result deviates from a normal model acquired by the certify algorithm, the result is deleted and the face detection is restarted in the next frame. If the next frame of image data does not come at step 301, the processing proceeds to END.
  • FIG 4 illustrates an exemplary block diagram of a system (410) according to an embodiment of the present invention.
  • the system 410 can be a 3D TV set, computer system, tablet, portable game, smart-phone, or the like.
  • the system 410 comprises a CPU (Central Processing Unit) 411, a camera 412, a storage 413, a display 414, and a user input module 415.
  • a memory 416 such as RAM (Random Access Memory) may be connected to the CPU 411 as shown in Figure 4.
  • RAM Random Access Memory
  • the camera 412 is an element for capturing the left and right images with a single lens.
  • the CPU 411 processes the steps as explained above.
  • the display 414 is configured to visually present text, image, video, and any other contents to a user of the system 410.
  • the display 414 can apply any type that is compatible with 3D contents.
  • the storage 413 is configured to store software programs and data for the CPU 511 to drive and operate the process as explained above.
  • the user input module 415 may include keys or buttons to input characters or commands, and also comprises a function for recognizing the characters or commands input with the keys or buttons.
  • the user input module 415 can be omitted in the system depending on use application of the system.
  • teachings of the present principles may be implemented in various forms of hardware, software, firmware, special purpose processors, or combinations thereof. Most preferably, the teachings of the present principles are implemented as a combination of hardware and software Moreover, the software may be implemented as an application program tangibly embodied on a program storage unit.
  • the application program may be uploaded to, and executed by, a machine comprising any suitable architecture.
  • the machine is implemented on a computer platform having hardware such as one or more central processing units (“CPU”) , a random access memory (“RAM”), and input/output (“I/O”) interfaces.
  • CPU central processing units
  • RAM random access memory
  • I/O input/output
  • the computer platform may also include an operating system and microinstruction code.
  • the various processes and functions described herein may be either part of the microinstruction code or part of the application program, or any combination thereof, which may be executed by a CPU.
  • various other peripheral units may be connected to the computer platform such as an additional data storage unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Ophthalmology & Optometry (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention concerne un procédé de détection de positions des yeux, lequel procédé consiste à recevoir des données d'image d'une trame, à déterminer au moins un bloc identifié dans la trame, à vérifier s'il existe des informations sur le bloc identifié, s'il existe des informations, à calculer la position de visage sur la base des informations au moyen d'un algorithme de détection de visage, et à détecter les positions des yeux au moyen d'un algorithme de détection des yeux.
PCT/CN2013/074273 2013-04-16 2013-04-16 Procédé et système d'oculométrie utilisant une combinaison de détection et d'estimation du mouvement WO2014169441A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2013/074273 WO2014169441A1 (fr) 2013-04-16 2013-04-16 Procédé et système d'oculométrie utilisant une combinaison de détection et d'estimation du mouvement

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2013/074273 WO2014169441A1 (fr) 2013-04-16 2013-04-16 Procédé et système d'oculométrie utilisant une combinaison de détection et d'estimation du mouvement

Publications (1)

Publication Number Publication Date
WO2014169441A1 true WO2014169441A1 (fr) 2014-10-23

Family

ID=51730678

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2013/074273 WO2014169441A1 (fr) 2013-04-16 2013-04-16 Procédé et système d'oculométrie utilisant une combinaison de détection et d'estimation du mouvement

Country Status (1)

Country Link
WO (1) WO2014169441A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104732202A (zh) * 2015-02-12 2015-06-24 杭州电子科技大学 一种人眼检测中消除眼镜框影响的方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101216885A (zh) * 2008-01-04 2008-07-09 中山大学 一种基于视频的行人人脸检测与跟踪算法
CN102004905A (zh) * 2010-11-18 2011-04-06 无锡中星微电子有限公司 人脸认证方法及装置
CN102193621A (zh) * 2010-03-17 2011-09-21 三星电子（中国）研发中心 基于视觉的交互式电子设备控制系统及其控制方法
WO2012138828A2 (fr) * 2011-04-08 2012-10-11 The Trustees Of Columbia University In The City Of New York Approche par filtre de kalman pour augmenter un suivi d'objet

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101216885A (zh) * 2008-01-04 2008-07-09 中山大学 一种基于视频的行人人脸检测与跟踪算法
CN102193621A (zh) * 2010-03-17 2011-09-21 三星电子（中国）研发中心 基于视觉的交互式电子设备控制系统及其控制方法
CN102004905A (zh) * 2010-11-18 2011-04-06 无锡中星微电子有限公司 人脸认证方法及装置
WO2012138828A2 (fr) * 2011-04-08 2012-10-11 The Trustees Of Columbia University In The City Of New York Approche par filtre de kalman pour augmenter un suivi d'objet

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104732202A (zh) * 2015-02-12 2015-06-24 杭州电子科技大学 一种人眼检测中消除眼镜框影响的方法

Similar Documents

Publication Publication Date Title
US11423695B2 (en) Face location tracking method, apparatus, and electronic device
CN109635621B (zh) 用于第一人称视角中基于深度学习识别手势的系统和方法
US8432357B2 (en) Tracking object selection apparatus, method, program and circuit
US10068135B2 (en) Face detection, identification, and tracking system for robotic devices
CN107808111B (zh) 用于行人检测和姿态估计的方法和装置
KR101706365B1 (ko) 이미지 세그멘테이션 방법 및 이미지 세그멘테이션 장치
US10810438B2 (en) Setting apparatus, output method, and non-transitory computer-readable storage medium
CN107251096B (zh) 图像捕获装置和方法
US10255504B2 (en) Object position tracking using motion estimation
US10254831B2 (en) System and method for detecting a gaze of a viewer
US9047504B1 (en) Combined cues for face detection in computing devices
JP2019186955A (ja) 情報処理システム、情報処理方法及びプログラム
US20160349511A1 (en) See-through binocular head mounted device
CN112927259A (zh) 基于多相机的裸手追踪显示方法、装置及系统
CN108875506B (zh) 人脸形状点跟踪方法、装置和系统及存储介质
CN112749655A (zh) 视线追踪方法、装置、计算机设备和存储介质
CN108229281B (zh) 神经网络的生成方法和人脸检测方法、装置及电子设备
US20220122341A1 (en) Target detection method and apparatus, electronic device, and computer storage medium
WO2014169441A1 (fr) Procédé et système d'oculométrie utilisant une combinaison de détection et d'estimation du mouvement
CN114740975A (zh) 目标内容的获取方法及相关设备
CN111611941A (zh) 特效处理方法及相关设备
CN113807150A (zh) 数据处理、姿态预测方法、设备及存储介质
EP4026092A1 (fr) Mode de verrouillage de scène pour capturer des images de caméra
CN110210306B (zh) 一种人脸跟踪方法和相机
CN116980744B (zh) 基于特征的摄像头追踪方法、装置、电子设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13882182

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13882182

Country of ref document: EP

Kind code of ref document: A1