WO2011067788A3 - A cost effective and robust system and method for eye tracking and driver drowsiness identification - Google Patents

A cost effective and robust system and method for eye tracking and driver drowsiness identification Download PDF

Info

Publication number
WO2011067788A3
WO2011067788A3 PCT/IN2010/000781 IN2010000781W WO2011067788A3 WO 2011067788 A3 WO2011067788 A3 WO 2011067788A3 IN 2010000781 W IN2010000781 W IN 2010000781W WO 2011067788 A3 WO2011067788 A3 WO 2011067788A3
Authority
WO
WIPO (PCT)
Prior art keywords
histogram
target
eyes
tracking
model histogram
Prior art date
Application number
PCT/IN2010/000781
Other languages
French (fr)
Other versions
WO2011067788A2 (en
Inventor
K. S. Chidanand
Brojeshwar Bhowmick
Original Assignee
Tata Consultancy Services Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tata Consultancy Services Limited filed Critical Tata Consultancy Services Limited
Priority to EP10822865.1A priority Critical patent/EP2507742A2/en
Priority to JP2012541628A priority patent/JP5680667B2/en
Priority to CN201080059278.6A priority patent/CN102696041B/en
Priority to US13/513,495 priority patent/US9483695B2/en
Publication of WO2011067788A2 publication Critical patent/WO2011067788A2/en
Publication of WO2011067788A3 publication Critical patent/WO2011067788A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/19—Sensors therefor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/163—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state by tracking eye movement, gaze, or pupil change
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/18—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G06V40/193—Preprocessing; Feature extraction
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02—Alarms for ensuring the safety of persons
    • G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Ophthalmology & Optometry (AREA)
  • Multimedia (AREA)
  • Animal Behavior & Ethology (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • Veterinary Medicine (AREA)
  • General Engineering & Computer Science (AREA)
  • Hospice & Palliative Care (AREA)
  • Developmental Disabilities (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Psychology (AREA)
  • Child & Adolescent Psychology (AREA)
  • Educational Technology (AREA)
  • Pathology (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Traffic Control Systems (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
  • Emergency Alarm Devices (AREA)

Abstract

A cost-effective and robust method for localizing and tracking drowsiness state of the eyes of driver by using images captured by near infrared camera disposed on the vehicle, the said method comprising the processor implemented steps of: Real-time tracking of the face and localizing eye bounding box within the face bounding box in the captured image by comparing the gray values with threshold using the segmentation process; tracking the eyes by computing centroid of the eye, target model histogram and target candidate model histogram for one location to another by comparing them to identify distance and calculating the displacement of the target centre by weighted means, wherein the target model histogram and target candidate model histogram are computed based on the feature space; and detecting the drowsiness state of the eyes using histogram equalization, Morphological operations and texture based parameters using histogram and grey level co-occurrence matrices.
PCT/IN2010/000781 2009-12-02 2010-12-02 A cost effective and robust system and method for eye tracking and driver drowsiness identification WO2011067788A2 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
EP10822865.1A EP2507742A2 (en) 2009-12-02 2010-12-02 A cost effective and robust system and method for eye tracking and driver drowsiness identification
JP2012541628A JP5680667B2 (en) 2009-12-02 2010-12-02 System and method for identifying driver wakefulness
CN201080059278.6A CN102696041B (en) 2009-12-02 2010-12-02 The system and method that the cost benefit confirmed for eye tracking and driver drowsiness is high and sane
US13/513,495 US9483695B2 (en) 2009-12-02 2010-12-02 Cost effective and robust system and method for eye tracking and driver drowsiness identification

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IN2784MU2009 2009-12-02
IN2784/MUM/2009 2009-12-02

Publications (2)

Publication Number Publication Date
WO2011067788A2 WO2011067788A2 (en) 2011-06-09
WO2011067788A3 true WO2011067788A3 (en) 2011-10-06

Family

ID=44115374

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IN2010/000781 WO2011067788A2 (en) 2009-12-02 2010-12-02 A cost effective and robust system and method for eye tracking and driver drowsiness identification

Country Status (5)

Country Link
US (1) US9483695B2 (en)
EP (1) EP2507742A2 (en)
JP (1) JP5680667B2 (en)
CN (1) CN102696041B (en)
WO (1) WO2011067788A2 (en)

Families Citing this family (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9460601B2 (en) * 2009-09-20 2016-10-04 Tibet MIMAR Driver distraction and drowsiness warning and sleepiness reduction for accident avoidance
US10922567B2 (en) 2010-06-07 2021-02-16 Affectiva, Inc. Cognitive state based vehicle manipulation using near-infrared image processing
KR101046677B1 (en) * 2011-03-15 2011-07-06 동국대학교 산학협력단 Methods for tracking position of eyes and medical head lamp using thereof
EP2827761A4 (en) 2012-03-22 2015-11-04 Univ Missouri Device to measure pupillary light reflex in infants and toddlers
US9050035B2 (en) 2012-03-22 2015-06-09 The Curators Of The University Of Missouri Device to measure pupillary light reflex in infants and toddlers
CN103411535B (en) * 2013-08-07 2015-08-05 北京信息科技大学 A kind of Changeable weight picture point localization method for retro-reflective target
CN103955695B (en) * 2013-11-27 2017-07-07 苏州清研微视电子科技有限公司 The method that computer is based on human eye state in gray level co-occurrence matrixes energy variation Intelligent Recognition video
JP6442942B2 (en) * 2014-09-11 2018-12-26 株式会社デンソー Driver status determination device
EP3040726A1 (en) * 2014-12-29 2016-07-06 General Electric Company Method and system to determine vehicle speed
US10984237B2 (en) * 2016-11-10 2021-04-20 Neurotrack Technologies, Inc. Method and system for correlating an image capturing device to a human user for analyzing gaze information associated with cognitive performance
EP3562394B1 (en) * 2016-12-28 2023-07-26 Ficosa Adas, S.L.U. Respiratory signal extraction
US10290158B2 (en) * 2017-02-03 2019-05-14 Ford Global Technologies, Llc System and method for assessing the interior of an autonomous vehicle
US10121084B2 (en) * 2017-03-07 2018-11-06 Wipro Limited Method and a system for detecting drowsiness state of a vehicle user
US10509974B2 (en) 2017-04-21 2019-12-17 Ford Global Technologies, Llc Stain and trash detection systems and methods
US10304165B2 (en) 2017-05-12 2019-05-28 Ford Global Technologies, Llc Vehicle stain and trash detection systems and methods
US20190012552A1 (en) * 2017-07-06 2019-01-10 Yves Lambert Hidden driver monitoring
TWI647666B (en) * 2017-08-28 2019-01-11 緯創資通股份有限公司 Drowsiness detection apparatus and drowsiness detection method thereof
CN107977623A (en) * 2017-11-30 2018-05-01 睿视智觉（深圳）算法技术有限公司 A kind of robustness human eye state determination methods
US10943092B2 (en) 2018-05-23 2021-03-09 ClairLabs Ltd. Monitoring system
CN109086740B (en) * 2018-08-25 2019-08-23 上海首安工程技术有限公司 Stagewise ferry safety check mechanism
CN109584303B (en) * 2018-12-03 2023-04-14 电子科技大学 Infrared weak and small target detection method based on Lp norm and nuclear norm
US11200438B2 (en) 2018-12-07 2021-12-14 Dus Operating Inc. Sequential training method for heterogeneous convolutional neural network
CN109886780B (en) * 2019-01-31 2022-04-08 苏州经贸职业技术学院 Commodity target detection method and device based on eyeball tracking
US11068069B2 (en) * 2019-02-04 2021-07-20 Dus Operating Inc. Vehicle control with facial and gesture recognition using a convolutional neural network
CN110264670A (en) * 2019-06-24 2019-09-20 广州鹰瞰信息科技有限公司 Based on passenger stock tired driver driving condition analytical equipment
CN110400274B (en) * 2019-07-19 2022-02-15 西安科技大学 Infrared image enhancement method for vehicle-mounted infrared pedestrian detection
EP4033961A1 (en) * 2019-09-27 2022-08-03 Alcon Inc. Patient-induced trigger of a measurement for ophthalmic diagnostic devices
CN112764524A (en) * 2019-11-05 2021-05-07 沈阳智能机器人国家研究院有限公司 Myoelectric signal gesture action recognition method based on texture features
WO2021156989A1 (en) * 2020-02-06 2021-08-12 三菱電機株式会社 Sign sensing device, operation assistance control device, and sign sensing method
CN113807126B (en) * 2020-06-12 2024-07-16 广州汽车集团股份有限公司 Fatigue driving detection method and system, computer equipment and storage medium thereof
JP7048997B2 (en) * 2020-06-26 2022-04-06 みこらった株式会社 Programs for self-driving cars and self-driving cars
US11763595B2 (en) * 2020-08-27 2023-09-19 Sensormatic Electronics, LLC Method and system for identifying, tracking, and collecting data on a person of interest
US11861916B2 (en) * 2021-10-05 2024-01-02 Yazaki Corporation Driver alertness monitoring system
CN114758403B (en) * 2022-06-10 2022-09-13 武汉憬然智能技术有限公司 Intelligent analysis method and device for fatigue driving

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6717518B1 (en) * 1998-01-15 2004-04-06 Holding B.E.V.S.A. Method and apparatus for detection of drowsiness
EP1867281A1 (en) * 2006-06-13 2007-12-19 Delphi Technologies, Inc. Improved dynamic eye tracking system

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5345281A (en) 1992-12-17 1994-09-06 John Taboada Eye tracking system and method
US5481622A (en) * 1994-03-01 1996-01-02 Rensselaer Polytechnic Institute Eye tracking apparatus and method employing grayscale threshold values
US5689241A (en) 1995-04-24 1997-11-18 Clarke, Sr.; James Russell Sleep detection and driver alert apparatus
JPH1086696A (en) * 1996-09-13 1998-04-07 Toyota Motor Corp Method for detecting characteristic part in face image
US6283954B1 (en) 1998-04-21 2001-09-04 Visx, Incorporated Linear array eye tracker
US5900819A (en) 1998-04-21 1999-05-04 Meritor Heavy Vehicle Systems, Llc Drowsy driver detection system
JP2003006654A (en) * 2001-06-20 2003-01-10 Nippon Telegr & Teleph Corp <Ntt> Method and device for extracting feature amount of moving object in moving image and method and device for automatically tracking moving object in moving image, program for executing the method and recording medium with the program recorded thereon
US6927694B1 (en) 2001-08-20 2005-08-09 Research Foundation Of The University Of Central Florida Algorithm for monitoring head/eye motion for driver alertness with one camera
US7130446B2 (en) 2001-12-03 2006-10-31 Microsoft Corporation Automatic detection and tracking of multiple individuals using multiple cues
CN1275185C (en) * 2002-06-30 2006-09-13 贺贵明 Driver's face image identification and alarm device and method
JP2004356683A (en) * 2003-05-27 2004-12-16 Fuji Photo Film Co Ltd Image management system
US7430315B2 (en) * 2004-02-13 2008-09-30 Honda Motor Co. Face recognition system
US7362885B2 (en) 2004-04-20 2008-04-22 Delphi Technologies, Inc. Object tracking and eye state identification method
JP4501003B2 (en) * 2005-07-15 2010-07-14 国立大学法人静岡大学 Face posture detection system
US8265392B2 (en) * 2006-02-07 2012-09-11 Qualcomm Incorporated Inter-mode region-of-interest video object segmentation
JP4895874B2 (en) * 2007-03-15 2012-03-14 アイシン精機株式会社 Eye state determination device, eye state determination method, and eye state determination program
JP4307496B2 (en) * 2007-03-19 2009-08-05 株式会社豊田中央研究所 Facial part detection device and program
JP4898532B2 (en) 2007-04-13 2012-03-14 富士フイルム株式会社 Image processing apparatus, photographing system, blink state detection method, blink state detection program, and recording medium on which the program is recorded
CN101375796B (en) * 2008-09-18 2010-06-02 浙江工业大学 Real-time detection system of fatigue driving

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6717518B1 (en) * 1998-01-15 2004-04-06 Holding B.E.V.S.A. Method and apparatus for detection of drowsiness
EP1867281A1 (en) * 2006-06-13 2007-12-19 Delphi Technologies, Inc. Improved dynamic eye tracking system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
TABRIZI P R ET AL: "Open/Closed Eye Analysis for Drowsiness Detection", IMAGE PROCESSING THEORY, TOOLS AND APPLICATIONS, 2008. IPTA 2008. FIRST WORKSHOPS ON, IEEE, PISCATAWAY, NJ, USA, 23 November 2008 (2008-11-23), pages 1 - 7, XP031404056, ISBN: 978-1-4244-3321-6 *
TOCK, D. AND CRAW, I.: "Blink Rate Monitoring for a Driver Awareness System", PROCEEDINGS OF THE BRITISH MACHINE VISION CONFERENCE (BMVC), 1992, XP002651999, Retrieved from the Internet <URL:http://www.bmva.org/bmvc/1992/bmvc-92-054.pdf> [retrieved on 20110721] *

Also Published As

Publication number Publication date
US9483695B2 (en) 2016-11-01
WO2011067788A2 (en) 2011-06-09
US20130010096A1 (en) 2013-01-10
CN102696041A (en) 2012-09-26
CN102696041B (en) 2016-03-02
JP5680667B2 (en) 2015-03-04
EP2507742A2 (en) 2012-10-10
JP2013513155A (en) 2013-04-18

Similar Documents

Publication Publication Date Title
WO2011067788A3 (en) A cost effective and robust system and method for eye tracking and driver drowsiness identification
US11643076B2 (en) Forward collision control method and apparatus, electronic device, program, and medium
US9607227B2 (en) Boundary detection apparatus and boundary detection method
Ikemura et al. Real-time human detection using relational depth similarity features
CA2775757C (en) Method of processing body inspection image and body inspection apparatus
CN105460009B (en) Automobile control method and device
CN102930334B (en) Video recognition counter for body silhouette
CN109334563A (en) A kind of anticollision method for early warning based on road ahead pedestrian and bicyclist
CN111553214B (en) Method and system for detecting smoking behavior of driver
CN103902985B (en) High-robustness real-time lane detection algorithm based on ROI
CN106919895B (en) Tracking method and system for moving object
Bhujbal et al. Lane departure warning system based on Hough transform and Euclidean distance
CN105469054B (en) The model building method of normal behaviour and the detection method of abnormal behaviour
US20150161447A1 (en) Vision based pedestrian and cyclist detection method
CN105740751A (en) Object detection and identification method and system
CN102201120A (en) Multifeature-based target object contour detection method
CN105354856A (en) Human matching and positioning method and system based on MSER and ORB
CN106881716A (en) Human body follower method and system based on 3D cameras robot
Chang et al. An efficient method for lane-mark extraction in complex conditions
CN108021849B (en) Pedestrian early warning method and device
CN107133600A (en) A kind of real-time lane line detection method based on intra-frame trunk
CN108596057A (en) A kind of Information Security Management System based on recognition of face
Lin et al. A vision-based obstacle detection system for parking assistance
Cheng et al. Vehicle tracking in daytime and nighttime traffic surveillance videos
CN106339666B (en) A kind of night monitoring method of human body target

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10822865

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2012541628

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 2010822865

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 13513495

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE