CN101833770A - Driver eye movement characteristics handover detecting and tracing method based on light sensing - Google Patents

Driver eye movement characteristics handover detecting and tracing method based on light sensing Download PDF

Info

Publication number
CN101833770A
CN101833770A CN 201010172720 CN201010172720A CN101833770A CN 101833770 A CN101833770 A CN 101833770A CN 201010172720 CN201010172720 CN 201010172720 CN 201010172720 A CN201010172720 A CN 201010172720A CN 101833770 A CN101833770 A CN 101833770A
Authority
CN
China
Prior art keywords
driver
eye movement
noise
human eye
eye
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN 201010172720
Other languages
Chinese (zh)
Other versions
CN101833770B (en
Inventor
张家树
张祖涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southwest Jiaotong University
Original Assignee
Southwest Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southwest Jiaotong University filed Critical Southwest Jiaotong University
Priority to CN2010101727208A priority Critical patent/CN101833770B/en
Publication of CN101833770A publication Critical patent/CN101833770A/en
Application granted granted Critical
Publication of CN101833770B publication Critical patent/CN101833770B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Abstract

The invention discloses a driver eye movement characteristics handover detecting and tracing method based on light sensing, comprising the following steps: detecting illuminance by a light sensor, positioning a high illuminance condition by a Harr human eye detection method, and positioning a low illuminance condition by a bi-dimensional orthotropic Log-Gabor human eye detection method started by a computer; maintaining all-day position without influence of illuminance; and adding two dimensional parameters of system noise and observation noise on the basis of an uniform model after positioning, performing logarithmic transform on the two dimensional parameters, establishing a human eye non-linear model and a sampling strong tracking non-linear filter, adjusting system process noise and observation noise covariance matrix by fading factors, improving the non-linear strong tracking capability and model state robustness of a UKF filter, adapting to driver normal emergent operation reaction, and improving real time property of an algorithm by adopting a sampling strategy that (n+2) numbered sampling points perform strong tracking on Sigma point in the UKF filter.

Description

Driver eye movement characteristics change detection and tracking based on light sensing
Technical field
The present invention relates to a kind of image detection and tracking, relate in particular to driver's human eye detection and eye movement tracking under a kind of driving environment.
Technical background
Along with science and technology and socioeconomic fast development, whole world automobile pollution is when rapidly increasing, and pernicious road traffic accident also is synchronous increase trend.Statistics of traffic accidents according to European and American countries the analysis showed that: in numerous traffic hazards, have 80~90%, driver's human factor causes, and fatigue driving is wherein the easiest " first killer " who causes major traffic accidents.The statistics of American National highway traffic safety administration, annual wherein have 1500 approximately and directly cause death owing to the driver enters nearly 100,000 of U.S.'s road traffic accident that sleep state causes in driving procedure, and 7.1 ten thousand accidents cause personal injury.Add up according to Chinese transportation accident yearbook, China takes place particularly serious because of the traffic hazard that the driver artificial origin causes, unexpectedly reached 92.69% in 2005, bring massive losses for China's national wealth and people's lives and properties, the expenditure of whole no better than education and scientific research funds is equivalent to 2~3% of GDP then.Show that by theoretical research human perception objective world has the information more than 90% to obtain by eyes, and driver's eye moving process (comprises the motion of the whole eyes that cause because of facial movement; And the motion of eyes itself, as watch attentively, sweep, close one's eyes, blink) directly reflected driver's state of attention.Whether whether therefore, carry out based on the driver eye movement video tracking, analyze driver's notice situation, pass judgment on driver's notice and concentrate, be that fatigue driving has become the mainstream technology that reduces traffic hazard, be one of the frontier nature problem in automobile active safety field.In a word, carrying out human eye location and eye movement video tracking under the driving environment, is to analyze driver's driving condition, effectively prevents the prerequisite of driver tired driving, reduction traffic hazard and casualties rate.
At present, the driver is carried out video human eye location to existing researchist and eye movement is followed the tracks of, and carried out some researchs, and obtained certain achievement.Mainly contain based drive method, based on the eye movement track algorithm of feature and template with based on the eye movement track algorithm of nonlinear filtering theory.They all expose following problem demanding prompt solutions in actual applications: 1, difficulty is followed the tracks of in eye movement under the illumination condition at night; 2, complexity height, the computing time of eye movement track algorithm is long under the actual driving environment, and real-time is poor; 3, driver's normal emergency operation often causes eye movement to follow the tracks of failure, and its robustness and accuracy are relatively poor.Therefore, be necessary to invent a kind of eye movement tracking that illumination effect, computation complexity are little, can adapt to the high robust of the normal emergency operation reaction of driver that is not subjected to.
Summary of the invention
The object of the present invention is to provide a kind of driver eye movement characteristics change detection and tracking based on light sensing, this method be not subjected to illumination effect, computation complexity little, can adapt to the normal emergency operation of driver and react, have higher robustness and accuracy.
The object of the present invention is achieved like this: based on the driver eye movement characteristics change detection and the tracking of light sensing, its step is as follows:
A, feature extracting method switch and the location:
In the driving procedure, adopt the USB camera to obtain driver's face-image, and detect the illuminance of driving environment with light sensor;
As detected illuminance 〉=4Lux (lux) and when keeping more than 1 minute, launch computer Harr eye detection method location, that is: according to Harr feature separate on horizontal direction and the vertical direction, the eigenwert of the driver's face-image that utilizes the integral image method to calculate fast to obtain, according to eigenwert, locate human eye fast again with human eye Region Segmentation method;
As light luminance<4Lux (lux) and when keeping more than 1 minute, computer then starts two-dimensional quadrature Log-Gabor eye detection method location, that is: adopt the phase encoding method of two-dimensional quadrature Log-Gabor filtering, driver's face-image is extracted the quadrature phase feature of its human eye, the location human eye;
B, eye movement follow the tracks of: increase the scale parameter of tracker noise and observation noise on model based at the uniform velocity, and two scale parameters that will increase do log-transformation, carry out the eye movement modeling; Adopt UKF (Unscented kalman filtering) filtering algorithm, the factor that will fade is adjusted the systematic procedure noise and the observation noise variance battle array of UKF filtering, and adopt n+2 Sigma point sampling strategy (n is the dimension of Unscented kalman filtering), the driver under the driving environment is carried out eye movement follow the tracks of.
Above eye movement tracking, the applicant is with the strong non-linear filtering method of following the tracks of of its called after sampling.
Compared with prior art, income effect of the present invention is:
One, can realize driver's human eye location under round-the-clock, the various illumination condition effectively:
Adopt light sensing to carry out the illumination hand-off process at the human eye detection pretreatment stage, at illuminance 〉=4Lux (lux) and when keeping (being generally daytime) more than 1 minute, start the Harr eye detection method and can fast and effeciently carry out driver's human eye location, real-time is good, the accuracy height;
At light luminance<4Lux (lux) and when keeping (being generally night) more than 1 minute, start two-dimensional quadrature Log-Gabor human eye detection algorithm, utilize two-dimentional Log-Gabor wave filter always not have DC component (DC), the feature of extracting is not subjected to the influence of illumination condition, and by extracting the quadrature phase feature of eye image, and calculating that need not a plurality of trend pass filterings can be avoided the influence of illumination to driver's human eye detection effectively, has higher real-time, robustness and accuracy again.
Two, eye movement tracking phase, the present invention proposes a kind of new sampling and follows the tracks of non-linear filtering method by force, can effectively carry out the non-linear eye movement of driver and follow the tracks of.
Propose the non-linear eye movement model of driver, solved the modeling problem that the non-linear eye movement of driver is followed the tracks of.This model is in eye movement is followed the tracks of, scale parameter by drawing-in system noise and observation noise, trace model is just non-linear from becoming of linearity, noise model also becomes non-Gauss model, make this model have online in real time and follow the tracks of the advantage that nonlinear state is estimated by force, driver's nonlinear mutation eye movement like this (i.e. the eyes mass motion that produces based on driver's facial movement), as the sudden change eye movement that the normal emergency operation because of emergency case produces, also can be corresponding by Nonlinear Tracking.
In eye movement is followed the tracks of, propose the strong non-linear filtering method of following the tracks of of sampling and carry out tracing of human eye.That is: with factor Adjustment System process noise and the observation noise variance battle array of fading, improve the non-linear strong tracking power of UKF wave filter and the robustness of model state.And adopt n+2 Sigma point sampling strategy, and sampled point is reduced to n+2 from 2n+1 is individual, reduced the complexity of algorithm, guaranteeing to follow the tracks of under accuracy and the robustness condition, reduce calculated amount, the real-time performance of raising algorithm.
In a word, driver's human eye detection of the present invention is not subjected to illumination effect, can carry out round-the-clock driver's human eye location and follow the tracks of, and can adapt to the normal emergency operation reaction of driver, and robustness, real-time and accuracy are preferably arranged.
Emulation and on-the-spot test experiment illustrate that also the inventive method has higher robustness, accuracy and real-time:
At positioning stage: the inventive method reaches by day that the human eye correct localization all can reach 99.98% under the riving condition at night.On the contrary, locate as carry out human eye with the Harr algorithm under the driving environment at night, its accuracy only is 95.21%, can not satisfy application need.
At tracking phase: tracking accuracy of the present invention is up to 99.85%, on algorithm complex, n+2 of the present invention sampling be strong followed the tracks of the nonlinear filtering tracking in sample calculation on the time, and circulation primary of the present invention needs 0.015 second, far below 0.047 second of UKF filtering algorithm.
On robustness, RMSE of the present invention (root-mean-square error) is 0.17, MSE (square error) is 0.056, is far smaller than the RMSE (1.05) and the MSE (0.21) of standard UKF algorithm, illustrates that this algorithm has higher robustness.
The present invention is further detailed explanation below in conjunction with accompanying drawing and concrete embodiment.
Description of drawings
Fig. 1 is the step schematic block diagram of the embodiment of the invention.
During Fig. 2 in the embodiment of the invention based on the human eye of harr algorithm location synoptic diagram.
Fig. 3 is that the human eye based on two-dimensional quadrature Log-Gabor filtering is located synoptic diagram in the embodiment of the invention.
Embodiment
Embodiment
Fig. 1 illustrates, a kind of driver eye movement characteristics change detection and tracking based on light sensing, and its step is as follows:
A, feature extracting method switch and the location:
In the driving procedure, adopt the USB camera to obtain driver's face-image, and detect the illuminance of driving environment with light sensor;
As detected illuminance 〉=4Lux (lux) and when keeping more than 1 minute, launch computer Harr eye detection method location, that is: according to Harr feature separate on horizontal direction and the vertical direction, the eigenwert of the driver's face-image that utilizes the integral image method to calculate fast to obtain, according to eigenwert, locate human eye fast again with human eye Region Segmentation method;
As light luminance<4Lux (lux) and when keeping more than 1 minute, computer then starts two-dimensional quadrature Log-Gabor eye detection method location, that is: adopt the phase encoding method of two-dimensional quadrature Log-Gabor filtering, driver's face-image is extracted the quadrature phase feature of its human eye, the location human eye;
B, eye movement follow the tracks of: increase the scale parameter of tracker noise and observation noise on model based at the uniform velocity, and two scale parameters that will increase do log-transformation, carry out the eye movement modeling; Adopt UKF (Unscented kalman filtering) filtering algorithm, the factor that will fade is adjusted the systematic procedure noise and the observation noise variance battle array of UKF filtering, and adopt n+2 Sigma point sampling strategy (n is the dimension of Unscented kalman filtering), the driver under the driving environment is carried out eye movement follow the tracks of.
This eye movement tracking, the strong non-linear filtering method of following the tracks of of applicant's called after sampling.
Wherein, the at the uniform velocity scale parameter of increase tracker noise and observation noise on the model based, and two scale parameters that will increase is done log-transformation, carries out the eye movement modeling, and the model that obtains is:
x k=[x k,y k,x k-1,y k-1,lnρ 2 k,lnλ 2 k] T
ρ wherein k, λ kThe k that is respectively increase constantly system noise and the scale parameter of observation noise, x kBe k state vector constantly, y kBe k observed quantity constantly, T is a transposition.
The computer simulation experiment of present embodiment method is as follows:
One, real-time performance
The simulation experiment result shows, circulation primary of the present invention (promptly finish whole position fixing process to begin follow the tracks of) only needs 0.015 second, and far below standard UKF filtering algorithm 0.047 second has higher real-time performance.(in the experiment, the dimension n of UKF filtering gets 4)
Two, positioning performance
Table 1 is the inventive method and human eye location simulation at night test findings based on the Harr feature.
Figure GDA0000021408110000051
The video image frame number of correct localization=correctly the detect driver's position of human eye/frame number that detects the facial video image of driver that is useful on.
Table 1 shows, at the human eye positioning stage, and method of the present invention when keeping more than 1 minute (light luminance<4Lux (lux) and) under the night riving condition, human eye location (detection) accuracy can reach 99.98%; And single Harr characterization method, its human eye correct localization rate only is 95.21%, can not satisfy round-the-clock human eye positioning requirements.
Three, robustness
Table 2 is that the RMSE (root-mean-square error) and the MSE (square error) of different trackings compares.
Table 2 shows that RMSE of the inventive method (0.17) and MSE (0.056) are far smaller than standard UKF algorithm, simplify the RMSE and the MSE of Sigma point UKF algorithm and self-adaptation UKF algorithm, illustrate that this algorithm has higher robustness.
Four, follow the tracks of accuracy
Table 3 is that the driver eye movement of algorithms of different under the real driving environment is followed the tracks of Simulation results.
Figure GDA0000021408110000061
Annotate: the frame number of following the tracks of all videos in the video frame number/tracing process that correctly traces into driver's human eye in accuracy=tracing process.
Above table 3 shows, at the driver eye movement tracking phase: the tracking accuracy of this algorithm is 99.85%, is higher than based on 92.5% of Kalman filtering algorithm, is higher than 99.30% of UKF filtering algorithm, and the accuracy of visible the example method is higher.

Claims (2)

1. driver eye movement characteristics change detection and tracking based on a light sensing, its step is as follows:
A, feature extracting method switch and the location:
In the driving procedure, adopt the USB camera to obtain driver's face-image, and detect the illuminance of driving environment with light sensor;
As detected illuminance 〉=4Lux (lux) and when keeping more than 1 minute, launch computer Harr eye detection method location, that is: according to Harr feature separate on horizontal direction and the vertical direction, the eigenwert of the driver's face-image that utilizes the integral image method to calculate fast to obtain, according to eigenwert, locate human eye fast again with human eye Region Segmentation method;
As light luminance<4Lux (lux) and when keeping more than 1 minute, computer then starts two-dimensional quadrature Log-Gabor eye detection method location, that is: adopt the phase encoding method of two-dimensional quadrature Log-Gabor filtering, driver's face-image is extracted the quadrature phase feature of its human eye, the location human eye;
B, eye movement follow the tracks of: increase the scale parameter of tracker noise and observation noise on model based at the uniform velocity, and two scale parameters that will increase do log-transformation, carry out the eye movement modeling; Adopt UKF (Unscented kalman filtering) filtering algorithm, the factor that will fade is adjusted the systematic procedure noise and the observation noise variance battle array of UKF filtering, and adopt n+2 Sigma point sampling strategy (n is the dimension of Unscented kalman filtering), the driver under the driving environment is carried out eye movement follow the tracks of.
2. driver eye movement characteristics change detection and tracking based on light sensing according to claim 1, it is characterized in that: the scale parameter that on model based at the uniform velocity, increases tracker noise and observation noise, and two scale parameters that will increase are done log-transformation, carry out the eye movement modeling, the model that obtains is:
x k=[x k,y k,x k-1,y k-1,lnρ 2 k,lnλ 2 k] T
ρ wherein k, λ kThe k that is respectively increase constantly system noise and the scale parameter of observation noise, x kBe k state vector constantly, y kBe k observed quantity constantly, T is a transposition.
CN2010101727208A 2010-05-17 2010-05-17 Driver eye movement characteristics handover detecting and tracing method based on light sensing Expired - Fee Related CN101833770B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2010101727208A CN101833770B (en) 2010-05-17 2010-05-17 Driver eye movement characteristics handover detecting and tracing method based on light sensing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2010101727208A CN101833770B (en) 2010-05-17 2010-05-17 Driver eye movement characteristics handover detecting and tracing method based on light sensing

Publications (2)

Publication Number Publication Date
CN101833770A true CN101833770A (en) 2010-09-15
CN101833770B CN101833770B (en) 2012-01-25

Family

ID=42717831

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2010101727208A Expired - Fee Related CN101833770B (en) 2010-05-17 2010-05-17 Driver eye movement characteristics handover detecting and tracing method based on light sensing

Country Status (1)

Country Link
CN (1) CN101833770B (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102609984A (en) * 2012-02-02 2012-07-25 西南交通大学 Method for 3D-reconstructing and tracking eyes of driver based on orthogonal binocular dimension reduction space
CN103942542A (en) * 2014-04-18 2014-07-23 重庆卓美华视光电有限公司 Human eye tracking method and device
CN104704380A (en) * 2012-10-05 2015-06-10 日本康奈可株式会社 Device for estimating parameters of battery, and estimation method
CN104881635A (en) * 2015-05-05 2015-09-02 昆明理工大学 Image texture extraction and identification method by non-Gauss two-dimension Gabor filter
CN107392153A (en) * 2017-07-24 2017-11-24 中国科学院苏州生物医学工程技术研究所 Human-body fatigue degree decision method
CN109447122A (en) * 2018-09-28 2019-03-08 浙江大学 It is a kind of distribution fusion structure in strong tracking fading factor calculation method
CN111292548A (en) * 2020-02-06 2020-06-16 温州大学 Safe driving method based on visual attention

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030142041A1 (en) * 2002-01-30 2003-07-31 Delphi Technologies, Inc. Eye tracking/HUD system
CN101286195A (en) * 2008-06-03 2008-10-15 西南交通大学 High precision palm print recognition method based on two-dimensional quadrature Log-Gabor filtering

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030142041A1 (en) * 2002-01-30 2003-07-31 Delphi Technologies, Inc. Eye tracking/HUD system
CN101286195A (en) * 2008-06-03 2008-10-15 西南交通大学 High precision palm print recognition method based on two-dimensional quadrature Log-Gabor filtering

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
《IEEE Transactions on Pattern Analysis and Machine Intelligence》 19931130 John G. Daugman High Confidence Visual Recognition of Persons by a Test of Statistical Independence 全文 1-2 第15卷, 第11期 2 *
《International Journal of Computer Science and Network Security》 20090131 Zutao Zhang et al A Novel Vehicle Safety Model : Vehicle speed Controller under Driver Fatigue 全文 1-2 第9卷, 第1期 2 *
《Proceedings of the American Control Conference》 20020510 Simon J. Julier et al Reduced Sigma Point Filters for the Propagation of Means and Covariances Through Nonlinear Transformations 全文 1-2 , 2 *
《西南交通大学学报》 20081231 张祖涛 等 基于UKF非线性人眼跟踪的驾驶员疲劳检测 全文 1-2 第43卷, 第6期 2 *
《铁道学报》 20090430 张祖涛 等 结合UKF和小波变换的改进粒子滤波及其在机车驾驶员人眼跟踪中的应用 全文 1-2 第31卷, 第2期 2 *

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102609984A (en) * 2012-02-02 2012-07-25 西南交通大学 Method for 3D-reconstructing and tracking eyes of driver based on orthogonal binocular dimension reduction space
CN104704380A (en) * 2012-10-05 2015-06-10 日本康奈可株式会社 Device for estimating parameters of battery, and estimation method
CN104704380B (en) * 2012-10-05 2016-11-23 日本康奈可株式会社 For estimating device and the method for estimation of battery parameter
CN103942542A (en) * 2014-04-18 2014-07-23 重庆卓美华视光电有限公司 Human eye tracking method and device
CN104881635A (en) * 2015-05-05 2015-09-02 昆明理工大学 Image texture extraction and identification method by non-Gauss two-dimension Gabor filter
CN104881635B (en) * 2015-05-05 2018-03-06 昆明理工大学 Non-gaussian two-dimensional Gabor filter image texture extracts and recognition methods
CN107392153A (en) * 2017-07-24 2017-11-24 中国科学院苏州生物医学工程技术研究所 Human-body fatigue degree decision method
CN107392153B (en) * 2017-07-24 2020-09-29 中国科学院苏州生物医学工程技术研究所 Human body fatigue degree judging method
CN109447122A (en) * 2018-09-28 2019-03-08 浙江大学 It is a kind of distribution fusion structure in strong tracking fading factor calculation method
CN109447122B (en) * 2018-09-28 2021-07-13 浙江大学 Strong tracking fading factor calculation method in distributed fusion structure
CN111292548A (en) * 2020-02-06 2020-06-16 温州大学 Safe driving method based on visual attention
CN111292548B (en) * 2020-02-06 2021-02-05 温州大学 Safe driving method based on visual attention

Also Published As

Publication number Publication date
CN101833770B (en) 2012-01-25

Similar Documents

Publication Publication Date Title
CN101833770B (en) Driver eye movement characteristics handover detecting and tracing method based on light sensing
Wang Research and application of traffic sign detection and recognition based on deep learning
CN106407931B (en) A kind of depth convolutional neural networks moving vehicle detection method
CN103413395B (en) Flue gas intelligent detecting prewarning method and device
CN101639894B (en) Method for detecting train driver behavior and fatigue state on line and detection system thereof
CN101950355B (en) Method for detecting fatigue state of driver based on digital video
CN101425136B (en) Method and device for acquiring partial binary mode characteristic of video image
CN103886307A (en) Sight tracking and fatigue early warning method
CN109875568A (en) A kind of head pose detection method for fatigue driving detection
CN103465857A (en) Mobile-phone-based active safety early-warning method for automobile
CN103426179A (en) Target tracking method and system based on mean shift multi-feature fusion
CN104224204A (en) Driver fatigue detection system on basis of infrared detection technology
CN109993122A (en) A kind of pedestrian based on depth convolutional neural networks multiplies staircase anomaly detection method
CN107038423A (en) A kind of vehicle is detected and tracking in real time
CN105184229A (en) Online learning based real-time pedestrian detection method in dynamic scene
CN105516661B (en) Principal and subordinate's target monitoring method that fisheye camera is combined with ptz camera
CN105718904A (en) Blind people detection and identification method and system based on combined characteristics and vehicle-mounted cameras
CN101320477A (en) Human body tracing method and equipment thereof
CN108304764B (en) Fatigue state detection device and detection method in simulated flight driving process
CN110176128A (en) A kind of driver tired driving alarming processing system
CN109766780A (en) A kind of ship smog emission on-line checking and method for tracing based on deep learning
Cui et al. Skip Connection YOLO Architecture for Noise Barrier Defect Detection Using UAV-Based Images in High-Speed Railway
CN103136511B (en) Behavioral value method and device
Jiang et al. Fusion of texture features and color information of the license plate location algorithm
Zhang et al. Surface defect detection of rails based on convolutional neural network multi-scale-cross fastflow

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
C17 Cessation of patent right
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20120125

Termination date: 20140517