CN101833770A - Driver eye movement characteristics handover detecting and tracing method based on light sensing - Google Patents
Driver eye movement characteristics handover detecting and tracing method based on light sensing Download PDFInfo
- Publication number
- CN101833770A CN101833770A CN 201010172720 CN201010172720A CN101833770A CN 101833770 A CN101833770 A CN 101833770A CN 201010172720 CN201010172720 CN 201010172720 CN 201010172720 A CN201010172720 A CN 201010172720A CN 101833770 A CN101833770 A CN 101833770A
- Authority
- CN
- China
- Prior art keywords
- driver
- eye movement
- noise
- human eye
- eye
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention discloses a driver eye movement characteristics handover detecting and tracing method based on light sensing, comprising the following steps: detecting illuminance by a light sensor, positioning a high illuminance condition by a Harr human eye detection method, and positioning a low illuminance condition by a bi-dimensional orthotropic Log-Gabor human eye detection method started by a computer; maintaining all-day position without influence of illuminance; and adding two dimensional parameters of system noise and observation noise on the basis of an uniform model after positioning, performing logarithmic transform on the two dimensional parameters, establishing a human eye non-linear model and a sampling strong tracking non-linear filter, adjusting system process noise and observation noise covariance matrix by fading factors, improving the non-linear strong tracking capability and model state robustness of a UKF filter, adapting to driver normal emergent operation reaction, and improving real time property of an algorithm by adopting a sampling strategy that (n+2) numbered sampling points perform strong tracking on Sigma point in the UKF filter.
Description
Technical field
The present invention relates to a kind of image detection and tracking, relate in particular to driver's human eye detection and eye movement tracking under a kind of driving environment.
Technical background
Along with science and technology and socioeconomic fast development, whole world automobile pollution is when rapidly increasing, and pernicious road traffic accident also is synchronous increase trend.Statistics of traffic accidents according to European and American countries the analysis showed that: in numerous traffic hazards, have 80~90%, driver's human factor causes, and fatigue driving is wherein the easiest " first killer " who causes major traffic accidents.The statistics of American National highway traffic safety administration, annual wherein have 1500 approximately and directly cause death owing to the driver enters nearly 100,000 of U.S.'s road traffic accident that sleep state causes in driving procedure, and 7.1 ten thousand accidents cause personal injury.Add up according to Chinese transportation accident yearbook, China takes place particularly serious because of the traffic hazard that the driver artificial origin causes, unexpectedly reached 92.69% in 2005, bring massive losses for China's national wealth and people's lives and properties, the expenditure of whole no better than education and scientific research funds is equivalent to 2~3% of GDP then.Show that by theoretical research human perception objective world has the information more than 90% to obtain by eyes, and driver's eye moving process (comprises the motion of the whole eyes that cause because of facial movement; And the motion of eyes itself, as watch attentively, sweep, close one's eyes, blink) directly reflected driver's state of attention.Whether whether therefore, carry out based on the driver eye movement video tracking, analyze driver's notice situation, pass judgment on driver's notice and concentrate, be that fatigue driving has become the mainstream technology that reduces traffic hazard, be one of the frontier nature problem in automobile active safety field.In a word, carrying out human eye location and eye movement video tracking under the driving environment, is to analyze driver's driving condition, effectively prevents the prerequisite of driver tired driving, reduction traffic hazard and casualties rate.
At present, the driver is carried out video human eye location to existing researchist and eye movement is followed the tracks of, and carried out some researchs, and obtained certain achievement.Mainly contain based drive method, based on the eye movement track algorithm of feature and template with based on the eye movement track algorithm of nonlinear filtering theory.They all expose following problem demanding prompt solutions in actual applications: 1, difficulty is followed the tracks of in eye movement under the illumination condition at night; 2, complexity height, the computing time of eye movement track algorithm is long under the actual driving environment, and real-time is poor; 3, driver's normal emergency operation often causes eye movement to follow the tracks of failure, and its robustness and accuracy are relatively poor.Therefore, be necessary to invent a kind of eye movement tracking that illumination effect, computation complexity are little, can adapt to the high robust of the normal emergency operation reaction of driver that is not subjected to.
Summary of the invention
The object of the present invention is to provide a kind of driver eye movement characteristics change detection and tracking based on light sensing, this method be not subjected to illumination effect, computation complexity little, can adapt to the normal emergency operation of driver and react, have higher robustness and accuracy.
The object of the present invention is achieved like this: based on the driver eye movement characteristics change detection and the tracking of light sensing, its step is as follows:
A, feature extracting method switch and the location:
In the driving procedure, adopt the USB camera to obtain driver's face-image, and detect the illuminance of driving environment with light sensor;
As detected illuminance 〉=4Lux (lux) and when keeping more than 1 minute, launch computer Harr eye detection method location, that is: according to Harr feature separate on horizontal direction and the vertical direction, the eigenwert of the driver's face-image that utilizes the integral image method to calculate fast to obtain, according to eigenwert, locate human eye fast again with human eye Region Segmentation method;
As light luminance<4Lux (lux) and when keeping more than 1 minute, computer then starts two-dimensional quadrature Log-Gabor eye detection method location, that is: adopt the phase encoding method of two-dimensional quadrature Log-Gabor filtering, driver's face-image is extracted the quadrature phase feature of its human eye, the location human eye;
B, eye movement follow the tracks of: increase the scale parameter of tracker noise and observation noise on model based at the uniform velocity, and two scale parameters that will increase do log-transformation, carry out the eye movement modeling; Adopt UKF (Unscented kalman filtering) filtering algorithm, the factor that will fade is adjusted the systematic procedure noise and the observation noise variance battle array of UKF filtering, and adopt n+2 Sigma point sampling strategy (n is the dimension of Unscented kalman filtering), the driver under the driving environment is carried out eye movement follow the tracks of.
Above eye movement tracking, the applicant is with the strong non-linear filtering method of following the tracks of of its called after sampling.
Compared with prior art, income effect of the present invention is:
One, can realize driver's human eye location under round-the-clock, the various illumination condition effectively:
Adopt light sensing to carry out the illumination hand-off process at the human eye detection pretreatment stage, at illuminance 〉=4Lux (lux) and when keeping (being generally daytime) more than 1 minute, start the Harr eye detection method and can fast and effeciently carry out driver's human eye location, real-time is good, the accuracy height;
At light luminance<4Lux (lux) and when keeping (being generally night) more than 1 minute, start two-dimensional quadrature Log-Gabor human eye detection algorithm, utilize two-dimentional Log-Gabor wave filter always not have DC component (DC), the feature of extracting is not subjected to the influence of illumination condition, and by extracting the quadrature phase feature of eye image, and calculating that need not a plurality of trend pass filterings can be avoided the influence of illumination to driver's human eye detection effectively, has higher real-time, robustness and accuracy again.
Two, eye movement tracking phase, the present invention proposes a kind of new sampling and follows the tracks of non-linear filtering method by force, can effectively carry out the non-linear eye movement of driver and follow the tracks of.
Propose the non-linear eye movement model of driver, solved the modeling problem that the non-linear eye movement of driver is followed the tracks of.This model is in eye movement is followed the tracks of, scale parameter by drawing-in system noise and observation noise, trace model is just non-linear from becoming of linearity, noise model also becomes non-Gauss model, make this model have online in real time and follow the tracks of the advantage that nonlinear state is estimated by force, driver's nonlinear mutation eye movement like this (i.e. the eyes mass motion that produces based on driver's facial movement), as the sudden change eye movement that the normal emergency operation because of emergency case produces, also can be corresponding by Nonlinear Tracking.
In eye movement is followed the tracks of, propose the strong non-linear filtering method of following the tracks of of sampling and carry out tracing of human eye.That is: with factor Adjustment System process noise and the observation noise variance battle array of fading, improve the non-linear strong tracking power of UKF wave filter and the robustness of model state.And adopt n+2 Sigma point sampling strategy, and sampled point is reduced to n+2 from 2n+1 is individual, reduced the complexity of algorithm, guaranteeing to follow the tracks of under accuracy and the robustness condition, reduce calculated amount, the real-time performance of raising algorithm.
In a word, driver's human eye detection of the present invention is not subjected to illumination effect, can carry out round-the-clock driver's human eye location and follow the tracks of, and can adapt to the normal emergency operation reaction of driver, and robustness, real-time and accuracy are preferably arranged.
Emulation and on-the-spot test experiment illustrate that also the inventive method has higher robustness, accuracy and real-time:
At positioning stage: the inventive method reaches by day that the human eye correct localization all can reach 99.98% under the riving condition at night.On the contrary, locate as carry out human eye with the Harr algorithm under the driving environment at night, its accuracy only is 95.21%, can not satisfy application need.
At tracking phase: tracking accuracy of the present invention is up to 99.85%, on algorithm complex, n+2 of the present invention sampling be strong followed the tracks of the nonlinear filtering tracking in sample calculation on the time, and circulation primary of the present invention needs 0.015 second, far below 0.047 second of UKF filtering algorithm.
On robustness, RMSE of the present invention (root-mean-square error) is 0.17, MSE (square error) is 0.056, is far smaller than the RMSE (1.05) and the MSE (0.21) of standard UKF algorithm, illustrates that this algorithm has higher robustness.
The present invention is further detailed explanation below in conjunction with accompanying drawing and concrete embodiment.
Description of drawings
Fig. 1 is the step schematic block diagram of the embodiment of the invention.
During Fig. 2 in the embodiment of the invention based on the human eye of harr algorithm location synoptic diagram.
Fig. 3 is that the human eye based on two-dimensional quadrature Log-Gabor filtering is located synoptic diagram in the embodiment of the invention.
Embodiment
Embodiment
Fig. 1 illustrates, a kind of driver eye movement characteristics change detection and tracking based on light sensing, and its step is as follows:
A, feature extracting method switch and the location:
In the driving procedure, adopt the USB camera to obtain driver's face-image, and detect the illuminance of driving environment with light sensor;
As detected illuminance 〉=4Lux (lux) and when keeping more than 1 minute, launch computer Harr eye detection method location, that is: according to Harr feature separate on horizontal direction and the vertical direction, the eigenwert of the driver's face-image that utilizes the integral image method to calculate fast to obtain, according to eigenwert, locate human eye fast again with human eye Region Segmentation method;
As light luminance<4Lux (lux) and when keeping more than 1 minute, computer then starts two-dimensional quadrature Log-Gabor eye detection method location, that is: adopt the phase encoding method of two-dimensional quadrature Log-Gabor filtering, driver's face-image is extracted the quadrature phase feature of its human eye, the location human eye;
B, eye movement follow the tracks of: increase the scale parameter of tracker noise and observation noise on model based at the uniform velocity, and two scale parameters that will increase do log-transformation, carry out the eye movement modeling; Adopt UKF (Unscented kalman filtering) filtering algorithm, the factor that will fade is adjusted the systematic procedure noise and the observation noise variance battle array of UKF filtering, and adopt n+2 Sigma point sampling strategy (n is the dimension of Unscented kalman filtering), the driver under the driving environment is carried out eye movement follow the tracks of.
This eye movement tracking, the strong non-linear filtering method of following the tracks of of applicant's called after sampling.
Wherein, the at the uniform velocity scale parameter of increase tracker noise and observation noise on the model based, and two scale parameters that will increase is done log-transformation, carries out the eye movement modeling, and the model that obtains is:
x
k=[x
k,y
k,x
k-1,y
k-1,lnρ
2 k,lnλ
2 k]
T
ρ wherein
k, λ
kThe k that is respectively increase constantly system noise and the scale parameter of observation noise, x
kBe k state vector constantly, y
kBe k observed quantity constantly, T is a transposition.
The computer simulation experiment of present embodiment method is as follows:
One, real-time performance
The simulation experiment result shows, circulation primary of the present invention (promptly finish whole position fixing process to begin follow the tracks of) only needs 0.015 second, and far below standard UKF filtering algorithm 0.047 second has higher real-time performance.(in the experiment, the dimension n of UKF filtering gets 4)
Two, positioning performance
Table 1 is the inventive method and human eye location simulation at night test findings based on the Harr feature.
The video image frame number of correct localization=correctly the detect driver's position of human eye/frame number that detects the facial video image of driver that is useful on.
Table 1 shows, at the human eye positioning stage, and method of the present invention when keeping more than 1 minute (light luminance<4Lux (lux) and) under the night riving condition, human eye location (detection) accuracy can reach 99.98%; And single Harr characterization method, its human eye correct localization rate only is 95.21%, can not satisfy round-the-clock human eye positioning requirements.
Three, robustness
Table 2 is that the RMSE (root-mean-square error) and the MSE (square error) of different trackings compares.
Table 2 shows that RMSE of the inventive method (0.17) and MSE (0.056) are far smaller than standard UKF algorithm, simplify the RMSE and the MSE of Sigma point UKF algorithm and self-adaptation UKF algorithm, illustrate that this algorithm has higher robustness.
Four, follow the tracks of accuracy
Table 3 is that the driver eye movement of algorithms of different under the real driving environment is followed the tracks of Simulation results.
Annotate: the frame number of following the tracks of all videos in the video frame number/tracing process that correctly traces into driver's human eye in accuracy=tracing process.
Above table 3 shows, at the driver eye movement tracking phase: the tracking accuracy of this algorithm is 99.85%, is higher than based on 92.5% of Kalman filtering algorithm, is higher than 99.30% of UKF filtering algorithm, and the accuracy of visible the example method is higher.
Claims (2)
1. driver eye movement characteristics change detection and tracking based on a light sensing, its step is as follows:
A, feature extracting method switch and the location:
In the driving procedure, adopt the USB camera to obtain driver's face-image, and detect the illuminance of driving environment with light sensor;
As detected illuminance 〉=4Lux (lux) and when keeping more than 1 minute, launch computer Harr eye detection method location, that is: according to Harr feature separate on horizontal direction and the vertical direction, the eigenwert of the driver's face-image that utilizes the integral image method to calculate fast to obtain, according to eigenwert, locate human eye fast again with human eye Region Segmentation method;
As light luminance<4Lux (lux) and when keeping more than 1 minute, computer then starts two-dimensional quadrature Log-Gabor eye detection method location, that is: adopt the phase encoding method of two-dimensional quadrature Log-Gabor filtering, driver's face-image is extracted the quadrature phase feature of its human eye, the location human eye;
B, eye movement follow the tracks of: increase the scale parameter of tracker noise and observation noise on model based at the uniform velocity, and two scale parameters that will increase do log-transformation, carry out the eye movement modeling; Adopt UKF (Unscented kalman filtering) filtering algorithm, the factor that will fade is adjusted the systematic procedure noise and the observation noise variance battle array of UKF filtering, and adopt n+2 Sigma point sampling strategy (n is the dimension of Unscented kalman filtering), the driver under the driving environment is carried out eye movement follow the tracks of.
2. driver eye movement characteristics change detection and tracking based on light sensing according to claim 1, it is characterized in that: the scale parameter that on model based at the uniform velocity, increases tracker noise and observation noise, and two scale parameters that will increase are done log-transformation, carry out the eye movement modeling, the model that obtains is:
x
k=[x
k,y
k,x
k-1,y
k-1,lnρ
2 k,lnλ
2 k]
T
ρ wherein
k, λ
kThe k that is respectively increase constantly system noise and the scale parameter of observation noise, x
kBe k state vector constantly, y
kBe k observed quantity constantly, T is a transposition.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2010101727208A CN101833770B (en) | 2010-05-17 | 2010-05-17 | Driver eye movement characteristics handover detecting and tracing method based on light sensing |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2010101727208A CN101833770B (en) | 2010-05-17 | 2010-05-17 | Driver eye movement characteristics handover detecting and tracing method based on light sensing |
Publications (2)
Publication Number | Publication Date |
---|---|
CN101833770A true CN101833770A (en) | 2010-09-15 |
CN101833770B CN101833770B (en) | 2012-01-25 |
Family
ID=42717831
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN2010101727208A Expired - Fee Related CN101833770B (en) | 2010-05-17 | 2010-05-17 | Driver eye movement characteristics handover detecting and tracing method based on light sensing |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN101833770B (en) |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102609984A (en) * | 2012-02-02 | 2012-07-25 | 西南交通大学 | Method for 3D-reconstructing and tracking eyes of driver based on orthogonal binocular dimension reduction space |
CN103942542A (en) * | 2014-04-18 | 2014-07-23 | 重庆卓美华视光电有限公司 | Human eye tracking method and device |
CN104704380A (en) * | 2012-10-05 | 2015-06-10 | 日本康奈可株式会社 | Device for estimating parameters of battery, and estimation method |
CN104881635A (en) * | 2015-05-05 | 2015-09-02 | 昆明理工大学 | Image texture extraction and identification method by non-Gauss two-dimension Gabor filter |
CN107392153A (en) * | 2017-07-24 | 2017-11-24 | 中国科学院苏州生物医学工程技术研究所 | Human-body fatigue degree decision method |
CN109447122A (en) * | 2018-09-28 | 2019-03-08 | 浙江大学 | It is a kind of distribution fusion structure in strong tracking fading factor calculation method |
CN111292548A (en) * | 2020-02-06 | 2020-06-16 | 温州大学 | Safe driving method based on visual attention |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030142041A1 (en) * | 2002-01-30 | 2003-07-31 | Delphi Technologies, Inc. | Eye tracking/HUD system |
CN101286195A (en) * | 2008-06-03 | 2008-10-15 | 西南交通大学 | High precision palm print recognition method based on two-dimensional quadrature Log-Gabor filtering |
-
2010
- 2010-05-17 CN CN2010101727208A patent/CN101833770B/en not_active Expired - Fee Related
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030142041A1 (en) * | 2002-01-30 | 2003-07-31 | Delphi Technologies, Inc. | Eye tracking/HUD system |
CN101286195A (en) * | 2008-06-03 | 2008-10-15 | 西南交通大学 | High precision palm print recognition method based on two-dimensional quadrature Log-Gabor filtering |
Non-Patent Citations (5)
Title |
---|
《IEEE Transactions on Pattern Analysis and Machine Intelligence》 19931130 John G. Daugman High Confidence Visual Recognition of Persons by a Test of Statistical Independence 全文 1-2 第15卷, 第11期 2 * |
《International Journal of Computer Science and Network Security》 20090131 Zutao Zhang et al A Novel Vehicle Safety Model : Vehicle speed Controller under Driver Fatigue 全文 1-2 第9卷, 第1期 2 * |
《Proceedings of the American Control Conference》 20020510 Simon J. Julier et al Reduced Sigma Point Filters for the Propagation of Means and Covariances Through Nonlinear Transformations 全文 1-2 , 2 * |
《西南交通大学学报》 20081231 张祖涛 等 基于UKF非线性人眼跟踪的驾驶员疲劳检测 全文 1-2 第43卷, 第6期 2 * |
《铁道学报》 20090430 张祖涛 等 结合UKF和小波变换的改进粒子滤波及其在机车驾驶员人眼跟踪中的应用 全文 1-2 第31卷, 第2期 2 * |
Cited By (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102609984A (en) * | 2012-02-02 | 2012-07-25 | 西南交通大学 | Method for 3D-reconstructing and tracking eyes of driver based on orthogonal binocular dimension reduction space |
CN104704380A (en) * | 2012-10-05 | 2015-06-10 | 日本康奈可株式会社 | Device for estimating parameters of battery, and estimation method |
CN104704380B (en) * | 2012-10-05 | 2016-11-23 | 日本康奈可株式会社 | For estimating device and the method for estimation of battery parameter |
CN103942542A (en) * | 2014-04-18 | 2014-07-23 | 重庆卓美华视光电有限公司 | Human eye tracking method and device |
CN104881635A (en) * | 2015-05-05 | 2015-09-02 | 昆明理工大学 | Image texture extraction and identification method by non-Gauss two-dimension Gabor filter |
CN104881635B (en) * | 2015-05-05 | 2018-03-06 | 昆明理工大学 | Non-gaussian two-dimensional Gabor filter image texture extracts and recognition methods |
CN107392153A (en) * | 2017-07-24 | 2017-11-24 | 中国科学院苏州生物医学工程技术研究所 | Human-body fatigue degree decision method |
CN107392153B (en) * | 2017-07-24 | 2020-09-29 | 中国科学院苏州生物医学工程技术研究所 | Human body fatigue degree judging method |
CN109447122A (en) * | 2018-09-28 | 2019-03-08 | 浙江大学 | It is a kind of distribution fusion structure in strong tracking fading factor calculation method |
CN109447122B (en) * | 2018-09-28 | 2021-07-13 | 浙江大学 | Strong tracking fading factor calculation method in distributed fusion structure |
CN111292548A (en) * | 2020-02-06 | 2020-06-16 | 温州大学 | Safe driving method based on visual attention |
CN111292548B (en) * | 2020-02-06 | 2021-02-05 | 温州大学 | Safe driving method based on visual attention |
Also Published As
Publication number | Publication date |
---|---|
CN101833770B (en) | 2012-01-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN101833770B (en) | Driver eye movement characteristics handover detecting and tracing method based on light sensing | |
Wang | Research and application of traffic sign detection and recognition based on deep learning | |
CN106407931B (en) | A kind of depth convolutional neural networks moving vehicle detection method | |
CN103413395B (en) | Flue gas intelligent detecting prewarning method and device | |
CN101639894B (en) | Method for detecting train driver behavior and fatigue state on line and detection system thereof | |
CN101950355B (en) | Method for detecting fatigue state of driver based on digital video | |
CN101425136B (en) | Method and device for acquiring partial binary mode characteristic of video image | |
CN103886307A (en) | Sight tracking and fatigue early warning method | |
CN109875568A (en) | A kind of head pose detection method for fatigue driving detection | |
CN103465857A (en) | Mobile-phone-based active safety early-warning method for automobile | |
CN103426179A (en) | Target tracking method and system based on mean shift multi-feature fusion | |
CN104224204A (en) | Driver fatigue detection system on basis of infrared detection technology | |
CN109993122A (en) | A kind of pedestrian based on depth convolutional neural networks multiplies staircase anomaly detection method | |
CN107038423A (en) | A kind of vehicle is detected and tracking in real time | |
CN105184229A (en) | Online learning based real-time pedestrian detection method in dynamic scene | |
CN105516661B (en) | Principal and subordinate's target monitoring method that fisheye camera is combined with ptz camera | |
CN105718904A (en) | Blind people detection and identification method and system based on combined characteristics and vehicle-mounted cameras | |
CN101320477A (en) | Human body tracing method and equipment thereof | |
CN108304764B (en) | Fatigue state detection device and detection method in simulated flight driving process | |
CN110176128A (en) | A kind of driver tired driving alarming processing system | |
CN109766780A (en) | A kind of ship smog emission on-line checking and method for tracing based on deep learning | |
Cui et al. | Skip Connection YOLO Architecture for Noise Barrier Defect Detection Using UAV-Based Images in High-Speed Railway | |
CN103136511B (en) | Behavioral value method and device | |
Jiang et al. | Fusion of texture features and color information of the license plate location algorithm | |
Zhang et al. | Surface defect detection of rails based on convolutional neural network multi-scale-cross fastflow |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
C17 | Cessation of patent right | ||
CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120125 Termination date: 20140517 |