CN107292251A - A kind of Driver Fatigue Detection and system based on human eye state - Google Patents
A kind of Driver Fatigue Detection and system based on human eye state Download PDFInfo
- Publication number
- CN107292251A CN107292251A CN201710430629.3A CN201710430629A CN107292251A CN 107292251 A CN107292251 A CN 107292251A CN 201710430629 A CN201710430629 A CN 201710430629A CN 107292251 A CN107292251 A CN 107292251A
- Authority
- CN
- China
- Prior art keywords
- image
- driver
- eyes
- face
- eye
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Ophthalmology & Optometry (AREA)
- Geometry (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Traffic Control Systems (AREA)
- Emergency Alarm Devices (AREA)
Abstract
The invention discloses a kind of Driver Fatigue Detection based on human eye state and system, with reference to Face datection, face tracking, determine the preliminary rectangular extent of eye of driver's image, pre-processed for the preliminary rectangular extent of eye, searched using profile and rectangle fitting positions eyes exact rectangular range image, judge that driver is eye opening or eye closing in frame driver's image according to upright projection, the continuous frame number for being in closed-eye state according to driver in continuous driver's image judges driver's blink or continued eye closure, and calculate frequency of wink, so as to judge whether driver is in fatigue state.Beneficial effect：The different choice whether present invention has worn glasses and intensity of illumination for driver carries out different pretreatments to the preliminary rectangular extent image of eyes so that fatigue detection method of the invention is more accurate, and robustness is higher；For being all suitable under different illumination conditions, real-time is higher, and detection speed is faster.
Description
Technical field
It is tired more particularly, to a kind of driver based on human eye state the present invention relates to computer vision processing technology field
Labor detection method and system.
Background technology
Fatigue detecting (Fatigue Detecting) is, by monitoring the various fatigue characteristics of human body, tired shape to be found in time
State simultaneously provides pre-warning signal, is related to the multiple fields such as physiology, psychology, image procossing, motion tracking, pattern-recognition, is
One research topic that is complicated and having theoretical and realistic price concurrently simultaneously.In the driving procedure of long period, driver's
Degree of fatigue is gradually put aside, from the superficial to the deep.If driver's Jing Zhong states can in real time be detected using technological means,
Once there is tired sign to occur, early warning is just sent at once, then safe driving coefficient will be effectively improved.
The detection of driver fatigue state has the method for more research at present, can be roughly divided into by the classification of detection and be based on driving
Sail the detection of human physiology signal, the detection based on driver's operation behavior, the detection based on car status information and based on driving
The methods such as the detection of human physiology response feature.Wherein, the detection method based on driver's physiological reaction feature is contactless inspection
Survey, fatigue is judged using machine vision, measurement process will not be interfered to the normal driving behavior of driver, with very
Big development potentiality.But existing detection and analysis human eye state is so as to realize the method accuracy of driver fatigue state detection
Not enough, real-time is not strong.
The content of the invention
It is an object of the invention to overcome above-mentioned technical deficiency, a kind of driver fatigue detection based on human eye state is proposed
Method and system, solve above-mentioned technical problem of the prior art.
To reach above-mentioned technical purpose, technical scheme provides a kind of driver fatigue inspection based on human eye state
Survey method, including：
S1, in real time acquisition driver's image, and Face datection is carried out to driver's image, obtain the face square for including face
Shape frame, obtains the preliminary rectangular extent image of eyes and the preliminary rectangular extent image of eyes in face rectangle frame in face rectangle
Coordinate in frame；
After S2, the face rectangle frame of one frame driver's image of acquisition, face tracking is carried out for follow-up driver's image, is obtained
Take the face rectangle frame of follow-up driver's image；
S3, the face rectangle frame for follow-up driver's image, according to the preliminary rectangular extent image of eyes in face rectangle
Coordinate in frame, obtains the preliminary rectangular extent image of eyes of follow-up driver's image；
S4, successively cut for the preliminary rectangular extent image of eyes that S3 is obtained and binaryzation pretreatment；
S5, profile lookup is carried out to the preliminary rectangular extent image of eyes after binaryzation in S4, and utilize rectangle fitting essence
It is determined that position eyes exact rectangular range image, obtains coordinate of the eyes exact rectangular range image in face rectangle frame；
S6, according to coordinate of the eyes exact rectangular range image in face rectangle frame non-two are extracted from face rectangle frame
The eyes exact rectangular range image of value, and closed being formed to open after the eyes exact rectangular range image progress binaryzation of extraction
Eye judges image, will open and close eyes and judges that image carries out upright projection to X-axis, passes through black in upright projection and the ratio of white pixel
Example judges that driver is eye opening or eye closing in frame driver's image；
Driver is in the continuous frame number of closed-eye state in S7, the continuous driver's image of statistics, according to being continuously in
Scope residing for the frame number of closed-eye state judges that driver is blink or continued eye closure, and calculates frequency of wink；
If S8, frequency of wink are in outside the normal range (NR) of setting, judge that driver is in fatigue state, if driven
The person's of sailing continued eye is closed, then judges that driver is in fatigue state.
The present invention also provides a kind of Study in Driver Fatigue State Surveillance System based on human eye state, including：
Face detection module：Driver's image is obtained in real time, and Face datection is carried out to driver's image, is obtained and is included people
The face rectangle frame of face, obtains the preliminary rectangular extent image of eyes and the preliminary rectangular extent image of eyes in face rectangle frame
Coordinate in face rectangle frame；
Face tracking module：After the face rectangle frame for obtaining frame driver's image, carried out for follow-up driver's image
Face tracking, obtains the face rectangle frame of follow-up driver's image；
Eyes initial ranges image collection module：It is preliminary according to eyes for the face rectangle frame of follow-up driver's image
Coordinate of the rectangular extent image in face rectangle frame, obtains the preliminary rectangular extent image of eyes of follow-up driver's image；
Pretreatment module：For the preliminary rectangular extent image elder generation of eyes obtained in eyes initial ranges image collection module
Carry out cutting the pretreatment with binaryzation afterwards；
Eyes exact extension image collection module：To the preliminary rectangular extent image of eyes after binaryzation in pretreatment module
Profile lookup is carried out, and eyes exact rectangular range image is accurately positioned using rectangle fitting, eyes exact rectangular scope is obtained
Coordinate of the image in face rectangle frame；
Eye opening eye closing judge module：According to coordinate of the eyes exact rectangular range image in face rectangle frame from face square
The eyes exact rectangular range image of non-binaryzation is extracted in shape frame, and the eyes exact rectangular range image of extraction is carried out two
The judgement image that opens and closes eyes is formed after value, will open and close eyes and judge that image carries out upright projection to X-axis, pass through black in upright projection
Judge that driver is eye opening or eye closing in frame driver's image with the ratio of white pixel；
Blink is persistently closed one's eyes judge module：Count driver in continuous driver's image and be in the continuous of closed-eye state
Frame number, the scope according to residing for the continuous frame number in closed-eye state judges that driver is blink or continued eye closure, and counts
Calculate frequency of wink；
Tired judge module：If frequency of wink is in outside the normal range (NR) of setting, judge that driver is in fatigue
State, if driver's eyes are persistently closed, judges that driver is in fatigue state.
Compared with prior art, beneficial effects of the present invention include：Whether the present invention has worn glasses progress for driver
Detection, and different cuttings are carried out to the preliminary rectangular extent image of eyes according to whether wearing glasses, selected according to the difference of intensity of illumination
Select different average gray threshold values and binaryzation is carried out to the preliminary rectangular extent image of eyes so that fatigue detection method of the invention is more
Plus it is accurate, robustness is higher；For being all suitable under different illumination conditions, real-time is higher, and detection speed is faster.
Brief description of the drawings
Fig. 1 is a kind of Driver Fatigue Detection flow chart based on human eye state that the present invention is provided；
Fig. 2 is a kind of Study in Driver Fatigue State Surveillance System structured flowchart based on human eye state that the present invention is provided；
Fig. 3 is the face rectangle frame and the preliminary rectangular extent image schematic diagram of eyes of the present invention；
Fig. 4 is that the design sketch after rim detection is carried out for Glasses detection region；
The upright projection process schematic opened eyes, closed one's eyes when Fig. 5 is infrared light filling and does not open infrared light filling.
In accompanying drawing：1st, the Study in Driver Fatigue State Surveillance System based on human eye state, 11, face detection module, 12, face tracking
Module, 13, eyes initial ranges image collection module, 14, pretreatment module, 15, eyes exact extension image collection module,
16th, eye opening eye closing judge module, 17, blink persistently close one's eyes judge module, 18, tired judge module.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, it is right below in conjunction with drawings and Examples
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
The invention provides a kind of Driver Fatigue Detection based on human eye state, including：
S1, in real time acquisition driver's image, and Face datection is carried out to driver's image, obtain the face square for including face
Shape frame, obtains the preliminary rectangular extent image of eyes and the preliminary rectangular extent image of eyes in face rectangle frame in face rectangle
Coordinate in frame；
After S2, the face rectangle frame of one frame driver's image of acquisition, face tracking is carried out for follow-up driver's image, is obtained
Take the face rectangle frame of follow-up driver's image；
S3, the face rectangle frame for follow-up driver's image, according to the preliminary rectangular extent image of eyes in face rectangle
Coordinate in frame, obtains the preliminary rectangular extent image of eyes of follow-up driver's image；
S4, successively cut for the preliminary rectangular extent image of eyes that S3 is obtained and binaryzation pretreatment；
S5, profile lookup is carried out to the preliminary rectangular extent image of eyes after binaryzation in S4, and utilize rectangle fitting essence
It is determined that position eyes exact rectangular range image, obtains coordinate of the eyes exact rectangular range image in face rectangle frame；
S6, according to coordinate of the eyes exact rectangular range image in face rectangle frame non-two are extracted from face rectangle frame
The eyes exact rectangular range image of value, and closed being formed to open after the eyes exact rectangular range image progress binaryzation of extraction
Eye judges image, will open and close eyes and judges that image carries out upright projection to X-axis, passes through black in upright projection and the ratio of white pixel
Example judges that driver is eye opening or eye closing in frame driver's image；
Driver is in the continuous frame number of closed-eye state in S7, the continuous driver's image of statistics, according to being continuously in
Scope residing for the frame number of closed-eye state judges that driver is blink or continued eye closure, and calculates frequency of wink；
If S8, frequency of wink are in outside the normal range (NR) of setting, judge that driver is in fatigue state, if driven
The person's of sailing continued eye is closed, then judges that driver is in fatigue state.
In Driver Fatigue Detection of the present invention based on human eye state, step S1：
Driver's image is obtained using the infrared camera with infrared light filling function, infrared camera is provided with intensity of illumination
Inductor, when intensity of illumination inductor senses driving indoor illumination intensity higher than setting Intensity threshold, is not turned on infrared light filling
Function, obtains the coloured image of driver, and intensity of illumination inductor senses driving indoor illumination intensity less than setting light intensity threshold
During value, infrared light filling is automatically turned on, and obtain the infrared black white image of driver so that under different illumination intensity, can obtain
Get clear driver's image and analyze and process, strong adaptability, applicability is wide.
In Driver Fatigue Detection of the present invention based on human eye state, step S1：
Train grader to carry out Face datection to driver's image using Adaboost algorithm, get the people comprising face
Face rectangle frame, stasm positioning feature points are carried out to the face rectangle frame internal image got, so that eye feature point is recognized,
The image for obtaining the default size rectangular extent more than eye feature point range is the preliminary rectangular extent image of eyes；
As shown in figure 3, the preliminary rectangular extent image of eyes is to include eye feature point and the image of eyebrow characteristic point, it is eye
One approximate region image of eyeball, that is, to a Primary Location of eyes, the preliminary rectangle model of eyes in face rectangle frame
Enclosing image has two, the preliminary rectangular extent image of eyes of respectively left eye and the preliminary rectangular extent image of eyes of right eye, left
The preliminary rectangular extent image of eyes of eye and right eye is in the same size, in same level position.
In Driver Fatigue Detection of the present invention based on human eye state, step S2：
Utilize KCF (High-speed tracking with kernelized correlation filters) algorithm
Face tracking is carried out for follow-up driver's image, to producing evaluation index data after each frame driver image face tracking
To assess tracking effect, when evaluation index data are less than given threshold, tracking strategy is adjusted；
To producing an evaluation index data peak_value after driver's image progress KCF face trackings of each frame,
Peak_value values are 0 to 1 decimal, and value is bigger, and the confidence level for representing tracking result is higher, and tracking effect is better；Positive face situation
Under peak_value values it is larger, the peak_value values in the case of anon-normal face (turn to, put first-class) are smaller；For evaluation index
Data peak_value has preset a fixed threshold, when evaluation index size of data is higher than predetermined threshold value, judges to comment
The corresponding frame driver image of valency achievement data is positive face, when evaluation index size of data is less than or equal to predetermined threshold value,
Judge the corresponding frame driver image of evaluation index data for anon-normal face；When a frame driver image of tracking is anon-normal face
When, then do not continue to carry out face tracking, if continuing to track, because the drift in the case of anon-normal face is the most serious, can cause
Driver's image face tracking mistake of follow-up all frames, now should enter pedestrian at interval of 5 frames to subsequent frame driver's image
Face detects that can judgement obtain the face rectangle frame comprising face, if having got the face rectangle frame for including face, explanation
This frame driver's image is positive face (driver's image of anon-normal face is to can't detect face rectangle frame), proceeds by face
Tracking, otherwise, continues to be spaced 5 frames to subsequent frame driver image progress Face datection.
In Driver Fatigue Detection of the present invention based on human eye state, step S4：
Judge that whether driver wears glasses in driver's image, and the size of clipping region is selected to eye according to judged result
The preliminary rectangular extent image of eyeball is cut, further according to the average ash of different choice difference of intensity of illumination when obtaining driver's image
Spend threshold value and binaryzation is carried out to the preliminary rectangular extent image of eyes after cutting；
Judge driver in driver's image whether the method for wearing glasses：Determine Glasses detection region, Glasses detection area
Domain is that, using face rectangle width of frame as width, Glasses detection region is covered using the height of the preliminary rectangular extent image of eyes as height
The preliminary rectangular extent image of eyes of lid left eye and right eye；Picture frame detection zone is determined, picture frame detection zone is located at Glasses detection
In region, particularly between two, above the bridge of the nose, between can determining two according to stasm positioning feature points, the bridge of the nose
The position of top may thereby determine that picture frame detection zone；
As shown in figure 4, rim detection is carried out for the image in Glasses detection region, in the case where not wearing glasses,
Picture frame detection zone is almost without marginal information, in the case of wearing spectacles, and picture frame detection zone contains abundant edge letter
Cease (not considering transparent spectacle frame), because during wearing spectacles, the picture frame detection zone between two, above the bridge of the nose contains glasses
Frame, by counting the number of each row white pixel in rim detection rear mirror frame detection zone, if arranging white picture in the presence of continuous n
The number of element is all 0, then it is considered that a frame driver image driver does not wear glasses, otherwise it is assumed that having worn glasses, n is actual
Situation chooses suitable value.
In Driver Fatigue Detection of the present invention based on human eye state, step S4：
Because the influence of illumination, the preliminary rectangular extent of eyes of right and left eyes are easily brought in the left side of left eye and the right side of right eye
Image will be treated with a certain discrimination when cutting, and the subregion on the left of left eye and on the right side of right eye be dismissed respectively, and wear in driver
During glasses, expand and cut scope, the interference of picture frame is reduced as far as possible.
In Driver Fatigue Detection of the present invention based on human eye state, step S4：
Average gray threshold value is multiplied by Coefficient m for the average gray of the preliminary rectangular extent image of eyes after cutting, according to difference
Intensity of illumination, decide whether to open infrared light filling, so as to obtain the coloured image or infrared black white image of driver, driving
The person's of sailing image is colored or during infrared black white image, using different Coefficient ms, so that when realizing according to driver's image is obtained
The difference of intensity of illumination, two-value is carried out using different average gray threshold values to the preliminary rectangular extent image of eyes after cutting
Change；If some pixel gray value of the preliminary rectangular extent image of eyes is less than average gray threshold value, the pixel is set
Gray value is 255, and it is 0 otherwise to set the pixel gray value.
In Driver Fatigue Detection of the present invention based on human eye state, step S5：
Because the preliminary rectangular extent image of eyes is to include in eye feature point and the image of eyebrow characteristic point, S4 at the beginning of eyes
Walk after rectangular extent image binaryzation, be after being deeper than other parts, eyes and eyebrow binaryzation due to eyes and eyebrow color
White, other parts are black, carry out profile lookup to the preliminary rectangular extent image of eyes after binaryzation in S4, find all
The region of white, the geometrical relationship further according to eyes and eyebrow identifies that profile above is eyebrow, and profile below is eyes,
Determine after eye contour, the exact position image that eyes are positioned using rectangle fitting is eyes exact rectangular range image, is obtained
Take coordinate of the eyes exact rectangular range image in face rectangle frame.
In Driver Fatigue Detection of the present invention based on human eye state, step S6：
The eyes exact rectangular range image of extraction is carried out to form open and close eyes judgement image, binarization method after binaryzation
It is the method binaryzation using average gray threshold value, average gray threshold value is averaged for the eyes exact rectangular range image of extraction
Gray scale is multiplied by Coefficient m, is colored or during infrared black white image in driver's image, using different Coefficient ms, if eyes are smart
When some pixel gray value of true rectangular extent image is less than average gray threshold value, it is 255 to set the pixel gray value, no
It is 0 then to set the pixel gray value；
After the completion of such as Fig. 5, binaryzation, it will open and close eyes and judge that image carries out upright projection to X-axis, specifically：Closed if opened
Eye judges that the original driver's image of image is coloured image, then will open and close eyes and judge that image carries out upright projection, statistics to X-axis
Open and close eyes and judge the number of each row black picture element of image, whether undergone mutation between each row black picture element number of analysis, if
Undergo mutation, then it is assumed that open and close eyes and judge that driver opens eyes in image, otherwise it is assumed that driver closes one's eyes；Whether measurement undergos mutation
Method be：Upright projection is drawn with 4 straight lines of vertical X axis and is divided into 5 regions, each row black picture in each region is calculated
The average value of plain number, calculates the difference of 5 average values between any two, is more than difference threshold if there is a difference, then it is assumed that
Undergone mutation between each row black picture element number, otherwise it is assumed that not undergoing mutation；
Judge that the original driver's image of image is infrared image if opened and closed eyes, will open and close eyes and judge image to X-axis
Upright projection is carried out, statistics, which opens and closes eyes, judges the number of each row white pixel of image, if it find that each row white pixel
Number is 0, then it is assumed that opens and closes eyes and judges that driver closes one's eyes in image, otherwise it is assumed that driver opens eyes；
If it find that the number of each row white pixel is 0, then it is assumed that open and close eyes and judge driver's eye closing in image
Reason is verified by test of many times, under infrared environmental, during eye closing, opening after eyes exact rectangular range image binaryzation
Eye closing judges image to be completely black, will open and close eyes and judges that image carries out upright projection to X-axis, opens and closes eyes and judges each row white of image
The number of pixel is 0.
In Driver Fatigue Detection of the present invention based on human eye state, step S7：
The continuous frame number that driver in continuous driver's image is in closed-eye state is counted, and according to frame number and duration
Relation, such as camera collection driver image is 60 frames/second, calculates the corresponding continuous duration of continuous frame number, works as consecutive hours
It is long to be in 0.2-0.4 seconds, it is believed that driver is blinked, and calculates frequency of wink, and frequency of wink is blinked according to driver in one section of duration
Eye number of times is calculated, when continuous duration was more than 2 seconds, it is believed that driver's eyes are persistently closed.
In Driver Fatigue Detection of the present invention based on human eye state, step S8：
Frequency of wink is too low or too high, all shows the intensification of degree of fatigue, the state that the too low eyes of frequency of wink are opened
Duration is longer, illustrates that driver is One's eyesight is restrained, in absent-minded state, indicates showing for fatigue state；Frequency of wink mistake
It hurry up, illustrate that driver's eyes are dry and astringent or making great efforts to make oneself to keep clear-headed, show fatigue state；If at frequency of wink
Outside the normal range (NR) of setting, then judge that driver is in fatigue state, if driver's eyes are persistently closed, judgement is driven
The person of sailing is in fatigue state.
The present invention also provides a kind of Study in Driver Fatigue State Surveillance System 1 based on human eye state, including：
Face detection module 11：Driver's image is obtained in real time, and Face datection is carried out to driver's image, and acquisition is included
The face rectangle frame of face, obtains the preliminary rectangular extent image of eyes and the preliminary rectangular extent figure of eyes in face rectangle frame
As the coordinate in face rectangle frame；
Face tracking module 12：After the face rectangle frame for obtaining frame driver's image, enter for follow-up driver's image
Row face tracking, obtains the face rectangle frame of follow-up driver's image；
Eyes initial ranges image collection module 13：For the face rectangle frame of follow-up driver's image, according at the beginning of eyes
Coordinate of the rectangular extent image in face rectangle frame is walked, the preliminary rectangular extent image of eyes of follow-up driver's image is obtained；
Pretreatment module 14：For the preliminary rectangular extent image of eyes obtained in eyes initial ranges image collection module
Successively carry out cutting the pretreatment with binaryzation；
Eyes exact extension image collection module 15：To the preliminary rectangular extent figure of eyes after binaryzation in pretreatment module
Eyes exact rectangular range image is accurately positioned as carrying out profile lookup, and using rectangle fitting, eyes exact rectangular model is obtained
Enclose coordinate of the image in face rectangle frame；
Eye opening eye closing judge module 16：According to coordinate of the eyes exact rectangular range image in face rectangle frame from face
The eyes exact rectangular range image of non-binaryzation is extracted in rectangle frame, and the eyes exact rectangular range image of extraction is carried out
The judgement image that opens and closes eyes is formed after binaryzation, will open and close eyes and judge that image carries out upright projection to X-axis, by black in upright projection
The ratio of color and white pixel judges that driver is eye opening or eye closing in frame driver's image；
The lasting eye closing judge module 17 of blink：Count the company that driver in continuous driver's image is in closed-eye state
Continuous frame number, the scope according to residing for the continuous frame number in closed-eye state judges that driver is that blink or continued eye are closed, and
Calculate frequency of wink；
Tired judge module 18：If frequency of wink is in outside the normal range (NR) of setting, judge that driver is in tired
Labor state, if driver's eyes are persistently closed, judges that driver is in fatigue state.
In Study in Driver Fatigue State Surveillance System 1 of the present invention based on human eye state, face detection module 11：
Driver's image is obtained using the infrared camera with infrared light filling function, driving indoor illumination intensity, which is higher than, to be set
When determining Intensity threshold, the coloured image of driver is obtained, when driving indoor illumination intensity less than setting Intensity threshold, is opened infrared
Light filling, and obtain the infrared black white image of driver.
In Study in Driver Fatigue State Surveillance System 1 of the present invention based on human eye state, face detection module 11：
Train grader to carry out Face datection to driver's image using Adaboost algorithm, get the people comprising face
Face rectangle frame, stasm positioning feature points are carried out to the face rectangle frame internal image got, so that eye feature point is recognized,
The image for obtaining the default size rectangular extent more than eye feature point range is the preliminary rectangular extent image of eyes.
In Study in Driver Fatigue State Surveillance System 1 of the present invention based on human eye state, face tracking module 12：
Face tracking is carried out for follow-up driver's image using KCF algorithms, to each frame driver image face tracking
Evaluation index data are produced afterwards to assess tracking effect, when evaluation index data are less than given threshold, adjustment tracking plan
Slightly.
In Study in Driver Fatigue State Surveillance System 1 of the present invention based on human eye state, pretreatment module 14：
Judge that whether driver wears glasses in driver's image, and the size of clipping region is selected to eye according to judged result
The preliminary rectangular extent image of eyeball is cut, further according to the average ash of different choice difference of intensity of illumination when obtaining driver's image
Spend threshold value and binaryzation is carried out to the preliminary rectangular extent image of eyes after cutting.
Compared with prior art, beneficial effects of the present invention include：Whether the present invention has worn glasses progress for driver
Detection, and different cuttings are carried out to the preliminary rectangular extent image of eyes according to whether wearing glasses, selected according to the difference of intensity of illumination
Select different average gray threshold values and binaryzation is carried out to the preliminary rectangular extent image of eyes so that fatigue detection method of the invention is more
Plus it is accurate, robustness is higher；For being all suitable under different illumination conditions, real-time is higher, and detection speed is faster；The present invention is compared
Judge in the number of black picture element in other use eye areas or using the ratio of width to height of eye fitted ellipse the side of blink
Method has higher accuracy rate and robustness.
The embodiment of present invention described above, is not intended to limit the scope of the present invention..Any basis
Various other corresponding changes and deformation that the technical concept of the present invention is made, should be included in the guarantor of the claims in the present invention
In the range of shield.
Claims (10)
1. a kind of Driver Fatigue Detection based on human eye state, it is characterised in that including：
S1, in real time acquisition driver's image, and Face datection is carried out to driver's image, obtain the face rectangle for including face
Frame, obtains the preliminary rectangular extent image of eyes and the preliminary rectangular extent image of the eyes in face rectangle frame in face square
Coordinate in shape frame；
After S2, the face rectangle frame of one frame driver's image of acquisition, face tracking is carried out for follow-up driver's image, after acquisition
The face rectangle frame of continuous driver's image；
S3, the face rectangle frame for follow-up driver's image, according to the preliminary rectangular extent image of the eyes in face rectangle
Coordinate in frame, obtains the preliminary rectangular extent image of the eyes of follow-up driver's image；
S4, successively cut for the preliminary rectangular extent image of eyes that S3 is obtained and binaryzation pretreatment；
S5, profile lookup is carried out to the preliminary rectangular extent image of eyes after binaryzation in S4, and utilize rectangle fitting essence
It is determined that position eyes exact rectangular range image, obtains coordinate of the eyes exact rectangular range image in face rectangle frame；
S6, according to coordinate of the eyes exact rectangular range image in face rectangle frame non-two are extracted from face rectangle frame
The eyes exact rectangular range image of value, and closed being formed to open after the eyes exact rectangular range image progress binaryzation of extraction
Eye judges image, will open and close eyes and judges that image carries out upright projection to X-axis, passes through black in upright projection and the ratio of white pixel
Example judges that driver is eye opening or eye closing in frame driver's image；
Driver is in the continuous frame number of closed-eye state in S7, the continuous driver's image of statistics, according to continuous in eye closing
Scope residing for the frame number of state judges that driver is blink or continued eye closure, and calculates frequency of wink；
If S8, frequency of wink are in outside the normal range (NR) of setting, judge that driver is in fatigue state, if driver
Continued eye is closed, then judges that driver is in fatigue state.
2. the Driver Fatigue Detection as claimed in claim 1 based on human eye state, it is characterised in that in step S1：
Driver's image is obtained using the infrared camera with infrared light filling function, indoor illumination intensity is driven higher than setting light
During strong threshold value, the coloured image of driver is obtained, when driving indoor illumination intensity less than setting Intensity threshold, infrared benefit is opened
Light, and obtain the infrared black white image of driver.
3. the Driver Fatigue Detection as claimed in claim 1 based on human eye state, it is characterised in that in step S1：
Train grader to carry out Face datection to driver's image using Adaboost algorithm, get the face square comprising face
Shape frame, stasm positioning feature points are carried out to the face rectangle frame internal image got, so as to recognize eye feature point, are obtained
Image more than the default size rectangular extent of eye feature point range is the preliminary rectangular extent image of the eyes.
4. the Driver Fatigue Detection as claimed in claim 1 based on human eye state, it is characterised in that in step S2：
Face tracking is carried out for follow-up driver's image using KCF algorithms, to equal after each frame driver image face tracking
Evaluation index data are produced to assess tracking effect, when evaluation index data are less than given threshold, tracking strategy are adjusted.
5. the Driver Fatigue Detection as claimed in claim 1 based on human eye state, it is characterised in that in step S4：
Judge that whether driver wears glasses in driver's image, and the size of clipping region is selected to the eye according to judged result
The preliminary rectangular extent image of eyeball is cut, further according to the average ash of different choice difference of intensity of illumination when obtaining driver's image
Spend threshold value and binaryzation is carried out to the preliminary rectangular extent image of the eyes after cutting.
6. a kind of Study in Driver Fatigue State Surveillance System based on human eye state, it is characterised in that including：
Face detection module：Driver's image is obtained in real time, and Face datection is carried out to driver's image, is obtained comprising face
Face rectangle frame, obtains the preliminary rectangular extent image of the eyes and the preliminary rectangular extent of the eyes in face rectangle frame
Coordinate of the image in face rectangle frame；
Face tracking module：After the face rectangle frame for obtaining frame driver's image, face is carried out for follow-up driver's image
Tracking, obtains the face rectangle frame of follow-up driver's image；
Eyes initial ranges image collection module：It is preliminary according to the eyes for the face rectangle frame of follow-up driver's image
Coordinate of the rectangular extent image in face rectangle frame, obtains the preliminary rectangular extent image of eyes of follow-up driver's image；
Pretreatment module：It is first laggard for the preliminary rectangular extent image of eyes that is obtained in eyes initial ranges image collection module
Row cuts the pretreatment with binaryzation；
Eyes exact extension image collection module：To the preliminary rectangular extent image of the eyes after binaryzation in pretreatment module
Profile lookup is carried out, and eyes exact rectangular range image is accurately positioned using rectangle fitting, the eyes exact rectangular is obtained
Coordinate of the range image in face rectangle frame；
Eye opening eye closing judge module：According to coordinate of the eyes exact rectangular range image in face rectangle frame from face square
The eyes exact rectangular range image of non-binaryzation is extracted in shape frame, and the eyes exact rectangular range image of extraction is carried out two
The judgement image that opens and closes eyes is formed after value, will open and close eyes and judge that image carries out upright projection to X-axis, pass through black in upright projection
Judge that driver is eye opening or eye closing in frame driver's image with the ratio of white pixel；
Blink is persistently closed one's eyes judge module：Count the successive frame that driver in continuous driver's image is in closed-eye state
Number, the scope according to residing for the continuous frame number in closed-eye state judges that driver is blink or continued eye closure, and calculates
Frequency of wink；
Tired judge module：If frequency of wink is in outside the normal range (NR) of setting, judge that driver is in fatigue state,
If driver's eyes are persistently closed, judge that driver is in fatigue state.
7. the Study in Driver Fatigue State Surveillance System as claimed in claim 6 based on human eye state, it is characterised in that Face datection mould
In block：
Driver's image is obtained using the infrared camera with infrared light filling function, indoor illumination intensity is driven higher than setting light
During strong threshold value, the coloured image of driver is obtained, when driving indoor illumination intensity less than setting Intensity threshold, infrared benefit is opened
Light, and obtain the infrared black white image of driver.
8. the Study in Driver Fatigue State Surveillance System as claimed in claim 6 based on human eye state, it is characterised in that Face datection mould
In block：
Train grader to carry out Face datection to driver's image using Adaboost algorithm, get the face square comprising face
Shape frame, stasm positioning feature points are carried out to the face rectangle frame internal image got, so as to recognize eye feature point, are obtained
Image more than the default size rectangular extent of eye feature point range is the preliminary rectangular extent image of eyes.
9. the Study in Driver Fatigue State Surveillance System as claimed in claim 6 based on human eye state, it is characterised in that face tracking mould
In block：
Face tracking is carried out for follow-up driver's image using KCF algorithms, to equal after each frame driver image face tracking
Evaluation index data are produced to assess tracking effect, when evaluation index data are less than given threshold, tracking strategy are adjusted.
10. the Study in Driver Fatigue State Surveillance System as claimed in claim 6 based on human eye state, it is characterised in that pretreatment mould
In block：
Judge whether driver wears glasses in driver's image, the size of clipping region is selected according to judged result at the beginning of eyes
Step rectangular extent image is cut, further according to the different average gray thresholds of different choice of intensity of illumination when obtaining driver's image
It is worth and binaryzation is carried out to the preliminary rectangular extent image of the eyes after cutting.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710430629.3A CN107292251B (en) | 2017-06-09 | 2017-06-09 | Driver fatigue detection method and system based on human eye state |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710430629.3A CN107292251B (en) | 2017-06-09 | 2017-06-09 | Driver fatigue detection method and system based on human eye state |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107292251A true CN107292251A (en) | 2017-10-24 |
| CN107292251B CN107292251B (en) | 2020-08-28 |
Family
ID=60096130
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710430629.3A Active CN107292251B (en) | 2017-06-09 | 2017-06-09 | Driver fatigue detection method and system based on human eye state |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107292251B (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108382396A (en) * | 2018-02-02 | 2018-08-10 | 辽宁友邦网络科技有限公司 | Driver's driving condition identifying system and its application process |
| CN108419012A (en) * | 2018-03-18 | 2018-08-17 | 广东欧珀移动通信有限公司 | Photographic method, device, storage medium and electronic equipment |
| CN108614999A (en) * | 2018-04-16 | 2018-10-02 | 贵州大学 | Eyes based on deep learning open closed state detection method |
| CN108919517A (en) * | 2018-09-25 | 2018-11-30 | 吉林大学 | A kind of car steering anti-fatigue eyeglasses and fatigue state recognition method |
| CN109145852A (en) * | 2018-08-31 | 2019-01-04 | 辽宁工业大学 | A kind of driver fatigue state recognition method for opening closed state based on eyes |
| CN109271875A (en) * | 2018-08-24 | 2019-01-25 | 中国人民解放军火箭军工程大学 | A kind of fatigue detection method based on supercilium and eye key point information |
| CN109646024A (en) * | 2019-01-09 | 2019-04-19 | 浙江强脑科技有限公司 | Method for detecting fatigue driving, device and computer readable storage medium |
| CN111784973A (en) * | 2020-07-30 | 2020-10-16 | 广州敏视数码科技有限公司 | MDVR equipment integration fatigue detection method of fleet management platform |
| CN113076801A (en) * | 2021-03-04 | 2021-07-06 | 广州铁路职业技术学院（广州铁路机械学校） | Train on-road state intelligent linkage detection system and method |
| CN113780150A (en) * | 2021-09-07 | 2021-12-10 | 广西电网有限责任公司贺州供电局 | Fatigue detection method and system based on multi-dimensional body state perception |
| CN115243000A (en) * | 2021-08-30 | 2022-10-25 | 广州汽车集团股份有限公司 | Automobile cabin monitoring system and automobile |
| CN116168374A (en) * | 2023-04-21 | 2023-05-26 | 南京淼瀛科技有限公司 | Active safety auxiliary driving method and system |
| CN116469085A (en) * | 2023-03-30 | 2023-07-21 | 万联易达物流科技有限公司 | Monitoring method and system for risk driving behavior |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101059836A (en) * | 2007-06-01 | 2007-10-24 | 华南理工大学 | Human eye positioning and human eye state recognition method |
| CN102054163A (en) * | 2009-10-27 | 2011-05-11 | 南京理工大学 | Method for testing driver fatigue based on monocular vision |
| WO2012000408A1 (en) * | 2010-06-29 | 2012-01-05 | Byd Company Limited | Eye detecting method and eye detecting device |
| CN104809445A (en) * | 2015-05-07 | 2015-07-29 | 吉林大学 | Fatigue driving detection method based on eye and mouth states |
| CN104881955A (en) * | 2015-06-16 | 2015-09-02 | 华中科技大学 | Method and system for detecting fatigue driving of driver |
| CN105224285A (en) * | 2014-05-27 | 2016-01-06 | 北京三星通信技术研究有限公司 | Eyes open and-shut mode pick-up unit and method |
| CN105335695A (en) * | 2015-08-19 | 2016-02-17 | 华南理工大学 | Glasses detection based eye positioning method |
-
2017
- 2017-06-09 CN CN201710430629.3A patent/CN107292251B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101059836A (en) * | 2007-06-01 | 2007-10-24 | 华南理工大学 | Human eye positioning and human eye state recognition method |
| CN102054163A (en) * | 2009-10-27 | 2011-05-11 | 南京理工大学 | Method for testing driver fatigue based on monocular vision |
| WO2012000408A1 (en) * | 2010-06-29 | 2012-01-05 | Byd Company Limited | Eye detecting method and eye detecting device |
| CN105224285A (en) * | 2014-05-27 | 2016-01-06 | 北京三星通信技术研究有限公司 | Eyes open and-shut mode pick-up unit and method |
| CN104809445A (en) * | 2015-05-07 | 2015-07-29 | 吉林大学 | Fatigue driving detection method based on eye and mouth states |
| CN104881955A (en) * | 2015-06-16 | 2015-09-02 | 华中科技大学 | Method and system for detecting fatigue driving of driver |
| CN105335695A (en) * | 2015-08-19 | 2016-02-17 | 华南理工大学 | Glasses detection based eye positioning method |
Non-Patent Citations (4)
| Title |
|---|
| 梁涛等: ""基于OpenCV的视频图像人眼跟踪方法的研究"", 《科学技术与工程》 * |
| 王威等: ""一种用于驾驶员疲劳检测的人眼状态判别方法"", 《计算机光盘软件与应用》 * |
| 田亮等: ""军用车辆驾驶中的眼睛状态检测法研究"", 《火力与指挥控制》 * |
| 陈岳林等: ""基于人脸图片边缘信息的眼镜检测"", 《软件导刊》 * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108382396B (en) * | 2018-02-02 | 2020-08-07 | 上海灏领科技有限公司 | Driver driving state recognition system and application method thereof |
| CN108382396A (en) * | 2018-02-02 | 2018-08-10 | 辽宁友邦网络科技有限公司 | Driver's driving condition identifying system and its application process |
| CN108419012A (en) * | 2018-03-18 | 2018-08-17 | 广东欧珀移动通信有限公司 | Photographic method, device, storage medium and electronic equipment |
| CN108614999A (en) * | 2018-04-16 | 2018-10-02 | 贵州大学 | Eyes based on deep learning open closed state detection method |
| CN108614999B (en) * | 2018-04-16 | 2022-09-16 | 贵州大学 | Eye opening and closing state detection method based on deep learning |
| CN109271875A (en) * | 2018-08-24 | 2019-01-25 | 中国人民解放军火箭军工程大学 | A kind of fatigue detection method based on supercilium and eye key point information |
| CN109145852A (en) * | 2018-08-31 | 2019-01-04 | 辽宁工业大学 | A kind of driver fatigue state recognition method for opening closed state based on eyes |
| CN108919517A (en) * | 2018-09-25 | 2018-11-30 | 吉林大学 | A kind of car steering anti-fatigue eyeglasses and fatigue state recognition method |
| CN109646024A (en) * | 2019-01-09 | 2019-04-19 | 浙江强脑科技有限公司 | Method for detecting fatigue driving, device and computer readable storage medium |
| CN111784973A (en) * | 2020-07-30 | 2020-10-16 | 广州敏视数码科技有限公司 | MDVR equipment integration fatigue detection method of fleet management platform |
| CN111784973B (en) * | 2020-07-30 | 2021-12-14 | 广州敏视数码科技有限公司 | MDVR equipment integration fatigue detection method of fleet management platform |
| CN113076801A (en) * | 2021-03-04 | 2021-07-06 | 广州铁路职业技术学院（广州铁路机械学校） | Train on-road state intelligent linkage detection system and method |
| CN115243000A (en) * | 2021-08-30 | 2022-10-25 | 广州汽车集团股份有限公司 | Automobile cabin monitoring system and automobile |
| CN115243000B (en) * | 2021-08-30 | 2024-02-02 | 广州汽车集团股份有限公司 | Automobile cabin monitoring system and automobile |
| CN113780150A (en) * | 2021-09-07 | 2021-12-10 | 广西电网有限责任公司贺州供电局 | Fatigue detection method and system based on multi-dimensional body state perception |
| CN116469085A (en) * | 2023-03-30 | 2023-07-21 | 万联易达物流科技有限公司 | Monitoring method and system for risk driving behavior |
| CN116469085B (en) * | 2023-03-30 | 2024-04-02 | 万联易达物流科技有限公司 | Monitoring method and system for risk driving behavior |
| CN116168374A (en) * | 2023-04-21 | 2023-05-26 | 南京淼瀛科技有限公司 | Active safety auxiliary driving method and system |
| CN116168374B (en) * | 2023-04-21 | 2023-12-12 | 南京淼瀛科技有限公司 | Active safety auxiliary driving method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107292251B (en) | 2020-08-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107292251A (en) | A kind of Driver Fatigue Detection and system based on human eye state | |
| CN104809445B (en) | method for detecting fatigue driving based on eye and mouth state | |
| CN112241658B (en) | Fatigue driving early warning method based on depth camera | |
| CN101593425B (en) | Machine vision based fatigue driving monitoring method and system | |
| CN102054163B (en) | Method for testing driver fatigue based on monocular vision | |
| CN106846734A (en) | A kind of fatigue driving detection device and method | |
| CN1225375C (en) | Method for detecting fatigue driving based on multiple characteristic fusion | |
| CN106250801A (en) | Based on Face datection and the fatigue detection method of human eye state identification | |
| CN108446600A (en) | A kind of vehicle driver's fatigue monitoring early warning system and method | |
| CN101593352A (en) | Driving safety monitoring system based on face orientation and visual focus | |
| CN112434611B (en) | Early fatigue detection method and system based on eye movement subtle features | |
| CN104318237A (en) | Fatigue driving warning method based on face identification | |
| CN106080194A (en) | The method for early warning of anti-fatigue-driving and system | |
| CN104224204A (en) | Driver fatigue detection system on basis of infrared detection technology | |
| CN106682603A (en) | Real time driver fatigue warning system based on multi-source information fusion | |
| JP2000235648A (en) | Extracting device for eye and blink detecting device | |
| CN106214166A (en) | One is worn glasses Driver Fatigue Detection | |
| CN106909879A (en) | A kind of method for detecting fatigue driving and system | |
| Luo et al. | The driver fatigue monitoring system based on face recognition technology | |
| CN103729646B (en) | Eye image validity detection method | |
| CN108256378A (en) | Driver Fatigue Detection based on eyeball action recognition | |
| JP4895874B2 (en) | Eye state determination device, eye state determination method, and eye state determination program | |
| CN203885510U (en) | Driver fatigue detection system based on infrared detection technology | |
| Boverie et al. | Driver vigilance diagnostic based on eyelid movement observation | |
| CN114998874A (en) | Driver abnormal behavior detection method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210611 Address after: 435100 Jinhu Ma Hai, Daye City, Huangshi City, Hubei Province Patentee after: Daye Xinye Special Steel Co.,Ltd. Address before: No.139 kuangye Avenue, Jinhu Street office, Daye City, Huangshi City, Hubei Province Patentee before: HUBEI TIANYE YUNSHANG NETWORK TECHNOLOGY Co.,Ltd. |
|
| TR01 | Transfer of patent right |