CN109543577A - A kind of fatigue driving detection method for early warning based on facial expression feature - Google Patents
A kind of fatigue driving detection method for early warning based on facial expression feature Download PDFInfo
- Publication number
- CN109543577A CN109543577A CN201811332007.8A CN201811332007A CN109543577A CN 109543577 A CN109543577 A CN 109543577A CN 201811332007 A CN201811332007 A CN 201811332007A CN 109543577 A CN109543577 A CN 109543577A
- Authority
- CN
- China
- Prior art keywords
- face
- facial expression
- fatigue driving
- early warning
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 33
- 230000008921 facial expression Effects 0.000 title claims abstract description 24
- 238000000034 method Methods 0.000 claims abstract description 33
- 239000000284 extract Substances 0.000 claims abstract description 5
- 230000001815 facial effect Effects 0.000 claims description 10
- 230000004927 fusion Effects 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 abstract description 5
- 206010039203 Road traffic accident Diseases 0.000 abstract description 4
- 206010048232 Yawning Diseases 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 206010041349 Somnolence Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000036632 reaction speed Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/174—Facial expression recognition
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/02—Alarms for ensuring the safety of persons
- G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
Abstract
The present invention relates to a kind of, and the fatigue driving based on facial expression feature detects method for early warning, comprising the following steps: acquires video information using infrared equipment；Utilize the face of Face datection algorithm detection video frame；Positioning feature point is carried out to face, extracts 68 people's face facial expression feature points；The characteristic point of eyes and mouth partial region is chosen in 68 people's face facial expression features as input feature vector；Using the input feature vector of extraction, by classifier completion judge single-frame images whether Pi Lao result.The present invention can remind driver to guard against fatigue driving, play forewarning function, reduce traffic accident.
Description
Technical field
The present invention relates to machine vision application technology fields, drive more particularly to a kind of fatigue based on facial expression feature
Sail detection method for early warning.
Background technique
Multiple studies have shown that, follow the bus caused by fatigue driving is excessively close, deviation, reaction speed is slack-off both at home and abroad, grasps
Make the major reason for leading to traffic accident when ability decline, the extremely dangerous driving behaviors such as out of control.Pacified according to United States highways
Full statistical data of the management board between 2011 to 2015 years, the traffic accident caused by fatigue driving has 4121, only 2015
The directly death toll as caused by fatigue driving is 824 people within 1 year.Therefore, fatigue-driving detection technology research has important society
It can be worth and realistic meaning.
Current method for detecting fatigue driving can be divided into three classes: in early days from medical angle, by medical electroencephalogram
The fatigue detecting based on physiological driver's signal such as instrument, electrocardiograph, this method can accurately detect the life of driver
Manage information, then accurately judge the fatigue state of driver, but such method detection device need and human body into
Row directly contacts, and test condition is harsh, and process is complicated, will affect driving-activity, is not easy to promote the use of；It is driven and is gone based on driver
For fatigue detecting, pass through installation sensor perceive such as steering wheel frequency of amendment, gas pedal and clutch pedal corner
Fatigue driving detection is carried out, such method is simple and exploitativeness is high, but cannot intuitively react the tired shape of driver
State is easy to be influenced court verdict by the driving habit of road conditions and driver；The fatigue detecting of view-based access control model feature according to
Whether the countenance variation detection driver of driver is tired, using machine vision technique to driver's facial expression packet
It includes blink and the identification of characteristic behaviors such as acts, yawns and being effectively estimated and detecting for fatigue state can be achieved, since the technology has
There is non-intruding, accurate, real-time, does not influence the normal driving of driver, it will not be by the shadow of road conditions and driving habit
The shortcomings that ringing, can overcoming first two method becomes the emphasis of research and development then.
Face contains as the very important a part of human body enriches useful information, such as identity, gender, expression
Deng.And a part that eyes are important as face, it can intuitively reflect very much the expression information of driver, Dinges etc.
People most already proposes the fatigue based on machine vision according to the percentage (PERCLOS) that the eyes closed time accounts for a period of time and examines
Survey method；Mouth a part the most active as face also can intuitively reflect tired information very much, it is many in the recent period its
Whether the work of its fatigue detecting is exactly to yawn progresss around driver, Mandalapu and Bajaj use SVM (support to
Amount machine) nozzle type is classified as the type normally and yawned, but only test 20 pictures and obtained 81% correct classification
Rate；Whether Zhang is yawned using CNN (convolutional neural networks) detection driver, obtains up to 92% accuracy rate, but accidentally
Report rate has but reached 13%, and it is all incomplete for only using any one of eyes or mouth and carrying out fatigue detecting, because
This inherently loses many useful facial informations.In addition, though the work based on deep learning may obtain it is higher
Detection accuracy, but for vehicle-mounted embedded type calculates equipment, to accomplish to handle and be able in real time larger using difficulty.
Summary of the invention
Technical problem to be solved by the invention is to provide a kind of, and the fatigue driving based on facial expression feature detects early warning
Method can remind driver to guard against fatigue driving, play forewarning function, reduce traffic accident.
The technical solution adopted by the present invention to solve the technical problems is: providing a kind of fatigue based on facial expression feature
Drive detection method for early warning, comprising the following steps:
(1) video information is acquired using infrared equipment；
(2) face of Face datection algorithm detection video frame is utilized；
(3) positioning feature point is carried out to face, extracts 68 people's face facial expression feature points；
(4) characteristic point of eyes and mouth partial region is chosen in 68 people's face facial expression features as separator
Input feature vector；
(5) using extract input feature vector, by classifier completion judge single-frame images whether Pi Lao result.
Referred in the step (1) using infrared equipment acquisition video information and carries out light filling using infrared light supply or utilize red
Outer camera acquires video.
When in the step (2) using the face of Face datection algorithm detection video frame, first frame processing is small using Haar
The method for detecting human face of wave characteristic and integrogram, detect after face using TLD method for tracking target to the face of subsequent frame into
Line trace detection, if the discovery tracking failure during face tracking of subsequent frame, then restarting Face datection algorithm, such as
This circulation.
When carrying out positioning feature point to face in the step (3), it is fixed that human face characteristic point is carried out using SDM supervision descent method
Position detection, according to the local feature of current signature point, continuous iteration obtains the position of final characteristic point.
The characteristic point of eyes described in the step (4) and mouth partial region is specially the depth-width ratio and mouth that eyes are opened
The depth-width ratio of Ba Zhang great.
State classification is carried out using Fisher linear discriminant algorithm in the step (5).
Further include carrying out data fusion using the result of video multiframe classification after the step (5), finally makes the stage
Whether Pi Lao judgement, to tired situation output signal carry out system sounds remind feedback.
Beneficial effect
Due to the adoption of the above technical solution, compared with prior art, the present invention having the following advantages that and actively imitating
Fruit: the present invention includes video acquisition, Face datection, facial modeling, feature extraction, fatigue state classification and most terminates
The judgement for closing multiframe data fusion, completely constitutes a detection early warning system, can play forewarning function to fatigue driving；This
Invention face detection part uses the method that detection is combined with tracking, meets while guaranteeing high-precision Face datection real
When the requirement that handles.
Detailed description of the invention
Fig. 1 is flow chart of the invention.
Specific embodiment
Present invention will be further explained below with reference to specific examples.It should be understood that these embodiments are merely to illustrate the present invention
Rather than it limits the scope of the invention.In addition, it should also be understood that, after reading the content taught by the present invention, those skilled in the art
Member can make various changes or modifications the present invention, and such equivalent forms equally fall within the application the appended claims and limited
Range.
Embodiments of the present invention be related to it is a kind of based on facial expression feature fatigue driving detection method for early warning, including with
Lower step: video information is acquired using infrared equipment；Utilize the face of Face datection algorithm detection video frame；Face is carried out special
Point location is levied, 68 people's face facial expression feature points are extracted；Eyes and mouth are chosen in 68 people's face facial expression features
The characteristic point of partial region is as input feature vector；Using the input feature vector of extraction, judge that single-frame images is by classifier completion
The result of no fatigue.It is specific as follows:
Acquisition has the positive face of driver to thermal camera (or infrared light filling video camera) before being mounted on bridge in real time
Video information.
Face datection algorithm is carried out first, during the face using Face datection algorithm detection video frame, in order to guarantee
The real-time of algorithm, using the method for detection plus tracking.First frame processing uses the face of Haar wavelet character and integrogram
Detection method detects to carry out tracing detection using face of the TLD method for tracking target to subsequent frame after face, if in subsequent frame
Face tracking during find tracking failure, then restarting Face datection algorithm, so recycle, guarantee high-precision people
Meet the requirement handled in real time while face detects.
Start facial modeling algorithm for the video frame for being tested with face.Positioning feature point mistake is carried out to face
Cheng Zhong carries out facial modeling detection using SDM (Supervised Descent Method) supervision descent method, and SDM will
Facial modeling problem is configured to the model of a regression problem, according to the local feature of current signature point, continuous iteration
It can be obtained by the position of final characteristic point.SDM belongs to a kind of method for solving the problems, such as non-linear minimisation, has speed smart fastly
Spend high advantage.
After orienting 68 human face characteristic points, eye is chosen, the feature input separator of mouth makes decisions.Pass through eyes
The input feature vector that the depth-width ratio that the depth-width ratio and mouth opened are magnified is classified as fatigue state.The combination of the two features will
Can improve yawn, the verification and measurement ratio of the fatigue states such as drowsiness, and the false detection rate of the normal conditions such as speech, laugh can be reduced simultaneously.
The eye of extraction and the characteristic quantity of mouth opening and closing input Fisher linear classifier are subjected to state classification, judged tired
Whether labor.Classifier needs to obtain preferable disaggregated model by data training early period.
The most of fatigue form of expression is all a lasting process, the facial image of single frames do not have show it is this
The ability of process carries out fatigue judgement using the facial image of single frames and inherently loses this useful temporal information, therefore logical
It crosses data fusion and carries out fatigue driving detection judgement using video clip.Consider the driver pole in actual driving procedure
It is possible that talk chatting and laughing are carried out with passenger, therefore its facial expression is also extremely complex, and the detectability of single frames facial image is non-
It is often limited, it is most likely that the case where erroneous detection occur, the fatigue driving detection based on video clip can integrate multiframe facial image
Testing result make final judgement, finally greatly reduce this erroneous judgement situation and occur.
It is not difficult to find that present invention comprises video acquisition, Face datection, positioning feature point, feature extraction and tired shapes
State adjudicates whole system flow, finally can fatigue driving situation be taken a good rest by sound, light alarm and reminding driver, be played pre-
Alert effect.
Claims (7)
1. a kind of fatigue driving based on facial expression feature detects method for early warning, which comprises the following steps:
(1) video information is acquired using infrared equipment；
(2) face of Face datection algorithm detection video frame is utilized；
(3) positioning feature point is carried out to face, extracts 68 people's face facial expression feature points；
(4) input of the characteristic point of eyes and mouth partial region as separator is chosen in 68 people's face facial expression features
Feature；
(5) using extract input feature vector, by classifier completion judge single-frame images whether Pi Lao result.
2. the fatigue driving according to claim 1 based on facial expression feature detects method for early warning, which is characterized in that institute
It states and is referred to using infrared equipment acquisition video information using infrared light supply progress light filling in step (1) or acquired using infrared camera
Video.
3. the fatigue driving according to claim 1 based on facial expression feature detects method for early warning, which is characterized in that institute
When stating the face in step (2) using Face datection algorithm detection video frame, first frame processing uses Haar wavelet character and product
The method for detecting human face of component detects to carry out tracing detection using face of the TLD method for tracking target to subsequent frame after face,
If the discovery tracking failure during face tracking of subsequent frame, then restarting Face datection algorithm, so recycles.
4. the fatigue driving according to claim 1 based on facial expression feature detects method for early warning, which is characterized in that institute
When stating in step (3) to face progress positioning feature point, facial modeling detection, root are carried out using SDM supervision descent method
According to the local feature of current signature point, continuous iteration obtains the position of final characteristic point.
5. the fatigue driving according to claim 1 based on facial expression feature detects method for early warning, which is characterized in that institute
The characteristic point for stating eyes described in step (4) and mouth partial region is specially the depth-width ratio that eyes are opened and the height that mouth magnifies
Wide ratio.
6. the fatigue driving according to claim 1 based on facial expression feature detects method for early warning, which is characterized in that institute
It states in step (5) and state classification is carried out using Fisher linear discriminant algorithm.
7. the fatigue driving according to claim 1 based on facial expression feature detects method for early warning, which is characterized in that institute
To state further include carrying out data fusion using the result of video multiframe classification after step (5), finally make the stage whether fatigue
Judgement carries out system sounds to tired situation output signal and reminds feedback.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811332007.8A CN109543577A (en) | 2018-11-09 | 2018-11-09 | A kind of fatigue driving detection method for early warning based on facial expression feature |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811332007.8A CN109543577A (en) | 2018-11-09 | 2018-11-09 | A kind of fatigue driving detection method for early warning based on facial expression feature |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109543577A true CN109543577A (en) | 2019-03-29 |
Family
ID=65846494
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811332007.8A Pending CN109543577A (en) | 2018-11-09 | 2018-11-09 | A kind of fatigue driving detection method for early warning based on facial expression feature |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109543577A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110021147A (en) * | 2019-05-07 | 2019-07-16 | 四川九洲视讯科技有限责任公司 | A kind of method for detecting fatigue driving demarcated based on machine learning and numerical value |
| CN110143202A (en) * | 2019-04-09 | 2019-08-20 | 南京交通职业技术学院 | A kind of dangerous driving identification and method for early warning and system |
| CN110263663A (en) * | 2019-05-29 | 2019-09-20 | 南京师范大学 | A kind of driver's multistage drowsiness monitor method based on multidimensional facial characteristics |
| CN110532976A (en) * | 2019-09-03 | 2019-12-03 | 湘潭大学 | Method for detecting fatigue driving and system based on machine learning and multiple features fusion |
| CN110796838A (en) * | 2019-12-03 | 2020-02-14 | 吉林大学 | Automatic positioning and recognition system for facial expressions of driver |
| CN111582129A (en) * | 2020-04-30 | 2020-08-25 | 中铁工程装备集团有限公司 | Real-time monitoring and alarming method and device for working state of shield machine driver |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102881024A (en) * | 2012-08-24 | 2013-01-16 | 南京航空航天大学 | Tracking-learning-detection (TLD)-based video object tracking method |
| CN103020594A (en) * | 2012-12-03 | 2013-04-03 | 清华大学苏州汽车研究院（吴江） | Fatigue state detecting method for eliminating driver individual difference by utilizing online learning |
| CN104240446A (en) * | 2014-09-26 | 2014-12-24 | 长春工业大学 | Fatigue driving warning system on basis of human face recognition |
| CN105893920A (en) * | 2015-01-26 | 2016-08-24 | 阿里巴巴集团控股有限公司 | Human face vivo detection method and device |
| CN107679468A (en) * | 2017-09-19 | 2018-02-09 | 浙江师范大学 | A kind of embedded computer vision detects fatigue driving method and device |
-
2018
- 2018-11-09 CN CN201811332007.8A patent/CN109543577A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102881024A (en) * | 2012-08-24 | 2013-01-16 | 南京航空航天大学 | Tracking-learning-detection (TLD)-based video object tracking method |
| CN103020594A (en) * | 2012-12-03 | 2013-04-03 | 清华大学苏州汽车研究院（吴江） | Fatigue state detecting method for eliminating driver individual difference by utilizing online learning |
| CN104240446A (en) * | 2014-09-26 | 2014-12-24 | 长春工业大学 | Fatigue driving warning system on basis of human face recognition |
| CN105893920A (en) * | 2015-01-26 | 2016-08-24 | 阿里巴巴集团控股有限公司 | Human face vivo detection method and device |
| CN107679468A (en) * | 2017-09-19 | 2018-02-09 | 浙江师范大学 | A kind of embedded computer vision detects fatigue driving method and device |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110143202A (en) * | 2019-04-09 | 2019-08-20 | 南京交通职业技术学院 | A kind of dangerous driving identification and method for early warning and system |
| CN110021147A (en) * | 2019-05-07 | 2019-07-16 | 四川九洲视讯科技有限责任公司 | A kind of method for detecting fatigue driving demarcated based on machine learning and numerical value |
| CN110263663A (en) * | 2019-05-29 | 2019-09-20 | 南京师范大学 | A kind of driver's multistage drowsiness monitor method based on multidimensional facial characteristics |
| CN110532976A (en) * | 2019-09-03 | 2019-12-03 | 湘潭大学 | Method for detecting fatigue driving and system based on machine learning and multiple features fusion |
| CN110532976B (en) * | 2019-09-03 | 2021-12-31 | 湘潭大学 | Fatigue driving detection method and system based on machine learning and multi-feature fusion |
| CN110796838A (en) * | 2019-12-03 | 2020-02-14 | 吉林大学 | Automatic positioning and recognition system for facial expressions of driver |
| CN111582129A (en) * | 2020-04-30 | 2020-08-25 | 中铁工程装备集团有限公司 | Real-time monitoring and alarming method and device for working state of shield machine driver |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109543577A (en) | A kind of fatigue driving detection method for early warning based on facial expression feature | |
| CN104637246B (en) | Driver multi-behavior early warning system and danger evaluation method | |
| CN108446600A (en) | A kind of vehicle driver's fatigue monitoring early warning system and method | |
| Ji et al. | Fatigue state detection based on multi-index fusion and state recognition network | |
| CN107704805B (en) | Method for detecting fatigue driving, automobile data recorder and storage device | |
| CN101593425B (en) | Machine vision based fatigue driving monitoring method and system | |
| Hossain et al. | IOT based real-time drowsy driving detection system for the prevention of road accidents | |
| CN105719431A (en) | Fatigue driving detection system | |
| Cheng et al. | Driver drowsiness detection based on multisource information | |
| CN104408878A (en) | Vehicle fleet fatigue driving early warning monitoring system and method | |
| CN104183091A (en) | System for adjusting sensitivity of fatigue driving early warning system in self-adaptive mode | |
| CN104361716A (en) | Method for detecting and reminding fatigue in real time | |
| CN109740477A (en) | Study in Driver Fatigue State Surveillance System and its fatigue detection method | |
| CN111753674A (en) | Fatigue driving detection and identification method based on deep learning | |
| CN108021875A (en) | A kind of vehicle driver's personalization fatigue monitoring and method for early warning | |
| Charniya et al. | Drunk driving and drowsiness detection | |
| CN106295474A (en) | The fatigue detection method of deck officer, system and server | |
| CN110264670A (en) | Based on passenger stock tired driver driving condition analytical equipment | |
| Jia et al. | Fatigue driving detection based on deep learning and multi-index fusion | |
| Ying et al. | The monitoring method of driver's fatigue based on neural network | |
| CN108108651B (en) | Method and system for detecting driver non-attentive driving based on video face analysis | |
| Alam et al. | Active vision-based attention monitoring system for non-distracted driving | |
| CN109308467A (en) | Traffic accident prior-warning device and method for early warning based on machine learning | |
| Boverie et al. | Driver vigilance diagnostic based on eyelid movement observation | |
| CN206039557U (en) | Driving situation monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190329 |
|
| RJ01 | Rejection of invention patent application after publication |