CN105096528A - Fatigue driving detection method and system - Google Patents
Fatigue driving detection method and system Download PDFInfo
- Publication number
- CN105096528A CN105096528A CN201510475351.2A CN201510475351A CN105096528A CN 105096528 A CN105096528 A CN 105096528A CN 201510475351 A CN201510475351 A CN 201510475351A CN 105096528 A CN105096528 A CN 105096528A
- Authority
- CN
- China
- Prior art keywords
- driver
- present period
- time
- state
- vehicle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Traffic Control Systems (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a fatigue driving detection method, which comprises the following steps of authenticating the driver identity, accumulating the driving time, extracting the expression state, extracting the body state, collecting the road condition information, extracting the vehicle control state and detecting the fatigue driving. The invention also provides a fatigue driving detection system. According to the invention, the facial expression and the body movement of a driver is tracked through the face recognition, and the driver becomes fatigue from a normal state, for example, the eye-closed time is getting longer, the nod frequency is increased, and the like; the vehicle control state of the driver is tracked and judged whether to become bad along with the time, for example, the steering wheel rotation variance is getting larger, so the driver is detected whether to be in a fatigue driving state. The fatigue driving detection method and system avoid presetting a define threshold, and the accuracy can not be affect by the individual differences of drivers.
Description
Technical field
The invention belongs to car steering field, be specifically related to a kind of method for detecting fatigue driving and system.
Background technology
In recent years, along with the growth of automobile use amount, the incidence of traffic hazard increases year by year, and driving safety becomes the problem be day by day concerned.In the traffic hazard caused due to dangerous driving behavior, fatigue driving occupies quite a few ratio, and according to national departments concerned statistics, the traffic hazard caused because of fatigue driving accounts for about 20%.Therefore for many years, driving fatigue detection is the study hotspot of academia and industry member always.Up to now, existing method for detecting fatigue driving can be summarized as three classes:
The first kind: human body bio signal, such as by wavelength coverage specific in the electroencephalogram/cardiogram of analysis driver, thus whether detection driver is in fatigue driving state;
Equations of The Second Kind: detect facial expression and limb action, such as by detecting the time that driver closes one's eyes, the yawning frequency, the frequency of bowing, whether health leans forward etc., thus judges whether driver is in fatigue driving state;
3rd class: detect vehicle performance state, for example by detecting the change width frequently of bearing circle rotational angle, the change width frequently of acceleration (throttle), lane shift degree etc., whether the driving ability analyzing driver declines, thus judges whether driver is in fatigue driving state.
The present inventor finds through research, in existing three class method for detecting fatigue driving, wherein based on the detection method of human-body biological signal, there is higher accuracy, but need driver to wear the electronic equipment of some contacts, for example detect the headgear assembly of EEG signals, for these devices or equipment, driver normally compares dislike and repels.And based on the detection method of " facial expression and limb action " or " vehicle performance state ", convenient and easy, but for different drivers, when distinguishing tired or not tired, the delimited area (i.e. threshold value) of these signals there are differences, thus these detection methods proposed at present, normally based on a universal model irrelevant with individuality, there is larger difference in its accuracy between different drivers.
Summary of the invention
For in the method for detecting fatigue driving that prior art exists, normally based on a universal model irrelevant with individuality, there is the technical matters of larger difference in its accuracy, the invention provides a kind of method for detecting fatigue driving followed the tracks of based on drivers working states between different drivers.
To achieve these goals, the present invention adopts following technical scheme:
A kind of method for detecting fatigue driving, the method comprises the following steps:
Driver identity certification, according to the video image of driver at present period, by recognition of face, just carries out certification at the driver identity of steering vehicle to current;
Driving time adds up, and according to the identity authentication result of driver, obtains the driving time record of driver;
Emotional state extracts, and according to the video image of driver at present period, extracts the emotional state of driver at present period;
Limbs state is extracted, and according to the video image of driver at present period, extracts the limbs state of driver at present period;
Traffic information is collected, and according to the video image that scene camera is taken at present period, obtains the environmental information of driver vehicle-surroundings;
Vehicle performance state is extracted, the driving condition of combining road condition information and vehicle, extracts driver in the manipulation state of present period to vehicle;
Fatigue driving detects, the timestamp of driving time as driver's emotional state, limbs state and vehicle performance state is added up using present period, upgrade the data of following the tracks of drivers working states change, analyze the state change of driver in more new data, judge whether current time driver is in fatigue driving.
Method for detecting fatigue driving provided by the invention, by recognition of face, tracking driver's facial expression and limb action are passed in time, present and change from normal condition to fatigue state, such as closed-eye time is elongated, the frequency of bowing increase etc.; Follow the tracks of driver vehicle to manipulate state and whether decline in time, such as rotating of steering wheel variance becomes large etc., detects driver whether be in fatigue driving state with this.Method for detecting fatigue driving provided by the invention avoids and presets one and define threshold value, and its accuracy is by the impact of driver's individual difference.
Further, described driver identity certification comprises the following steps:
According to the video image of driver at present period, extract the key point of driver's face area in start-stop frame/intermediate frame video image;
According to the relative position of key point, size normalization is carried out to face area;
By convolutional neural networks, from the face area after size normalization, extract face feature;
Use probability Linear analysis model, calculate the similarity of current extraction face feature and registered driver's face feature, if similarity is more than or equal to 0.9, then the identity of present period driver is exactly this registered driver; If similarity is less than 0.9, then need to register the driver of present period, the face area detected with present period intermediate frame, as the identity marks of this driver.
Further, described driving time is accumulative comprises the following steps:
If the last time detects the time of this driver vehicle, 4 hours are more than or equal to the lead time of current time, then the last time is detected that the time of this driver vehicle replaces with current time, driving time accumulative when the last time is detected replaces with 120 seconds, and face area during driver login is replaced with the face area that present period intermediate frame detects;
If the last time detects the time of this driver vehicle, 4 hours are less than with the lead time of current time, then the last time is detected that the time of this driver vehicle replaces with current time, cumulative 120 seconds of driving time accumulative when the last time is detected.
Further, described emotional state extracts and comprises the following steps:
According to the video image of driver at present period, extract eye key point and the mouth key point of driver in every frame video image;
According to the relative position of eye key point in every frame video image, calculate the form parameter of eyelid profile;
The eyelid contour shape parameter of every frame image in present period, calculates the distribution of shapes of driver at present period palpebra interna profile;
According to the relative position of mouth key point in every frame video image, judge the opening and closing state of mouth;
From the opening and closing status switch of mouth, calculate the number of times of driver's yawn in present period.
Further, described limbs state is extracted and is comprised the following steps:
According to the video image of driver at present period, extract the key point of driver's face area in every frame video image;
By the position of face's key point when face's key point and driver login in more every frame video image, calculate driver relative to head rotation angle during registration;
From the head rotation angle of every frame, calculate the time scale of driver's head pitch attitude in present period.
Further, described traffic information is collected and is comprised the following steps:
According to the video image that scene camera is taken at present period, obtain the lane boundary information of driver vehicle-surroundings;
According to the video image that scene camera is taken at present period, obtain the vehicle distributed intelligence of driver vehicle-surroundings;
According to the video image that scene camera is taken at present period, obtain pedestrian's distributed intelligence of driver vehicle-surroundings.
Further, described vehicle performance state is extracted and is comprised the following steps:
Calculate mean value respectively in present period of the rotational angle of steering vehicle bearing circle and velocity of rotation and standard deviation;
Calculate mean value respectively in present period of the rotational angle of steering vehicle gas pedal and velocity of rotation and standard deviation;
The bearing circle of combining road condition information and steering vehicle and the driving condition parameter of gas pedal, form 15 dimensional feature vectors, and by this 15 dimensional feature vector input neural network regression model, calculate driver at the score value of present period to vehicle performance state.
Further, described fatigue driving detects and comprises the following steps:
Add up the timestamp of driving time as driver's emotional state, limbs state and vehicle performance state using present period, present period is added up the transport condition value of driving time, be updated in the historical data of following the tracks of drivers working states change;
From the drivers working states tracking data upgraded, extract 5 transport condition subsequences, the data corresponding with each transport condition subsequence time period are intercepted from transport condition tracking data, and the data extracting 3 constant duration points from the data segment intercepted insert corresponding traveling between state subgroup sequence data, after combination, reconstruct a new transport condition subsequence;
Operate time series model, resolves each new transport condition subsequence, and whether existence declines each new transport condition subsequence to judge driver status;
Combine the judged result of 5 new transport condition subsequences, calculate the fatigue exponent of current time driver, to judge whether current time driver is in fatigue driving.
The present invention also provides a kind of fatigue driving detecting system, and this system comprises:
Driver identity authentication ' unit, is suitable for according to the video image of driver at present period, by recognition of face, just carries out certification at the driver identity of steering vehicle to current;
Driving time accumulated unit, is suitable for the identity authentication result according to driver, obtains the driving time record of driver;
Emotional state extraction unit, is suitable for according to the video image of driver at present period, extracts the emotional state of driver at present period;
Limbs state extraction unit, is suitable for according to the video image of driver at present period, extracts the limbs state of driver at present period;
Traffic information collector unit, is suitable for the video image taken at present period according to scene camera, obtains the environmental information of driver vehicle-surroundings;
Vehicle performance state extraction unit, is suitable for the driving condition of combining road condition information and vehicle, extracts driver in the manipulation state of present period to vehicle;
Fatigue driving detecting unit, be suitable for adding up the timestamp of driving time as driver's emotional state, limbs state and vehicle performance state using present period, upgrade the data of following the tracks of drivers working states change, analyze the state change of driver in more new data, judge whether current time driver is in fatigue driving.
Fatigue driving detecting system provided by the invention, by recognition of face, tracking driver's facial expression and limb action are passed in time, present and change from normal condition to fatigue state, such as closed-eye time is elongated, the frequency of bowing increase etc.; Follow the tracks of driver vehicle to manipulate state and whether decline in time, such as rotating of steering wheel variance becomes large etc., detects driver whether be in fatigue driving state with this.Fatigue driving detecting system provided by the invention avoids and presets one and define threshold value, and its accuracy is by the impact of driver's individual difference.
Accompanying drawing explanation
Fig. 1 is method for detecting fatigue driving logical schematic provided by the invention.
Embodiment
The technological means realized to make the present invention, creation characteristic, reaching object and effect is easy to understand, below in conjunction with concrete diagram, setting forth the present invention further.
Please refer to shown in Fig. 1, a kind of method for detecting fatigue driving, the method comprises the following steps:
Driver identity certification 1, according to the video image of driver at present period, by recognition of face, just carries out certification at the driver identity of steering vehicle to current;
Driving time accumulative 2, according to the identity authentication result of driver, obtains the driving time record of driver;
Emotional state extracts 3, according to the video image of driver at present period, extracts the emotional state of driver at present period;
Limbs state extracts 4, according to the video image of driver at present period, extracts the limbs state of driver at present period;
Traffic information collects 5, according to the video image that scene camera is taken at present period, obtains the environmental information of driver vehicle-surroundings;
Vehicle performance state extracts 6, the driving condition of combining road condition information and vehicle, extracts driver in the manipulation state of present period to vehicle;
Fatigue driving detects 7, the timestamp of driving time as driver's emotional state, limbs state and vehicle performance state is added up using present period, upgrade the data of following the tracks of drivers working states change, analyze the state change of driver in more new data, judge whether current time driver is in fatigue driving.
Method for detecting fatigue driving provided by the invention, by recognition of face, tracking driver's facial expression and limb action are passed in time, present and change from normal condition to fatigue state, such as closed-eye time is elongated, the frequency of bowing increase etc.; Follow the tracks of driver vehicle to manipulate state and whether decline in time, such as rotating of steering wheel variance becomes large etc., detects driver whether be in fatigue driving state with this.Method for detecting fatigue driving provided by the invention avoids and presets one and define threshold value, and its accuracy is by the impact of driver's individual difference.
As specific embodiment, described driver identity certification comprises the following steps:
Step 10, according to the video image of driver at present period, extract the key point of driver's face area in start-stop frame/intermediate frame video image.Particularly, in driver's video image of start-stop frame/intermediate frame, ＂ viola-johns ＂ (viola-Jones) algorithm frame is used to detect the rectangular area at driver face place; And utilize ＂ superviseddescentmethod ＂ (having the Gradient Descent of supervision) algorithm in the face rectangular area detected, extract 31 key points of face.These 31 key points are mainly distributed in the eyes of face, nose and lip, particularly: have 5 key points to be distributed on the bridge of the nose and the wing of nose, be referred to as the key point of nasal area; There are 6 key points to be distributed on right eye on palpebra inferior, are referred to as the key point of right eye region; There are 6 key points to be distributed on left eye on palpebra inferior, are referred to as the key point of left eye region; Have 12 key points to be distributed on mouth outline, have 1 key point to be distributed in the lower limb central point of upper lip, have 1 key point to be distributed in the coboundary central point of lower lip, these 14 key points are referred to as the key point of mouth region.Wherein, described ＂ viola-johns ＂ algorithm and ＂ superviseddescentmethod ＂ algorithm are the known technology means of this area, do not repeat them here.
Step 11, relative position according to key point, carry out size normalization to face area; Specifically can carry out affined transformation to 31 key points, the rectangular area at face place is mapped in the template of 128 × 128 sizes.Wherein, described affined transformation is the known technology means of this area, does not repeat them here.
Step 12, by convolutional neural networks, from the face area after size normalization, extract face feature.Particularly, adopt convolutional neural networks, from face area, extract face feature, namely by the convolution kernel of convolutional neural networks, to the image pixel value in face area, adopt a series of different convolution kernel to carry out convolution algorithm and obtain a series of convolution value, in this, as facial eigenvectors.Described convolutional neural networks can adopt technological means well known in the art, and preferably, the convolutional neural networks that the present invention adopts 1 input layer, 5 hidden layers and 1 output layer to form, extracting dimension is the facial eigenvectors of 1024.
Step 13, utilization probability Linear analysis model (plda), calculate in present period start-stop frame/intermediate frame, the driver's face feature extracted and the similarity of registered driver's face feature, if the driver of a registered mistake, the face feature extracted in its face feature and start-stop frame/intermediate frame has the similarity (output valve of plda) being more than or equal to 0.9, then the identity of present period driver is exactly this registered driver; Otherwise, if the driver of a registered mistake cannot be found, the face feature extracted in its face feature and start-stop frame/intermediate frame has the similarity being more than or equal to 0.9, namely both similarities are less than 0.9, then need to register the driver of present period, specifically with the face area that present period intermediate frame detects, as the identity marks of this driver.
As specific embodiment, the driving time record of described driver just generates when driver login, and constantly update in the process of moving.Particularly, the last time this driver vehicle being detected before driving time record comprises, and driver is to driving time accumulative during the last detection.Thus this driving time record, what record to a certain extent is the time that ＂ drives ＂ continuously, and it will as the timestamp following the tracks of drivers working states change.But in the present invention, it is not that free of discontinuities is truly driven that its ＂ drives ＂ continuously, and refers to that driving time is interrupted and be less than 4 hours, be namely considered to ＂ and drive ＂ continuously.Thus, as a kind of embodiment, when upgrading driving time record, described driving time is accumulative to be comprised the following steps:
If the last time detects the time of this driver vehicle, 4 hours are more than or equal to the lead time of current time, then the last time is detected that the time of this driver vehicle replaces with current time, driving time accumulative when the last time is detected replaces with 120 seconds, and face area during driver login is replaced with the face area that present period intermediate frame detects;
If the last time detects the time of this driver vehicle, 4 hours are less than with the lead time of current time, then the last time is detected that the time of this driver vehicle replaces with current time, cumulative 120 seconds of driving time accumulative when the last time is detected.
In driving time accumulating step, described driving time accumulative when the last time is detected is cumulative or replace with 120 seconds, be because in the detection method of the application, described driver identity authenticating step just identified once the driver of current steering vehicle every 120 seconds.Certainly, those skilled in the art, on the basis of aforementioned interval time, can also carry out corresponding conversion according to actual detection mode and arrange.
As specific embodiment, described emotional state extracts and comprises the following steps:
Step 30, according to the video image of driver at present period, extract eye key point and the mouth key point of driver in every frame video image.Particularly, in the video image of every frame driver, adopt the algorithm identical with step 10, ＂ viola-johns ＂ algorithm is used to detect the face area of driver, use ＂ superviseddescentmethod ＂ (having the Gradient Descent of supervision) algorithm to detect 31 key points again, these 31 key points are mainly distributed in the eyes of face, nose and lip.Described ＂ viola-johns ＂ algorithm and ＂ superviseddescentmethod ＂ algorithm are techniques well known means, repeat no more.
Step 31, relative position according to eye key point in every frame video image, calculate the form parameter of eyelid profile.Particularly, by the key point of palpebra inferior upper in every frame video image, by ＂ least square method ＂, an ellipse is simulated, as the profile describing upper palpebra inferior; Calculate the ratio between longitudinal axis radius and transverse axis radius in elliptic contour, as the form parameter describing driver's eyelid profile.This eyelid contour shape parameter can reflect that the eyes of driver are from change when opening closed, so as a feature judge driver be in clear-headed, or sleep, or hypnagogic state.When the clear-headed driving of driver, eyes are opened completely, and when driver fatigue is fallen asleep, eyes close completely, therefore compared with during eyes closed, have the larger radius of axle in length and breadth when eyes are opened.
Step 32, in present period the eyelid contour shape parameter of every frame image, calculate the distribution of shapes of driver at present period palpebra interna profile.Particularly, the codomain [0 at above-mentioned eyelid contour shape parameter place, 0.6] four sub-ranges are divided into: [0,0.15], [0.15,0.3], [0.3,0.45], [0.45,0.6], sub-range residing for metric, carries out disperse segmentaly, respectively the shape of corresponding four kinds of eyelid profiles.For each in four eyelid contour shapes, calculate in present period, comprise the video image frame number of this eyelid contour shape and the ratio during this period of time between interior video image totalframes.
Step 33, relative position according to mouth key point in every frame video image, judge the opening and closing state of mouth.Particularly, by the key point of lip outline in every frame video image, simulate an ellipse, as the shape describing lip outline; Then the ratio in elliptic contour between longitudinal axis radius and transverse axis radius is calculated, as the metric describing driver's mouth opening and closing state.Compared to conjunction mouth, there is when opening one's mouth the larger radius of axle in length and breadth.
Step 34, from the opening and closing status switch of mouth, calculate the number of times of driver yawn in present period.Particularly, use ＂ Hidden Markov ＂ time series models, resolve the metric of mouth opening and closing state and the change between adjacent image frame thereof in every frame video image, thus in the opening and closing status switch of mouth, detect the subsequence that yawn occurs, add up the number of yawn subsequence again, as the yawn number of times at present period.Wherein, described ＂ Hidden Markov ＂ time series models are techniques well known means, repeat no more.
So far, according to the video image of driver at present period, complete the emotional state of driver in present period and extract, described emotional state comprises eyes open and-shut mode and mouth opening and closing state.
As specific embodiment, described limbs state is extracted and is comprised the following steps:
Step 40, according to the video image of driver at present period, extract the key point of driver's face area in every frame video image.Particularly, adopt the algorithm identical with step 10, detect the rectangular area of driver place face in every frame video image, and 31 key points in face area.
Step 41, by the position of face's key point in more every frame video image with face's key point during driver login, calculate driver relative to head rotation angle when registering.Particularly, adopt posit algorithm, in more every frame video image, the relative position of face area key point when 31 face's key points and driver login, calculates the parameter projected between them, i.e. rotational angle in three dimensions, comprises the angle of pitch, crab angle and roll angle.Relative to the video image of sitting normal during driver login, the described angle of pitch refers to and rotates formed angle downward or upward around the transverse axis on the plane of delineation, described crab angle refers to the angle formed around the Y rotation with in surface on the plane of delineation, and described roll angle refers to the angle that the rotation in the plane of delineation is formed.Wherein, described ＂ posit ＂ algorithm is techniques well known means, repeats no more.
Step 42, from the head rotation angle of every frame, calculate the time scale of driver head pitch attitude in present period.Particularly, get the angle of pitch of head, describe the change of driver head's pitch attitude.The codomain at this metric place [-180 °, 180 °] be divided into 5 sub-ranges, its sub-range specifically comprises [-180 ° ,-45 °], [-45 ° ,-15 °], [-15 °, 15 °], [15 °, 45 °], [45 °, 180 °], the sub-range residing for metric, carry out disperse segmentaly, the angle of corresponding five head pitching respectively.For each in five head luffing angles, calculate in present period, comprise the video image frame number of this head luffing angle and the ratio during this period of time between interior video image totalframes, obtain the time scale of driver's head luffing angle in present period thus.
As specific embodiment, described traffic information is collected and is comprised the following steps:
Step 50, the video image taken at present period according to scene camera, obtain the lane boundary information of driver vehicle-surroundings.Particularly, to each two field picture from scene camera, carry out marginal point extraction by Canny operator, then use Hough transform matching quadratic polynomial from marginal point, as the absorbing boundary equation in track.
With the quadratic term coefficient of absorbing boundary equation, as the metric describing lane curvature.
With the Monomial coefficient of absorbing boundary equation, as the metric describing direction, track.
With the constant term of absorbing boundary equation, depart from the metric on (run-off-road border) as description vehicle lane border.
Wherein, described ＂ Canny operator ＂, ＂ Hough transform ＂ is techniques well known means, repeats no more.
Step 51, the video image taken at present period according to scene camera, obtain the vehicle distributed intelligence of driver vehicle-surroundings.Particularly, to each two field picture taken from scene camera, adopt the vehicle detecting algorithm detecting framework based on ＂ HarrWavelet ＂ feature and ＂ SVM ＂, detect the vehicle in image.Based on the image-forming principle of the near big and far smaller ＂ of ＂, the present invention, to detect the size at vehicle place rectangle frame (vehicle detects frame) in image, detects the metric of the spacing of vehicle and current driver's steering vehicle as description.Detect vehicle from every two field picture, select 3 cars detecting vehicle place rectangle frame size maximum (namely distance is the shortest), by following formula (1) calculating in this two field picture place moment, the traffic density of driver vehicle-surroundings.
If this frame is without detecting vehicle, then traffic density is 0.Wherein, described ＂ HarrWavelet ＂, ＂ SVM ＂ is techniques well known means, repeats no more.
Step 52, the video image taken at present period according to scene camera, obtain pedestrian's distributed intelligence of driver vehicle-surroundings.Particularly, to each two field picture taken from scene camera, the same vehicle detecting algorithm adopted based on ＂ HarrWavelet ＂ feature and ＂ SVM ＂ detection framework, detects the pedestrian in image.Based on the image-forming principle of the near big and far smaller ＂ of ＂, the present invention, to detect the size (the rectangle frame area at pedestrian place) at pedestrian place rectangle frame (pedestrian detects frame) in image, detects the metric of the spacing of pedestrian and current driver's steering vehicle as description.Detect vehicle from every two field picture, select 3 pedestrians of rectangle frame size maximum (namely apart from the shortest), calculate this two field picture place moment by following formula (2), the pedestrian density of driver vehicle-surroundings.
If this frame is without detecting pedestrian, pedestrian density is 0.
As specific embodiment, described vehicle performance state is extracted and is comprised the following steps:
Mean value respectively in present period of step 60, the rotational angle calculating steering vehicle bearing circle and velocity of rotation and standard deviation.Particularly, with each sampling time point in current slot for transverse axis, with the rotating of steering wheel angle on sampling time point each in current slot for the longitudinal axis, to obtain in current slot rotating of steering wheel angle relative to the waveform of time variations.By ＂ crosszero ＂ algorithm, in current slot rotating of steering wheel angle waveform, extract rotational angle Rapid Variable Design (as amplitude >30 °, width <10s) crest and trough, then add up the number of crest and trough in present period.Wherein, described ＂ crosszero ＂ algorithm is techniques well known means, repeats no more.
Mean value respectively in present period of step 61, the rotational angle calculating steering vehicle gas pedal and velocity of rotation and standard deviation.Particularly, mean value in present period of the rotational angle of described gas pedal and standard deviation, and the mean value of the velocity of rotation of described gas pedal in present period and standard deviation, with calculate mean value respectively in present period of the rotational angle of steering vehicle bearing circle and velocity of rotation in described step 60 and standard deviation similar, do not repeat them here.
The bearing circle of step 62, combining road condition information and steering vehicle and the driving condition parameter of gas pedal, form 15 dimensional feature vectors, and by this 15 dimensional feature vector input neural network regression model, calculate driver at the score value of present period to vehicle performance state.Particularly, the ＂ lane curvature using step 50 to obtain, direction, track, lane boundary depart from ＂, the ＂ traffic density ＂ that step 51 obtains, the ＂ pedestrian density ＂ that step 52 obtains, the mean value of the mean value of the ＂ rotating of steering wheel angle that step 60 obtains and standard deviation, rotating of steering wheel speed and standard deviation, rotating of steering wheel angle crest and trough number ＂, the mean value of the mean value of the ＂ gas pedal rotational angle that step 61 obtains and standard deviation, gas pedal velocity of rotation and standard deviation ＂ totally 15 driving condition parameters, form 15 dimensional feature vectors.And make ＂ neural network ＂ as prediction driver to the regression model of vehicle performance state, using the input of described 15 dimensional feature vectors as this model, output valve is and describes driver to the score value of vehicle performance state, its scope is between 0 ~ 1, its output valve is larger, represents that the vehicle performance state of this driver is better.Wherein, described ＂ neural network ＂ is techniques well known means, repeats no more.
As specific embodiment, described fatigue driving detects and comprises the following steps:
Step 70, add up the timestamp of driving time as driver's emotional state, limbs state and vehicle performance state using present period, present period is added up the transport condition value of driving time, be updated in the historical data of following the tracks of drivers working states change.Particularly, the form parameter of present period driver eyelid profile, mouth opening and closing state value, head pitch attitude value and vehicle performance state value, as the transport condition value of current accumulative driving time, add in the historical data of following the tracks of drivers working states change, its historical data is upgraded.
Step 71, from upgrade drivers working states tracking data, extract 5 transport condition subsequences, the data corresponding with each transport condition subsequence time period are intercepted from transport condition tracking data, and the data extracting 3 constant duration points from the data segment intercepted insert corresponding traveling between state subgroup sequence data, after combination, reconstruct a new transport condition subsequence.Particularly, the data of initial five time points in transport condition data are got, respectively as the starting point of five transport condition subsequences; Get the data stopping five time points in transport condition data, respectively as the terminal of five transport condition subsequences.To each transport condition subsequence, using its start time and terminal as terminal, from transport condition data, intercept the data of corresponding time period; And from each data segment intercepted, the data extracting 3 time points of constant duration, then, between the starting point sequentially inserting this transport condition subsequence and endpoint data, combine with starting point and endpoint data, form a new transport condition subsequence be made up of 5 time points.Therefore, the new transport condition subsequence be made up of 5 time points, its data formed are randomly drawed, and add the follow-up authenticity that judges accordingly and accuracy.
Step 72, operate time series model, resolve each new transport condition subsequence, and whether existence declines in each new transport condition subsequence to judge driver status.Particularly, to each new transport condition subsequence, use ＂ Hidden Markov ＂ time series models, resolve the transport condition of each time point in subsequence, and transport condition is at the diff of its adjacent two time points, thus whether judge driver status in this transport condition subsequence in decline, if, this subsequence is then referred to as ＂ state decline subsequence ＂, and from ＂ Hidden Markov ＂ time series models, obtain the degree of confidence (i.e. state decline degree of confidence) of described judgement, as the metric describing this subsequence state decline degree.
The judged result of step 73, associating 5 new transport condition subsequences, calculates the fatigue exponent of current time driver, to judge whether current time driver is in fatigue driving.Particularly, based on number and the degree of confidence of ＂ state decline subsequence ＂, calculated the fatigue exponent of current time driver by following formula (3).
If fatigue exponent is greater than 3, judge that current time driver is in fatigue state.
The present invention also provides a kind of fatigue driving detecting system, and this system comprises:
Driver identity authentication ' unit, is suitable for according to the video image of driver at present period, by recognition of face, just carries out certification at the driver identity of steering vehicle to current;
Driving time accumulated unit, is suitable for the identity authentication result according to driver, obtains the driving time record of driver;
Emotional state extraction unit, is suitable for according to the video image of driver at present period, extracts the emotional state of driver at present period;
Limbs state extraction unit, is suitable for according to the video image of driver at present period, extracts the limbs state of driver at present period;
Traffic information collector unit, is suitable for the video image taken at present period according to scene camera, obtains the environmental information of driver vehicle-surroundings;
Vehicle performance state extraction unit, is suitable for the driving condition of combining road condition information and vehicle, extracts driver in the manipulation state of present period to vehicle;
Fatigue driving detecting unit, be suitable for adding up the timestamp of driving time as driver's emotional state, limbs state and vehicle performance state using present period, upgrade the data of following the tracks of drivers working states change, analyze the state change of driver in more new data, judge whether current time driver is in fatigue driving.
Fatigue driving detecting system provided by the invention, by recognition of face, tracking driver's facial expression and limb action are passed in time, present and change from normal condition to fatigue state, such as closed-eye time is elongated, the frequency of bowing increase etc.; Follow the tracks of driver vehicle to manipulate state and whether decline in time, such as rotating of steering wheel variance becomes large etc., detects driver whether be in fatigue driving state with this.Fatigue driving detecting system provided by the invention avoids and presets one and define threshold value, and its accuracy is by the impact of driver's individual difference.
As specific embodiment, the implementation of described fatigue driving detecting system and aforementioned method for detecting fatigue driving is similar, does not repeat them here.
These are only embodiments of the present invention; not thereby the scope of the claims of the present invention is limited; every equivalent structure utilizing instructions of the present invention and accompanying drawing content to do, is directly or indirectly used in the technical field that other are relevant, all in like manner within scope of patent protection of the present invention.
Claims (9)
1. a method for detecting fatigue driving, is characterized in that, the method comprises the following steps:
Driver identity certification, according to the video image of driver at present period, by recognition of face, just carries out certification at the driver identity of steering vehicle to current;
Driving time adds up, and according to the identity authentication result of driver, obtains the driving time record of driver;
Emotional state extracts, and according to the video image of driver at present period, extracts the emotional state of driver at present period;
Limbs state is extracted, and according to the video image of driver at present period, extracts the limbs state of driver at present period;
Traffic information is collected, and according to the video image that scene camera is taken at present period, obtains the environmental information of driver vehicle-surroundings;
Vehicle performance state is extracted, the driving condition of combining road condition information and vehicle, extracts driver in the manipulation state of present period to vehicle;
Fatigue driving detects, the timestamp of driving time as driver's emotional state, limbs state and vehicle performance state is added up using present period, upgrade the data of following the tracks of drivers working states change, analyze the state change of driver in more new data, judge whether current time driver is in fatigue driving.
2. method for detecting fatigue driving according to claim 1, is characterized in that, described driver identity certification comprises the following steps:
According to the video image of driver at present period, extract the key point of driver's face area in start-stop frame/intermediate frame video image;
According to the relative position of key point, size normalization is carried out to face area;
By convolutional neural networks, from the face area after size normalization, extract face feature;
Use probability Linear analysis model, calculate the similarity of current extraction face feature and registered driver's face feature, if similarity is more than or equal to 0.9, then the identity of present period driver is exactly this registered driver; If similarity is less than 0.9, then need to register the driver of present period, the face area detected with present period intermediate frame, as the identity marks of this driver.
3. method for detecting fatigue driving according to claim 1, is characterized in that, described driving time is accumulative to be comprised the following steps:
If the last time detects the time of this driver vehicle, 4 hours are more than or equal to the lead time of current time, then the last time is detected that the time of this driver vehicle replaces with current time, driving time accumulative when the last time is detected replaces with 120 seconds, and face area during driver login is replaced with the face area that present period intermediate frame detects;
If the last time detects the time of this driver vehicle, 4 hours are less than with the lead time of current time, then the last time is detected that the time of this driver vehicle replaces with current time, cumulative 120 seconds of driving time accumulative when the last time is detected.
4. method for detecting fatigue driving according to claim 1, is characterized in that, described emotional state extracts and comprises the following steps:
According to the video image of driver at present period, extract eye key point and the mouth key point of driver in every frame video image;
According to the relative position of eye key point in every frame video image, calculate the form parameter of eyelid profile;
The eyelid contour shape parameter of every frame image in present period, calculates the distribution of shapes of driver at present period palpebra interna profile;
According to the relative position of mouth key point in every frame video image, judge the opening and closing state of mouth;
From the opening and closing status switch of mouth, calculate the number of times of driver's yawn in present period.
5. method for detecting fatigue driving according to claim 1, is characterized in that, described limbs state is extracted and comprised the following steps:
According to the video image of driver at present period, extract the key point of driver's face area in every frame video image;
By the position of face's key point when face's key point and driver login in more every frame video image, calculate driver relative to head rotation angle during registration;
From the head rotation angle of every frame, calculate the time scale of driver's head pitch attitude in present period.
6. method for detecting fatigue driving according to claim 1, is characterized in that, described traffic information is collected and comprised the following steps:
According to the video image that scene camera is taken at present period, obtain the lane boundary information of driver vehicle-surroundings;
According to the video image that scene camera is taken at present period, obtain the vehicle distributed intelligence of driver vehicle-surroundings;
According to the video image that scene camera is taken at present period, obtain pedestrian's distributed intelligence of driver vehicle-surroundings.
7. method for detecting fatigue driving according to claim 1, is characterized in that, described vehicle performance state is extracted and comprised the following steps:
Calculate mean value respectively in present period of the rotational angle of steering vehicle bearing circle and velocity of rotation and standard deviation;
Calculate mean value respectively in present period of the rotational angle of steering vehicle gas pedal and velocity of rotation and standard deviation;
The bearing circle of combining road condition information and steering vehicle and the driving condition parameter of gas pedal, form 15 dimensional feature vectors, and by this 15 dimensional feature vector input neural network regression model, calculate driver at the score value of present period to vehicle performance state.
8. method for detecting fatigue driving according to claim 1, is characterized in that, described fatigue driving detects and comprises the following steps:
Add up the timestamp of driving time as driver's emotional state, limbs state and vehicle performance state using present period, present period is added up the transport condition value of driving time, be updated in the historical data of following the tracks of drivers working states change;
From the drivers working states tracking data upgraded, extract 5 transport condition subsequences, the data corresponding with each transport condition subsequence time period are intercepted from transport condition tracking data, and the data extracting 3 constant duration points from the data segment intercepted insert corresponding traveling between state subgroup sequence data, after combination, reconstruct a new transport condition subsequence;
Operate time series model, resolves each new transport condition subsequence, and whether existence declines each new transport condition subsequence to judge driver status;
Combine the judged result of 5 new transport condition subsequences, calculate the fatigue exponent of current time driver, to judge whether current time driver is in fatigue driving.
9. a fatigue driving detecting system, is characterized in that, this system comprises:
Driver identity authentication ' unit, is suitable for according to the video image of driver at present period, by recognition of face, just carries out certification at the driver identity of steering vehicle to current;
Driving time accumulated unit, is suitable for the identity authentication result according to driver, obtains the driving time record of driver;
Emotional state extraction unit, is suitable for according to the video image of driver at present period, extracts the emotional state of driver at present period;
Limbs state extraction unit, is suitable for according to the video image of driver at present period, extracts the limbs state of driver at present period;
Traffic information collector unit, is suitable for the video image taken at present period according to scene camera, obtains the environmental information of driver vehicle-surroundings;
Vehicle performance state extraction unit, is suitable for the driving condition of combining road condition information and vehicle, extracts driver in the manipulation state of present period to vehicle;
Fatigue driving detecting unit, be suitable for adding up the timestamp of driving time as driver's emotional state, limbs state and vehicle performance state using present period, upgrade the data of following the tracks of drivers working states change, analyze the state change of driver in more new data, judge whether current time driver is in fatigue driving.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510475351.2A CN105096528B (en) | 2015-08-05 | 2015-08-05 | A kind of method for detecting fatigue driving and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510475351.2A CN105096528B (en) | 2015-08-05 | 2015-08-05 | A kind of method for detecting fatigue driving and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105096528A true CN105096528A (en) | 2015-11-25 |
| CN105096528B CN105096528B (en) | 2017-07-11 |
Family
ID=54576850
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510475351.2A Active CN105096528B (en) | 2015-08-05 | 2015-08-05 | A kind of method for detecting fatigue driving and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105096528B (en) |
Cited By (55)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105354987A (en) * | 2015-11-26 | 2016-02-24 | 南京工程学院 | Vehicle fatigue driving detection and identity authentication apparatus, and detection method thereof |
| CN105769120A (en) * | 2016-01-27 | 2016-07-20 | 深圳地平线机器人科技有限公司 | Fatigue driving detection method and device |
| CN105809770A (en) * | 2016-03-25 | 2016-07-27 | 安徽机电职业技术学院 | Automobile traveling data recorder with driver identity recognition function |
| CN106394673A (en) * | 2016-03-10 | 2017-02-15 | 吉林大学 | Driver attention monitoring and warning system suitable for uneven road surface |
| CN106710145A (en) * | 2016-12-29 | 2017-05-24 | 清华大学苏州汽车研究院（吴江） | Guided driver tiredness prevention method |
| CN106740862A (en) * | 2016-11-29 | 2017-05-31 | 深圳市元征科技股份有限公司 | Driver status monitoring method and monitoring controller for driver state |
| CN107007292A (en) * | 2015-12-10 | 2017-08-04 | 罗伯特·博世有限公司 | Method for knowing fatigue |
| CN107194346A (en) * | 2017-05-19 | 2017-09-22 | 福建师范大学 | A kind of fatigue drive of car Forecasting Methodology |
| CN107697069A (en) * | 2017-10-31 | 2018-02-16 | 上海汽车集团股份有限公司 | Fatigue of automobile driver driving intelligent control method |
| CN107704836A (en) * | 2017-10-17 | 2018-02-16 | 电子科技大学 | Method for detecting fatigue driving based on object detection |
| WO2018058266A1 (en) * | 2016-09-27 | 2018-04-05 | 深圳智乐信息科技有限公司 | Mobile terminal-based notification method and system |
| CN108021875A (en) * | 2017-11-27 | 2018-05-11 | 上海灵至科技有限公司 | A kind of vehicle driver's personalization fatigue monitoring and method for early warning |
| CN108153418A (en) * | 2017-12-25 | 2018-06-12 | 广州市尊浪电器有限公司 | A kind of sleeping speaker |
| CN108238053A (en) * | 2017-12-15 | 2018-07-03 | 北京车和家信息技术有限公司 | A kind of vehicle drive monitoring method, device and vehicle |
| CN108407813A (en) * | 2018-01-25 | 2018-08-17 | 惠州市德赛西威汽车电子股份有限公司 | A kind of antifatigue safe driving method of vehicle based on big data |
| CN108460345A (en) * | 2018-02-08 | 2018-08-28 | 电子科技大学 | A kind of facial fatigue detection method based on face key point location |
| CN108492404A (en) * | 2018-02-07 | 2018-09-04 | 上海灵至科技有限公司 | A kind of the face lock method for unlocking and device of additional Expression Recognition |
| CN108545080A (en) * | 2018-03-20 | 2018-09-18 | 北京理工大学 | Driver Fatigue Detection and system |
| CN108609019A (en) * | 2018-05-09 | 2018-10-02 | 浙江金之路信息科技有限公司 | A kind of electric vehicle automatic Pilot method based on artificial intelligence platform |
| CN108647708A (en) * | 2018-04-28 | 2018-10-12 | 清华-伯克利深圳学院筹备办公室 | Driver evaluation's method, apparatus, equipment and storage medium |
| CN108791299A (en) * | 2018-05-16 | 2018-11-13 | 浙江零跑科技有限公司 | A kind of driving fatigue detection of view-based access control model and early warning system and method |
| CN108960065A (en) * | 2018-06-01 | 2018-12-07 | 浙江零跑科技有限公司 | A kind of driving behavior detection method of view-based access control model |
| CN108961669A (en) * | 2018-07-19 | 2018-12-07 | 上海小蚁科技有限公司 | The safe early warning method and device, storage medium, server of net about vehicle |
| CN109191788A (en) * | 2018-09-11 | 2019-01-11 | 吉林大学 | Driver tired driving judgment method, storage medium and electronic equipment |
| CN109215293A (en) * | 2018-11-23 | 2019-01-15 | 深圳市元征科技股份有限公司 | A kind of method for detecting fatigue driving, device and vehicle-mounted terminal equipment |
| CN109376682A (en) * | 2018-11-06 | 2019-02-22 | 东莞市凯木金电子科技有限公司 | A kind of intelligent video camera head and fatigue state recognition method |
| CN109426765A (en) * | 2017-08-23 | 2019-03-05 | 厦门雅迅网络股份有限公司 | Driving dangerousness mood based reminding method, terminal device and storage medium |
| CN109711343A (en) * | 2018-12-27 | 2019-05-03 | 北京思图场景数据科技服务有限公司 | Behavioral structure method based on the tracking of expression, gesture recognition and expression in the eyes |
| CN109747656A (en) * | 2017-11-02 | 2019-05-14 | 罗曦明 | Artificial intelligence vehicle assistant drive method, apparatus, equipment and storage medium |
| CN109760689A (en) * | 2017-11-09 | 2019-05-17 | 丰田自动车株式会社 | Driver monitoring system for vehicle |
| CN109803583A (en) * | 2017-08-10 | 2019-05-24 | 北京市商汤科技开发有限公司 | Driver monitoring method, apparatus and electronic equipment |
| CN109934158A (en) * | 2019-03-11 | 2019-06-25 | 合肥工业大学 | Video feeling recognition methods based on local strengthening motion history figure and recursive convolution neural network |
| CN110063734A (en) * | 2019-03-22 | 2019-07-30 | 中国人民解放军空军特色医学中心 | Fatigue detection method, device, system and the fatigue detecting helmet |
| CN110090018A (en) * | 2019-05-06 | 2019-08-06 | 安徽建筑大学 | A kind of focus analysis system based on brain dateline band |
| CN110163037A (en) * | 2018-03-14 | 2019-08-23 | 北京航空航天大学 | Monitor method, equipment, system, processor and the storage medium of driver status |
| CN110334696A (en) * | 2019-07-31 | 2019-10-15 | 爱驰汽车有限公司 | Cockpit pseudo-experience system, method, equipment and storage medium |
| CN110348340A (en) * | 2019-06-26 | 2019-10-18 | 秒针信息技术有限公司 | The sending method and device of information warning, storage medium and electronic device |
| CN110472556A (en) * | 2019-08-12 | 2019-11-19 | 一汽轿车股份有限公司 | Driver attention's state analysis system and analysis method based on monocular vision |
| CN111027393A (en) * | 2019-11-12 | 2020-04-17 | 上海博泰悦臻电子设备制造有限公司 | Information recommendation method and system and vehicle |
| CN111753586A (en) * | 2019-03-28 | 2020-10-09 | 合肥工业大学 | Fatigue driving monitoring device and method |
| CN111797654A (en) * | 2019-04-09 | 2020-10-20 | Oppo广东移动通信有限公司 | Driver fatigue state detection method and device, storage medium and mobile terminal |
| CN111814615A (en) * | 2020-06-28 | 2020-10-23 | 湘潭大学 | Parkinson non-contact intelligent detection method based on instruction video |
| CN111866056A (en) * | 2020-03-23 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Information pushing method and device, electronic equipment and storage medium |
| CN112036217A (en) * | 2019-06-04 | 2020-12-04 | 初速度（苏州）科技有限公司 | Person state detection method and device based on mouth information |
| CN112309135A (en) * | 2020-01-07 | 2021-02-02 | 常俊杰 | Direction judgment system based on lane detection and corresponding terminal |
| CN112373478A (en) * | 2020-10-10 | 2021-02-19 | 湖南北斗赛格导航科技有限公司 | Fatigue driving early warning method and device |
| CN112399935A (en) * | 2019-06-17 | 2021-02-23 | 谷歌有限责任公司 | Seamless driver authentication using an in-vehicle camera in conjunction with a trusted mobile computing device |
| CN112528919A (en) * | 2020-12-21 | 2021-03-19 | 苏州中科华影健康科技有限公司 | Fatigue driving detection method and device and computer readable medium |
| CN112668393A (en) * | 2020-11-30 | 2021-04-16 | 海纳致远数字科技(上海)有限公司 | Fatigue degree detection device and method based on face recognition and key point detection |
| CN113239729A (en) * | 2021-04-08 | 2021-08-10 | 南京交通职业技术学院 | Fatigue driving identification method based on data fusion |
| CN113449678A (en) * | 2021-07-14 | 2021-09-28 | 宝能(广州)汽车研究院有限公司 | Driving state detection method and device, computer equipment and storage medium |
| CN113642522A (en) * | 2021-09-01 | 2021-11-12 | 中国科学院自动化研究所 | Audio and video based fatigue state detection method and device |
| CN114132331A (en) * | 2021-12-31 | 2022-03-04 | 阿维塔科技(重庆)有限公司 | Driver state information confirmation method, driver state information confirmation device and computer-readable storage medium |
| US11341769B2 (en) | 2017-12-25 | 2022-05-24 | Beijing Sensetime Technology Development Co., Ltd. | Face pose analysis method, electronic device, and storage medium |
| WO2023029966A1 (en) * | 2021-08-31 | 2023-03-09 | 华为技术有限公司 | Identity authentication method, and vehicle |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101540090A (en) * | 2009-04-14 | 2009-09-23 | 华南理工大学 | Driver fatigue monitoring device based on multivariate information fusion and monitoring method thereof |
| CN102717765A (en) * | 2012-07-09 | 2012-10-10 | 湖南赛格导航技术研究有限公司 | Fatigue driving detection method and anti-fatigue driving auxiliary device |
| CN103231693A (en) * | 2013-05-14 | 2013-08-07 | 苏州福丰科技有限公司 | Car alarming and monitoring system based on face recognition |
| US20130226408A1 (en) * | 2011-02-18 | 2013-08-29 | Honda Motor Co., Ltd. | Coordinated vehicle response system and method for driver behavior |
| CN104268591A (en) * | 2014-09-19 | 2015-01-07 | 海信集团有限公司 | Face key point detecting method and device |
| CN104318714A (en) * | 2014-11-12 | 2015-01-28 | 沈阳美行科技有限公司 | Fatigue driving pre-warning method |
-
2015
- 2015-08-05 CN CN201510475351.2A patent/CN105096528B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101540090A (en) * | 2009-04-14 | 2009-09-23 | 华南理工大学 | Driver fatigue monitoring device based on multivariate information fusion and monitoring method thereof |
| US20130226408A1 (en) * | 2011-02-18 | 2013-08-29 | Honda Motor Co., Ltd. | Coordinated vehicle response system and method for driver behavior |
| CN102717765A (en) * | 2012-07-09 | 2012-10-10 | 湖南赛格导航技术研究有限公司 | Fatigue driving detection method and anti-fatigue driving auxiliary device |
| CN103231693A (en) * | 2013-05-14 | 2013-08-07 | 苏州福丰科技有限公司 | Car alarming and monitoring system based on face recognition |
| CN104268591A (en) * | 2014-09-19 | 2015-01-07 | 海信集团有限公司 | Face key point detecting method and device |
| CN104318714A (en) * | 2014-11-12 | 2015-01-28 | 沈阳美行科技有限公司 | Fatigue driving pre-warning method |
Cited By (73)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105354987A (en) * | 2015-11-26 | 2016-02-24 | 南京工程学院 | Vehicle fatigue driving detection and identity authentication apparatus, and detection method thereof |
| CN105354987B (en) * | 2015-11-26 | 2018-06-08 | 南京工程学院 | Vehicle-mounted type fatigue driving detection and identification authentication system and its detection method |
| CN107007292A (en) * | 2015-12-10 | 2017-08-04 | 罗伯特·博世有限公司 | Method for knowing fatigue |
| CN105769120A (en) * | 2016-01-27 | 2016-07-20 | 深圳地平线机器人科技有限公司 | Fatigue driving detection method and device |
| CN105769120B (en) * | 2016-01-27 | 2019-01-22 | 深圳地平线机器人科技有限公司 | Method for detecting fatigue driving and device |
| CN106394673A (en) * | 2016-03-10 | 2017-02-15 | 吉林大学 | Driver attention monitoring and warning system suitable for uneven road surface |
| CN106394673B (en) * | 2016-03-10 | 2018-06-08 | 吉林大学 | A kind of driver attention suitable for uneven road surface monitors and caution system |
| CN105809770A (en) * | 2016-03-25 | 2016-07-27 | 安徽机电职业技术学院 | Automobile traveling data recorder with driver identity recognition function |
| WO2018058266A1 (en) * | 2016-09-27 | 2018-04-05 | 深圳智乐信息科技有限公司 | Mobile terminal-based notification method and system |
| CN106740862A (en) * | 2016-11-29 | 2017-05-31 | 深圳市元征科技股份有限公司 | Driver status monitoring method and monitoring controller for driver state |
| WO2018098947A1 (en) * | 2016-11-29 | 2018-06-07 | 深圳市元征科技股份有限公司 | Method and device for monitoring state of driver |
| CN106710145B (en) * | 2016-12-29 | 2019-10-25 | 清华大学苏州汽车研究院（吴江） | A kind of guiding driver drowsiness prevention method |
| CN106710145A (en) * | 2016-12-29 | 2017-05-24 | 清华大学苏州汽车研究院（吴江） | Guided driver tiredness prevention method |
| CN107194346A (en) * | 2017-05-19 | 2017-09-22 | 福建师范大学 | A kind of fatigue drive of car Forecasting Methodology |
| CN109803583A (en) * | 2017-08-10 | 2019-05-24 | 北京市商汤科技开发有限公司 | Driver monitoring method, apparatus and electronic equipment |
| CN109426765B (en) * | 2017-08-23 | 2023-03-28 | 厦门雅迅网络股份有限公司 | Driving danger emotion reminding method, terminal device and storage medium |
| CN109426765A (en) * | 2017-08-23 | 2019-03-05 | 厦门雅迅网络股份有限公司 | Driving dangerousness mood based reminding method, terminal device and storage medium |
| CN107704836A (en) * | 2017-10-17 | 2018-02-16 | 电子科技大学 | Method for detecting fatigue driving based on object detection |
| CN107704836B (en) * | 2017-10-17 | 2021-10-08 | 电子科技大学 | Fatigue driving detection method based on object detection |
| CN107697069A (en) * | 2017-10-31 | 2018-02-16 | 上海汽车集团股份有限公司 | Fatigue of automobile driver driving intelligent control method |
| CN109747656A (en) * | 2017-11-02 | 2019-05-14 | 罗曦明 | Artificial intelligence vehicle assistant drive method, apparatus, equipment and storage medium |
| CN109760689A (en) * | 2017-11-09 | 2019-05-17 | 丰田自动车株式会社 | Driver monitoring system for vehicle |
| CN108021875A (en) * | 2017-11-27 | 2018-05-11 | 上海灵至科技有限公司 | A kind of vehicle driver's personalization fatigue monitoring and method for early warning |
| CN108238053A (en) * | 2017-12-15 | 2018-07-03 | 北京车和家信息技术有限公司 | A kind of vehicle drive monitoring method, device and vehicle |
| CN108153418A (en) * | 2017-12-25 | 2018-06-12 | 广州市尊浪电器有限公司 | A kind of sleeping speaker |
| US11341769B2 (en) | 2017-12-25 | 2022-05-24 | Beijing Sensetime Technology Development Co., Ltd. | Face pose analysis method, electronic device, and storage medium |
| CN108407813A (en) * | 2018-01-25 | 2018-08-17 | 惠州市德赛西威汽车电子股份有限公司 | A kind of antifatigue safe driving method of vehicle based on big data |
| CN108492404A (en) * | 2018-02-07 | 2018-09-04 | 上海灵至科技有限公司 | A kind of the face lock method for unlocking and device of additional Expression Recognition |
| CN108492404B (en) * | 2018-02-07 | 2020-10-09 | 上海灵至科技有限公司 | Face lock unlocking method and device with additional expression recognition function |
| CN108460345A (en) * | 2018-02-08 | 2018-08-28 | 电子科技大学 | A kind of facial fatigue detection method based on face key point location |
| CN110163037A (en) * | 2018-03-14 | 2019-08-23 | 北京航空航天大学 | Monitor method, equipment, system, processor and the storage medium of driver status |
| CN108545080A (en) * | 2018-03-20 | 2018-09-18 | 北京理工大学 | Driver Fatigue Detection and system |
| CN108647708A (en) * | 2018-04-28 | 2018-10-12 | 清华-伯克利深圳学院筹备办公室 | Driver evaluation's method, apparatus, equipment and storage medium |
| CN108609019B (en) * | 2018-05-09 | 2020-07-10 | 浙江金之路信息科技有限公司 | Automatic electric vehicle driving method based on artificial intelligence platform |
| CN108609019A (en) * | 2018-05-09 | 2018-10-02 | 浙江金之路信息科技有限公司 | A kind of electric vehicle automatic Pilot method based on artificial intelligence platform |
| CN108791299B (en) * | 2018-05-16 | 2020-06-19 | 浙江零跑科技有限公司 | Driving fatigue detection and early warning system and method based on vision |
| CN108791299A (en) * | 2018-05-16 | 2018-11-13 | 浙江零跑科技有限公司 | A kind of driving fatigue detection of view-based access control model and early warning system and method |
| CN108960065A (en) * | 2018-06-01 | 2018-12-07 | 浙江零跑科技有限公司 | A kind of driving behavior detection method of view-based access control model |
| CN108960065B (en) * | 2018-06-01 | 2020-11-17 | 浙江零跑科技有限公司 | Driving behavior detection method based on vision |
| CN108961669A (en) * | 2018-07-19 | 2018-12-07 | 上海小蚁科技有限公司 | The safe early warning method and device, storage medium, server of net about vehicle |
| CN109191788A (en) * | 2018-09-11 | 2019-01-11 | 吉林大学 | Driver tired driving judgment method, storage medium and electronic equipment |
| CN109376682A (en) * | 2018-11-06 | 2019-02-22 | 东莞市凯木金电子科技有限公司 | A kind of intelligent video camera head and fatigue state recognition method |
| CN109215293A (en) * | 2018-11-23 | 2019-01-15 | 深圳市元征科技股份有限公司 | A kind of method for detecting fatigue driving, device and vehicle-mounted terminal equipment |
| CN109711343A (en) * | 2018-12-27 | 2019-05-03 | 北京思图场景数据科技服务有限公司 | Behavioral structure method based on the tracking of expression, gesture recognition and expression in the eyes |
| CN109934158A (en) * | 2019-03-11 | 2019-06-25 | 合肥工业大学 | Video feeling recognition methods based on local strengthening motion history figure and recursive convolution neural network |
| CN110063734A (en) * | 2019-03-22 | 2019-07-30 | 中国人民解放军空军特色医学中心 | Fatigue detection method, device, system and the fatigue detecting helmet |
| CN111753586A (en) * | 2019-03-28 | 2020-10-09 | 合肥工业大学 | Fatigue driving monitoring device and method |
| CN111797654A (en) * | 2019-04-09 | 2020-10-20 | Oppo广东移动通信有限公司 | Driver fatigue state detection method and device, storage medium and mobile terminal |
| CN110090018A (en) * | 2019-05-06 | 2019-08-06 | 安徽建筑大学 | A kind of focus analysis system based on brain dateline band |
| CN112036217B (en) * | 2019-06-04 | 2022-06-21 | 魔门塔(苏州)科技有限公司 | Person state detection method and device based on mouth information |
| CN112036217A (en) * | 2019-06-04 | 2020-12-04 | 初速度（苏州）科技有限公司 | Person state detection method and device based on mouth information |
| CN112399935A (en) * | 2019-06-17 | 2021-02-23 | 谷歌有限责任公司 | Seamless driver authentication using an in-vehicle camera in conjunction with a trusted mobile computing device |
| CN110348340A (en) * | 2019-06-26 | 2019-10-18 | 秒针信息技术有限公司 | The sending method and device of information warning, storage medium and electronic device |
| CN110334696A (en) * | 2019-07-31 | 2019-10-15 | 爱驰汽车有限公司 | Cockpit pseudo-experience system, method, equipment and storage medium |
| CN110472556A (en) * | 2019-08-12 | 2019-11-19 | 一汽轿车股份有限公司 | Driver attention's state analysis system and analysis method based on monocular vision |
| CN110472556B (en) * | 2019-08-12 | 2023-05-23 | 一汽奔腾轿车有限公司 | Monocular vision-based driver attention state analysis system and analysis method |
| CN111027393B (en) * | 2019-11-12 | 2024-05-03 | 博泰车联网科技(上海)股份有限公司 | Information recommendation method and system and vehicle |
| CN111027393A (en) * | 2019-11-12 | 2020-04-17 | 上海博泰悦臻电子设备制造有限公司 | Information recommendation method and system and vehicle |
| CN112309135A (en) * | 2020-01-07 | 2021-02-02 | 常俊杰 | Direction judgment system based on lane detection and corresponding terminal |
| CN111866056B (en) * | 2020-03-23 | 2023-11-21 | 北京嘀嘀无限科技发展有限公司 | Information pushing method, device, electronic equipment and storage medium |
| CN111866056A (en) * | 2020-03-23 | 2020-10-30 | 北京嘀嘀无限科技发展有限公司 | Information pushing method and device, electronic equipment and storage medium |
| CN111814615A (en) * | 2020-06-28 | 2020-10-23 | 湘潭大学 | Parkinson non-contact intelligent detection method based on instruction video |
| CN111814615B (en) * | 2020-06-28 | 2024-04-12 | 中国科学院计算技术研究所 | Parkinson non-contact intelligent detection method based on instruction video |
| CN112373478A (en) * | 2020-10-10 | 2021-02-19 | 湖南北斗赛格导航科技有限公司 | Fatigue driving early warning method and device |
| CN112668393A (en) * | 2020-11-30 | 2021-04-16 | 海纳致远数字科技(上海)有限公司 | Fatigue degree detection device and method based on face recognition and key point detection |
| CN112528919A (en) * | 2020-12-21 | 2021-03-19 | 苏州中科华影健康科技有限公司 | Fatigue driving detection method and device and computer readable medium |
| CN113239729B (en) * | 2021-04-08 | 2024-02-27 | 南京交通职业技术学院 | Fatigue driving identification method based on data fusion |
| CN113239729A (en) * | 2021-04-08 | 2021-08-10 | 南京交通职业技术学院 | Fatigue driving identification method based on data fusion |
| CN113449678A (en) * | 2021-07-14 | 2021-09-28 | 宝能(广州)汽车研究院有限公司 | Driving state detection method and device, computer equipment and storage medium |
| WO2023029966A1 (en) * | 2021-08-31 | 2023-03-09 | 华为技术有限公司 | Identity authentication method, and vehicle |
| CN113642522A (en) * | 2021-09-01 | 2021-11-12 | 中国科学院自动化研究所 | Audio and video based fatigue state detection method and device |
| CN114132331A (en) * | 2021-12-31 | 2022-03-04 | 阿维塔科技(重庆)有限公司 | Driver state information confirmation method, driver state information confirmation device and computer-readable storage medium |
| CN114132331B (en) * | 2021-12-31 | 2024-05-14 | 阿维塔科技(重庆)有限公司 | Driver status information confirmation method, device and computer readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105096528B (en) | 2017-07-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105096528B (en) | A kind of method for detecting fatigue driving and system | |
| CN104616438A (en) | Yawning action detection method for detecting fatigue driving | |
| CN102436715B (en) | Detection method for fatigue driving | |
| CN104269028B (en) | Fatigue driving detection method and system | |
| CN102663452B (en) | Suspicious act detecting method based on video analysis | |
| CN108230254B (en) | Automatic detection method for high-speed traffic full lane line capable of self-adapting scene switching | |
| CN104021370B (en) | The driver status monitoring method and system of a kind of view-based access control model information fusion | |
| CN202257856U (en) | Driver fatigue-driving monitoring device | |
| CN104013414A (en) | Driver fatigue detecting system based on smart mobile phone | |
| CN105654753A (en) | Intelligent vehicle-mounted safe driving assistance method and system | |
| CN104574817A (en) | Machine vision-based fatigue driving pre-warning system suitable for smart phone | |
| CN106687037A (en) | Device, method, and computer program for detecting momentary sleep | |
| CN108446678A (en) | A kind of dangerous driving behavior recognition methods based on skeleton character | |
| CN111460950B (en) | Cognitive distraction method based on head-eye evidence fusion in natural driving conversation behavior | |
| CN104794855A (en) | Driver's attention comprehensive assessment system | |
| CN109145852B (en) | Driver fatigue state identification method based on eye opening and closing state | |
| Xie et al. | Driver fatigue detection based on head gesture and PERCLOS | |
| CN109740477A (en) | Study in Driver Fatigue State Surveillance System and its fatigue detection method | |
| CN105286802A (en) | Driver fatigue detection method based on video information | |
| CN102129690A (en) | Tracking method of human body moving object with environmental disturbance resistance | |
| CN103150870A (en) | Train motorman fatigue detecting method based on videos | |
| CN112016429A (en) | Fatigue driving detection method based on train cab scene | |
| CN109543577A (en) | A kind of fatigue driving detection method for early warning based on facial expression feature | |
| CN107303907A (en) | For the apparatus and method for the sleepiness for determining driver | |
| Yazdi et al. | Driver drowsiness detection by Yawn identification based on depth information and active contour model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: 511457 Guangdong city of Guangzhou province Nansha District Golden Road No. 26 room 1306 (only for office use) Patentee after: Yuncong Technology Group Co.,Ltd. Address before: 511457 Guangdong city of Guangzhou province Nansha District Golden Road No. 26 room 1306 Patentee before: GUANGZHOU YUNCONG INFORMATION TECHNOLOGY Co.,Ltd. |
|
| CP03 | Change of name, title or address | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20220630 Address after: 241003 floor 10, building 9, Service Outsourcing Industrial Park, No. 717, Zhongshan South Road, Wuhu high tech Industrial Development Zone, Yijiang District, Wuhu City, Anhui Province Patentee after: Wuhu Yuncong Technology Co.,Ltd. Address before: 511457 room 1306, No. 26, Jinlong Road, Nansha District, Guangzhou City, Guangdong Province (office use only) Patentee before: Yuncong Technology Group Co.,Ltd. |
|
| TR01 | Transfer of patent right |