CN103020594A - Fatigue state detecting method for eliminating driver individual difference by utilizing online learning - Google Patents
Fatigue state detecting method for eliminating driver individual difference by utilizing online learning Download PDFInfo
- Publication number
- CN103020594A CN103020594A CN2012105059765A CN201210505976A CN103020594A CN 103020594 A CN103020594 A CN 103020594A CN 2012105059765 A CN2012105059765 A CN 2012105059765A CN 201210505976 A CN201210505976 A CN 201210505976A CN 103020594 A CN103020594 A CN 103020594A
- Authority
- CN
- China
- Prior art keywords
- driver
- time
- eyes
- fatigue
- tired
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The invention discloses a fatigue state detecting method for eliminating driver individual difference by utilizing online learning and through eye motion characteristics. The method comprises the following steps of obtaining a video image of a driver through a camera, detecting the face of the driver according to the video image and processing the video image to obtain driver eye motion characteristics; and comparing the driver eye motion characteristics and data under the condition that the driver is sober, wherein a result exceeds a threshold value shows that the driver is in a fatigue driving state, and if the result does not exceed the threshold value, the driver eye motion characteristics are learned on line in the meanwhile, and information obtained through online learning are continuously compared till a vehicle stops. By means of the fatigue state detecting method, the driver eye position characteristics are detected, and driver characteristics are learned to distinguish whether the driver is in the fatigue driving state or not, so that accidental risk is prevented in advance, and accordingly driving safety of the driver is improved.
Description
Technical field
The present invention relates to the automotive safety technical field, relate in particular to a kind of fatigue state detection method of utilizing on-line study to eliminate driver's individual difference.
Background technology
The increase of vehicle guaranteeding organic quantity is so that the quantity that road traffic accident occurs and death tolls are day by day soaring, data shows according to statistics, whole world road traffic accident accounts for about 90% of security incident sum, among unusual death, road traffic accident has become genuine " No.1 killer ".It is one of inducement that causes road traffic accident that driver fatigue is driven, and its probability that causes major traffic accidents is far above other traffic accident behaviors.In China, 20% of the annual accident occupied road traffic hazard that causes because of fatigue driving etc., more than 30% of highway directly causes annual more than 3000 people's death.Yet, fatigue driving extensively exists and often occurs, research and develop high performance driver fatigue state inspection, can effectively avoid driver's state of mind in the Vehicle Driving Cycle process to glide or enter the precarious position such as shallow degree sleep, guarantee traffic safety and personal safety.
Driver fatigue state on-line identification based on machine vision, calculate the technology with very high practical value in field as living things feature recognition and spiritual emotion, with characteristics such as the high-level semantic interpretation of its unique real-time, fatigue characteristic, non-intrudings, and become the technical scheme of tool development potentiality and practical application foreground in the driver fatigue state recognition.
Motion characteristic and the eyeball living features of the present invention's eyelid during according to driver fatigue, driver's microscopic feature was described when never ipsilateral was to fatigue, set up comparatively complete fatigue characteristic space, with respect to indexs such as the PERCLOS that generally uses in the world at present, the longest time of closing one's eyes, frequencies of wink, can be at early detection driver's tired sign more.Aspect the extraction of driver fatigue proper vector, most fatigue detection methods do not fully take into account the invisible of fatigue characteristic and individual difference problem, only relate to the common feature with statistics average meaning, the simple method that adopts fixed threshold to cut apart is classified to clear-headed/fatigue, do not adopt the method for time series analysis the variation of the external appearance features of detected object effectively to be extracted and fully utilization, accuracy of detection is difficult to guarantee.
Summary of the invention
The present invention seeks to: a kind of driver fatigue condition discrimination method with adaptivity function is provided, effectively extract driver's facial fatigue characteristic, for individual difference and the different tired pattern classifier of consubstantiality stability structure of tired performance characteristic.
Technical scheme of the present invention is: the method that a kind of realization fatigue state of eliminating driver's individual difference by the eye motion feature being carried out on-line study detects, utilize the facial video image of camera acquisition driver, adopt based on eyes motion characteristic parameter (such as frequency of wink at the driving task initial stage, blink speed etc.) generic classifier that makes up infers that to fatigue state stage is to adopt time varying characteristic (such as the variation of frequency of wink on the basis of self study, the variation of blink speed etc.) the tired pattern classifier that makes up is realized the identification to the driver fatigue state; Specifically comprise the steps:
(1) gather the facial video image of driver, adopt based on expert's methods of marking of facial video and set up the facial video database of driver, it is 30 seconds video segment that each sample in the database is with Fatigued level label and length;
(2) use process software to test the action of driver's eyelid in each sample and the motion state of eyeball, calculate each fatigue evaluation parameter of describing the eye motion feature, adopt the conspicuousness of method check discriminant parameter difference under the different fatigue level of statistical analysis, and then filter out the eye feature index of describing the driver fatigue state;
(3) with the whole introduced features of the driver's eyes motion characteristic parameter space that filters out, from video database, randomly draw great amount of samples and calculate the characteristic ginseng value of each sample, make up generic classifier according to the class label (clear-headed, tired or serious tired) of characteristic ginseng value and sample; At the driving task initial stage, adopt this universal model that driver's fatigue state is detected in real time;
(4) according to the great amount of samples that in the facial video database of driver, extracts, calculate respectively the variation multiple (time varying characteristic) of each sample characteristics parameter, the conspicuousness of check time varying characteristic difference under the different fatigue level, and then filter out the time varying characteristic index that can differentiate the driver fatigue state; According to the value of time varying characteristic and the class label of sample, and time varying characteristic validity is being carried out on the basis of discriminatory analysis, the off-line design is based on the tired pattern classifier of time varying characteristic; In the actual driving process, algorithm can carry out to the eyes motion characteristic under driver's waking state on-line study and organize in contrast, in the driving task later stage, by relatively the data of Real-time Obtaining and the diversity factor of control group are carried out identification to fatigue state.
Preferably, described step (1) specifically may further comprise the steps:
(11) utilize camera acquisition driver's facial video image, it is 30 seconds video segment that the facial video of the driver who has recorded in the experiment is cut into length according to time sequencing, and each video segment is called a sample;
(12) utilization is marked to the degree of fatigue of driver in each sample based on expert's methods of marking off-line of facial video, wherein regains consciousness and counts 0 minute, and fatigue is counted 1 minute, and serious fatigue is counted 2 minutes; The scoring criterion that adopts is:
Clear-headed---eyes are normally opened, and nictation, the eyeball state was active, concentrates, and keeps to external world notice rapidly, and head is rectified and followed side direction once in a while to cast a glance at, normal facial expression;
Tired---closed trend appears in eyes, and the eyeball active degree descends, and is One's eyesight is restrained, yawn, unexpected improper action, subconsciousness is nodded, or occurs winking, shaking the head etc. and resist tired action, correcting faulty sitting posture frequently;
Serious tired---eyes closed trend is serious, the long period occurs to continue to close one's eyes, and the degree of opening eyes descends, and it is crooked head to occur, nods, and it is wooden express one's feelings, health often occurs in a period of time and loses the ability of driving that continues without any movement response;
(13) several scoring experts are to the scoring average of the same video segment actual fatigue state as this time period driver, and carry out the design of sorter and the checking of fatigue detecting arithmetic accuracy as benchmark.
Preferably, described step (2) specifically may further comprise the steps:
(21) fatigue characteristic adopting parameters: choose eyes as the information source of tired identification, for driver's closure time that eyes show after entering fatigue state increase, the speed of closing one's eyes slows down, degradation feature under the eyeball activity, chosen 14 indexs relevant with the iris motion with eyelid movement the performance characteristic of eyes under the fatigue state be described;
(22) significance test of discriminant parameter difference under the different fatigue level: the video sample of choosing some from the facial video database of driver is used for the efficiency analysis of tired discriminant criterion, adopt the method for statistical analysis, (clear-headed at three varying levels of tired reason factor to tired discriminant criterion, fatigue, serious tired) on whether exist significant difference to test;
(23) discriminant criterion that finally is identified for the driver fatigue state-detection is eyes closed number percent (Percentage of eyelid Closure, PERCLOS), the longest wink time (Maximum Close Duration, MCD), frequency of wink (Blink Frequency, BF), the degree of on average opening eyes (Average Opening Level, AOL), time window length (Time Window Length correspon ding to a certain value of Closure that the specific time of closing one's eyes is corresponding, TWLCLOS), (Average Opening Time on average opens eyes the time, AOT), maximum (the Maximum Opening Time that opens eyes the time, MOT), (Average Closing Time on average closes one's eyes the time, ACT), maximum (the Maximum Closing Time that closes one's eyes the time, MCT), close one's eyes the time with open eyes maximal value (the Maximum Ratio of Closing and Opening Time of time ratio, MRCOT), close one's eyes the time with open eyes mean value (the Average Ratio of Closing and Opening Time of time ratio, ARCOT), maximum dwell (the Maximum Stationary Time ofIris of iris, MSTI), pupil is without rest index (Pupillary Unrest Index, PUI) and iris asymmetry (Asymmetry of Iris up and down, AI), amount to 14.
Preferably, described step (3) specifically may further comprise the steps:
(31) choosing of sample: from the facial video database of driver, randomly draw great amount of samples, guarantee that as far as possible each driver's video is pumped to, and keep quantity balance clear-headed, tired, serious tired sample;
(32) tired discriminant criterion efficiency analysis: the multi-C vector that consists of from two angle analysis discriminant criterions of quantitative and qualitative analysis respectively is to the classification capacity of driver fatigue state, on the one hand by the two-dimentional allusion quotation classification capacity of the distributive observation discriminant criterion of sample point in the function space then, on the other hand, investigate characteristic parameter to the separating capacity of different fatigue degree according to the nicety of grading of Fisher linear classifier on the training sample space;
(33) generic classifier design: the eyes motion characteristic that will describe the driver fatigue state amounts to 14 the whole introduced features of index spaces, according to the distributed structure Fisher linear classifier of unique point in the feature space; In driving task initial stage (front 20 minutes), use the Fisher linear classifier to carry out the identification of tired pattern.
Preferably, described step (4) specifically may further comprise the steps:
(41) off-line of time varying characteristic obtains and significance test: according to the great amount of samples that extracts in the facial video database of driver, calculate respectively the variation multiple (time varying characteristic) of each sample characteristics parameter, the conspicuousness of check time varying characteristic (variation multiple) difference under the different fatigue level filters out the significant indexes that can differentiate degree of fatigue;
(42) the tired pattern classifier based on time varying characteristic designs: on the basis to the tired classification capacity discriminatory analysis of time varying characteristic, with the time varying characteristic introduced feature space that filters out, the off-line design is based on the tired pattern classifier (component classifier) of time varying characteristic;
(43) real-time online of time varying characteristic obtains: in driving task initial stage (front 20 minutes), if the driver does not have to occur serious tired feature, the eigenwert of then calculating respectively under each driver's waking state is organized in contrast, asks for time varying characteristic (variation multiple) according to the characteristic ginseng value of online Real-time Obtaining;
(44) in the driving task later stage, in the time varying characteristic value substitution component classifier with Real-time Obtaining, carry out the identification of driver fatigue state according to acquired results.
Advantage of the present invention is:
1. the driver fatigue condition detection method emphasis that the present invention is based on time varying characteristic solves driver's individual difference to the impact of accuracy of detection, in actual driving process, organize in contrast by the discriminant criterion that obtains under driver's waking state, adopt the index relevant with individual " eigenwert variation ", the for example variation during the relative waking state of frequency of wink, the identification of driver fatigue state is carried out in variation when opening/closing one's eyes speed with respect to waking state etc., for the raising of tired identification precision and system reliability provides a kind of new Research Thinking.
2. the present invention provides relevant gordian technique solution for the fatigue detecting of driver under the real vehicle running environment, be of value to the practicalization that advances fatigue detecting system, it applies the safety that will ensure better driver, occupant and vehicle-mounted cargo, establishment driver's abnormality and bad behavior can significantly reduce the incidence of the pernicious traffic hazard of China.
Description of drawings
The invention will be further described below in conjunction with drawings and Examples:
Fig. 1 is the process flow diagram of the inventive method.
Embodiment
Embodiment: the present invention utilizes the facial video image of camera acquisition driver, adopt the generic classifier that makes up based on eyes motion characteristic parameter (such as frequency of wink, blink speed etc.) that fatigue state is inferred at the driving task initial stage, stage is to adopt the tired pattern classifier realization of time varying characteristic (such as the variation of frequency of wink, the variation of blink speed etc.) structure to the identification of driver fatigue state on the basis of self study; As shown in Figure 1, specifically comprise the steps:
(1) video image of collection driver face, employing is set up the facial video database of driver based on expert's methods of marking of facial video, it is 30 seconds video segment that each sample in the database is length, and each sample is endowed a fatigue state label; Specifically may further comprise the steps:
(11) utilize the facial video image of camera acquisition driver, adopting video slicing software to be cut in order length the facial video of the driver who has recorded in the experiment is 30 seconds video segment, and each video segment is called a sample;
(12) upset the order of video segment, utilize expert's methods of marking of facial video that the degree of fatigue of driver in each sample is marked, during evaluation driver's state is divided into clear-headed (0 minute), tired (1 minute) and serious tired (2 minutes) three grades; Expert's methods of marking based on facial video is according to certain rule (or whole sensation) off-line its degree of fatigue to be marked according to driver's facial video by one group of trained expert, and the scoring criterion that adopts is:
Clear-headed---eyes are normally opened, and nictation, the eyeball state was active, concentrates, and keeps to external world notice rapidly, and head is rectified and followed side direction once in a while to cast a glance at, normal facial expression;
Tired---closed trend appears in eyes, and the eyeball active degree descends, and is One's eyesight is restrained, yawn, unexpected improper action, subconsciousness is nodded, or occurs winking, shaking the head etc. and resist tired action, correcting faulty sitting posture frequently;
Serious tired---eyes closed trend is serious, the long period occurs to continue to close one's eyes, and the degree of opening eyes descends, and it is crooked head to occur, nods, and it is wooden express one's feelings, health often occurs in a period of time and loses the ability of driving that continues without any movement response;
(13) several scoring experts are to the scoring average of the same video segment actual fatigue state as this time period driver, and carry out the design of sorter and the checking of fatigue detecting arithmetic accuracy as benchmark.
(2) the utilization process software calculates each the fatigue evaluation parameter in each sample, adopt the method check of statistical analysis (clear-headed in the different fatigue level, fatigue, serious tired) descend the conspicuousness of discriminant parameter difference, and then filter out the eye feature index of describing the driver fatigue state; Specifically may further comprise the steps:
(21) fatigue characteristic adopting parameters: choose eyes as the information source of tired identification, for driver's closure time that eyes show after entering fatigue state increase, the speed of closing one's eyes slows down, degradation feature under the eyeball activity, chosen 14 parameters relevant with iris motion (being the activity of eyeball) with eyelid movement the performance characteristic of eyes under the fatigue state be described;
(22) significance test of discriminant parameter difference under the different fatigue level: the video sample of choosing some from the facial video database of driver is used for the efficiency analysis of tired discriminant criterion, adopt the method for statistical analysis, (clear-headed at three varying levels of tired reason factor to tired discriminant criterion by the difference between sample average under the comparison different fatigue state, fatigue, serious tired) on whether exist significant difference to test;
(23) discriminant criterion that finally is identified for the driver fatigue state-detection is eyes closed number percent (Percentage of eyelid Closure, PERCLOS), the longest wink time (Maximum Close Duration, MCD), frequency of wink (Blink Frequency, BF), the degree of on average opening eyes (Average Opening Level, AOL), time window length (Time Window Length corresponding to a certain value of Closure that the specific time of closing one's eyes is corresponding, TWLCLOS), (Average Opening Time on average opens eyes the time, AOT), maximum (the Maximum Opening Time that opens eyes the time, MOT), (Average Closing Time on average closes one's eyes the time, ACT), maximum (the Maximum Closing Time that closes one's eyes the time, MC T), close one's eyes the time with open eyes maximal value (the Maximum Ratio of Closing and Opening Time of time ratio, MRCOT), close one's eyes the time with open eyes mean value (the Average Ratio of Closing and Opening Time of time ratio, ARCOT), maximum dwell (the Maximum Stationary Time of Iris of iris, MSTI), pupil is without rest index (Pupillary Unrest Index, PUI) and iris asymmetry (Asymmetry of Iris up and down, AI), amount to 14; These parameters comprise two classes, and a class is the parameter relevant with eyelid movement, and another kind of is the parameter relevant with the iris motion, and the definition of each index is described below respectively: PERCLOS, and in a period of time, eyes accumulative total closure time accounts for the number percent of time window length; MCD, in a period of time, the duration of the longest eyes closed; BF, the per minute number of winks; AOL, in a period of time, eyes are opened the mean value of degree (number percent); TWLCLOS, accumulative total reaches specific needed T.T. of the time of closing one's eyes; AOT, in a period of time, the mean value of the action required time of opening eyes; MOT, in a period of time, the maximal value of the action required time of opening eyes; ACT, in a period of time, the mean value of the action required time of closing one's eyes; MCT, in a period of time, the maximal value of the action required time of closing one's eyes; MRCOT, close one's eyes time and the maximal value of time ratio within a period of time of opening eyes; ARCOT, the mean value of ratio within a period of time of close one's eyes time and the time of opening eyes; MSTI, in a period of time, iris is put the maximal value of nonmotile time with respect to the canthus; PUI, in a period of time, the pupil dwell accounts for the number percent of T.T.; AI, the iris center is to the distance and the ratio of iris center to the distance of palpebra inferior in upper eyelid.
The whole introduced features of the eyes motion characteristic parameter space of the description driver fatigue state that (3) will filter out utilizes the method for statistics discriminatory analysis to make up generic classifier; At the driving task initial stage, adopt this universal model realization to the on-line identification of driver fatigue state; Specifically may further comprise the steps:
(31) choosing of sample: extract great amount of samples from the facial video database of driver, guarantee that as far as possible each driver's video is chosen, and keep quantity balance clear-headed, tired, serious tired sample, it is too large that quantity should not differ;
(32) tired discriminant criterion efficiency analysis: for investigating multi-C vector that discriminant criterion consists of to the classification capacity of driver fatigue state, by the then distribution of sample point in the function space of two-dimentional allusion quotation, from qualitative angle the classification capacity of index is observed on the one hand; According to the nicety of grading of Fisher linear classifier on the training sample space, the quantitative expedition characteristic parameter is to the separating capacity of different fatigue degree on the other hand;
(33) generic classifier design: the eyes motion characteristic that will describe the driver fatigue state amounts to 14 the whole introduced features of index spaces, according to the distributed structure Fisher linear classifier of unique point in the feature space; In driving task initial stage (front 20 minutes), use the Fisher linear classifier to carry out the identification of tired pattern;
(4) randomly drawing sample from facial video database, according to the discriminant parameter value under each driver's waking state, obtain the variation multiple (time varying characteristic) of each sample characteristics parameter, and the conspicuousness of check time varying characteristic difference under the different fatigue level, filter out the time varying characteristic index of describing the driver fatigue state; On the basis to the discriminatory analysis of time varying characteristic target validity, the off-line design is based on the tired pattern classifier of time varying characteristic; In the actual driving process, algorithm can carry out to the eyes motion characteristic under driver's waking state on-line study and organize in contrast, in the driving task later stage, utilizes this tired pattern classifier based on time varying characteristic that fatigue state is carried out identification.
(41) off-line of time varying characteristic obtains and significance test: according to the great amount of samples that extracts in the facial video database of driver, calculate respectively the eyes motion characteristic PERCLOS under each driver's waking state, the average of 14 parameters such as MCD, AOL, be designated as:
R
i(Parameter)
Wherein i represents i tested personnel, and Parameter is one of 14 parameters such as PERCLOS, MCD; For each sample with Fatigued level label, the value of difference calculated characteristics parameter is denoted as:
T
ij(Parameter)
Wherein i still represents i tested personnel, and j represents j sample; The mean value of each parameter was done merchant's computing when the calculated value of each each characteristic parameter of sample and this driver is clear-headed, and the variation multiple of the characteristic ginseng value that can obtain each sample reference value during with respect to this driver's waking state is shown below:
C
ij＝T
ij(Parameter)/R
i(Parameter)
Adopt and change the time varying characteristic that multiple is described each discriminant criterion of driver fatigue state, check is (clear-headed in the different fatigue level, fatigue, serious tired) descend the conspicuousness of time varying characteristic difference, and filter out the time varying characteristic index of describing the driver fatigue state;
(42) the tired pattern classifier based on time varying characteristic designs: on the basis to the tired discriminating power efficiency analysis of time varying characteristic, according to the value of each training sample time varying characteristic and the class label of sample (clear-headed, tired or serious tired), the off-line design obtains discriminant equation based on the tired pattern classifier of time varying characteristic;
(43) real-time online of time varying characteristic obtains: serious tired feature do not occur if find the driver in driving task initial stage (front 20 minutes) the generic classifier Output rusults, think that namely the driver should regain consciousness in the time period, algorithm will calculate the average of eyes motion characteristic parameter under the waking state automatically, and ask for time varying characteristic (variation multiple) according to the characteristic ginseng value of online Real-time Obtaining;
(44) in the driving task later stage, with the time varying characteristic substitution discriminant equation of Real-time Obtaining, use component classifier to carry out the real-time estimation of driver fatigue state.
Below only be concrete exemplary applications of the present invention, protection scope of the present invention is not constituted any limitation.In addition to the implementation, the present invention can also have other embodiment.All employings are equal to the technical scheme of replacement or equivalent transformation formation, all drop within the present invention's scope required for protection.
Claims (5)
1. fatigue state detection method of utilizing on-line study to eliminate driver's individual difference, it is characterized in that: utilize the facial video image of camera acquisition driver, adopt the generic classifier that makes up based on eyes motion characteristic parameter (such as frequency of wink, blink speed etc.) that fatigue state is inferred at the driving task initial stage, stage is to adopt the tired pattern classifier realization of " time varying characteristic " (such as the variation of frequency of wink, variation of blink speed etc.) structure to the identification of driver fatigue state on the basis of self study; Specifically comprise the steps:
(1) gather the facial video image of driver, adopt based on expert's methods of marking of facial video and set up the facial video database of driver, it is 30 seconds video segment that each sample in the database is with Fatigued level label and length;
(2) use process software to test the action of driver's eyelid in each sample and the motion state of eyeball, calculate each fatigue evaluation parameter of describing the eye motion feature, adopt the conspicuousness of method check discriminant parameter difference under the different fatigue level of statistical analysis, and then filter out the eye feature index of describing the driver fatigue state;
(3) with the whole introduced features of the driver's eyes motion characteristic parameter space that filters out, from video database, randomly draw great amount of samples and calculate the characteristic ginseng value of each sample, make up generic classifier according to the class label (clear-headed, tired or serious tired) of characteristic ginseng value and sample; At the driving task initial stage, adopt this universal model that driver's fatigue state is detected in real time;
(4) according to the great amount of samples that in the facial video database of driver, extracts, calculate respectively the variation multiple (time varying characteristic) of each sample characteristics parameter, the conspicuousness of check variation characteristic difference under the different fatigue level, and then filter out the time varying characteristic index that can differentiate the driver fatigue state; According to the value of time varying characteristic and the class label of sample, and to the time become on the basis of target validity discriminatory analysis, the off-line design is based on the tired pattern classifier of time varying characteristic; In the actual driving process, algorithm can carry out to the eyes motion characteristic under driver's waking state on-line study and organize in contrast, in the driving task later stage, by relatively the data of Real-time Obtaining and the diversity factor of control group are carried out identification to fatigue state.
2. the fatigue state detection method of utilizing on-line study to eliminate driver's individual difference according to claim 1, it is characterized in that: described step (1) specifically may further comprise the steps:
(11) utilize camera acquisition driver's facial video image, it is 30 seconds video segment that the facial video of the driver who has recorded in the experiment is cut into length according to time sequencing, and each video segment is called a sample;
(12) utilization is marked to the degree of fatigue of driver in each sample based on expert's methods of marking off-line of facial video, wherein regains consciousness and counts 0 minute, and fatigue is counted 1 minute, and serious fatigue is counted 2 minutes; The scoring criterion that adopts is:
Clear-headed---eyes are normally opened, and nictation, the eyeball state was active, concentrates, and keeps to external world notice rapidly, and head is rectified and followed side direction once in a while to cast a glance at, normal facial expression;
Tired---closed trend appears in eyes, and the eyeball active degree descends, and is One's eyesight is restrained, yawn, unexpected improper action, subconsciousness is nodded, or occurs winking, shaking the head etc. and resist tired action, correcting faulty sitting posture frequently;
Serious tired---eyes closed trend is serious, the long period occurs to continue to close one's eyes, and the degree of opening eyes descends, and it is crooked head to occur, nods, and it is wooden express one's feelings, health often occurs in a period of time and loses the ability of driving that continues without any movement response;
(13) several scoring experts are to the scoring average of the same video segment actual fatigue state as this time period driver, and carry out the design of sorter and the checking of fatigue detecting arithmetic accuracy as benchmark.
3. the fatigue state detection method of utilizing on-line study to eliminate driver's individual difference according to claim 1, it is characterized in that: described step (2) specifically may further comprise the steps:
(21) fatigue characteristic adopting parameters: choose eyes as the information source of tired identification, for driver's closure time that eyes show after entering fatigue state increase, the speed of closing one's eyes slows down, degradation feature under the eyeball activity, chosen 14 indexs relevant with the iris motion with eyelid movement the performance characteristic of eyes under the fatigue state be described;
(22) significance test of discriminant parameter difference under the different fatigue level: the video sample of choosing some from the facial video database of driver is used for the efficiency analysis of tired discriminant criterion, adopt the method for statistical analysis, (clear-headed at three varying levels of tired reason factor to tired discriminant criterion, fatigue, serious tired) on whether exist significant difference to test;
(23) discriminant criterion that finally is identified for the driver fatigue state-detection is eyes closed number percent (Percentage of eyelid Closure, PERCLOS), the longest wink time (Maximum Close Duration, MCD), frequency of wink (Blink Frequency, BF), the degree of on average opening eyes (Average Opening Level, AOL), time window length (Time Window Length corresponding to a certain value of Closure that the specific time of closing one's eyes is corresponding, TWLCLOS), (Average Opening Time on average opens eyes the time, AOT), maximum (the Maximum Opening Time that opens eyes the time, MOT), (Average Closing Time on average closes one's eyes the time, ACT), maximum (the Maximum Closing Time that closes one's eyes the time, MCT), close one's eyes the time with open eyes maximal value (the Maximum Ratio of Closing and Opening Time of time ratio, MRCOT), close one's eyes the time with open eyes mean value (the Average Ratio of Closing and Opening Time of time ratio, ARCOT), maximum dwell (the Maximum Stationary Time of Iris of iris, MSTI), pupil is without rest index (Pupillary Unrest Index, PUI) and iris asymmetry (Asymmetry of Iris up and down, AI), amount to 14.
4. the fatigue state detection method of utilizing on-line study to eliminate driver's individual difference according to claim 1, it is characterized in that: described step (3) specifically may further comprise the steps:
(31) choosing of sample: from the facial video database of driver, randomly draw great amount of samples, guarantee that as far as possible each driver's video is pumped to, and keep quantity balance clear-headed, tired, serious tired sample;
(32) tired discriminant criterion efficiency analysis: the multi-C vector that consists of from two angle analysis discriminant criterions of quantitative and qualitative analysis respectively is to the classification capacity of driver fatigue state, on the one hand by the two-dimentional allusion quotation classification capacity of the distributive observation discriminant criterion of sample point in the function space then, on the other hand, investigate characteristic parameter to the separating capacity of different fatigue degree according to the nicety of grading of Fisher linear classifier on the training sample space;
(33) generic classifier design: the eyes motion characteristic that will describe the driver fatigue state amounts to 14 the whole introduced features of index spaces, according to the distributed structure Fisher linear classifier of unique point in the feature space; In driving task initial stage (front 20 minutes), use the Fisher linear classifier to carry out the identification of tired pattern.
5. the fatigue state detection method of utilizing on-line study to eliminate driver's individual difference according to claim 1, it is characterized in that: described step (4) specifically may further comprise the steps:
(41) off-line of time varying characteristic obtains and significance test: according to the great amount of samples that extracts in the facial video database of driver, calculate respectively the variation multiple (time varying characteristic) of each sample characteristics parameter, the conspicuousness of check time varying characteristic (variation multiple) difference under the different fatigue level filters out the significant indexes that can differentiate degree of fatigue;
(42) the tired pattern classifier based on time varying characteristic designs: on the basis to the tired classification capacity discriminatory analysis of time varying characteristic, with the time varying characteristic introduced feature space that filters out, the off-line design is based on the tired pattern classifier (component classifier) of time varying characteristic;
(43) real-time online of time varying characteristic obtains: in driving task initial stage (front 20 minutes), if the driver does not have to occur serious tired feature, the eigenwert of then calculating respectively under each driver's waking state is organized in contrast, asks for time varying characteristic (variation multiple) according to the characteristic ginseng value of online Real-time Obtaining;
(44) in the driving task later stage, in the time varying characteristic value substitution component classifier with Real-time Obtaining, carry out the identification of driver fatigue state according to acquired results.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210505976.5A CN103020594B (en) | 2012-12-03 | 2012-12-03 | On-line study is utilized to eliminate the fatigue state detection method of driver individual difference |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210505976.5A CN103020594B (en) | 2012-12-03 | 2012-12-03 | On-line study is utilized to eliminate the fatigue state detection method of driver individual difference |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103020594A true CN103020594A (en) | 2013-04-03 |
| CN103020594B CN103020594B (en) | 2016-01-20 |
Family
ID=47969185
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210505976.5A Active CN103020594B (en) | 2012-12-03 | 2012-12-03 | On-line study is utilized to eliminate the fatigue state detection method of driver individual difference |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103020594B (en) |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103617421A (en) * | 2013-12-17 | 2014-03-05 | 上海电机学院 | Fatigue detecting method and system based on comprehensive video feature analysis |
| CN103950386A (en) * | 2014-04-07 | 2014-07-30 | 临颍县贝克电子科技有限公司 | System for comprehensively monitoring by using state features of automobile driver |
| CN104484549A (en) * | 2014-11-06 | 2015-04-01 | 山东交通学院 | Multi-task driving safety state identification method based on visual attention mechanism of driver |
| CN104794438A (en) * | 2015-04-08 | 2015-07-22 | 天脉聚源（北京）教育科技有限公司 | Person state monitoring method and person state monitoring device |
| CN104850827A (en) * | 2015-04-23 | 2015-08-19 | 小米科技有限责任公司 | Fingerprint identification method and apparatus |
| CN105354988A (en) * | 2015-12-11 | 2016-02-24 | 东北大学 | Driver fatigue driving detection system based on machine vision and detection method |
| CN105632104A (en) * | 2016-03-18 | 2016-06-01 | 内蒙古大学 | Fatigue driving detection system and method |
| CN105809893A (en) * | 2014-12-30 | 2016-07-27 | 北京奇虎科技有限公司 | System and method for preventing fatigue driving |
| CN105931430A (en) * | 2016-05-25 | 2016-09-07 | 惠州华阳通用电子有限公司 | Alarm sensitivity detection method and apparatus for driver state early warning system |
| CN106355838A (en) * | 2016-10-28 | 2017-01-25 | 深圳市美通视讯科技有限公司 | Fatigue driving detection method and system |
| CN106407922A (en) * | 2016-09-08 | 2017-02-15 | 哈尔滨工程大学 | Online dictionary learning deformation model-based fatigue state recognition method |
| WO2017040519A1 (en) * | 2015-08-31 | 2017-03-09 | Sri International | Method and system for monitoring driving behaviors |
| CN104182338B (en) * | 2014-08-19 | 2017-05-24 | 苏州清研微视电子科技有限公司 | Fatigue driving early warning product detection accuracy test method |
| CN107480695A (en) * | 2015-02-06 | 2017-12-15 | 安徽大学 | A kind of contour of object detection recognition method |
| CN107844777A (en) * | 2017-11-16 | 2018-03-27 | 百度在线网络技术（北京）有限公司 | Method and apparatus for generating information |
| CN108108684A (en) * | 2017-12-15 | 2018-06-01 | 杭州电子科技大学 | A kind of attention detection method for merging line-of-sight detection |
| CN109192275A (en) * | 2018-08-06 | 2019-01-11 | 百度在线网络技术（北京）有限公司 | The determination method, apparatus and server of personage's state of mind |
| CN109543577A (en) * | 2018-11-09 | 2019-03-29 | 上海物联网有限公司 | A kind of fatigue driving detection method for early warning based on facial expression feature |
| CN109925678A (en) * | 2019-03-01 | 2019-06-25 | 北京七鑫易维信息技术有限公司 | A kind of training method based on eye movement tracer technique, training device and equipment |
| CN109977761A (en) * | 2019-02-01 | 2019-07-05 | 重庆交互科技有限公司 | Fatigue driving awakening method and system based on facial state recognition |
| CN110182050A (en) * | 2019-05-04 | 2019-08-30 | 王飞 | Fatigue driving intelligent controls speed limit parking device |
| CN110232327A (en) * | 2019-05-21 | 2019-09-13 | 浙江师范大学 | A kind of driving fatigue detection method based on trapezoidal concatenated convolutional neural network |
| CN110765807A (en) * | 2018-07-25 | 2020-02-07 | 阿里巴巴集团控股有限公司 | Driving behavior analysis method, driving behavior processing method, driving behavior analysis device, driving behavior processing device and storage medium |
| CN111295699A (en) * | 2017-08-31 | 2020-06-16 | 松下知识产权经营株式会社 | Assistance method, assistance system using the assistance method, and assistance device |
| CN111784973A (en) * | 2020-07-30 | 2020-10-16 | 广州敏视数码科技有限公司 | MDVR equipment integration fatigue detection method of fleet management platform |
| CN112906633A (en) * | 2021-03-18 | 2021-06-04 | 南通师范高等专科学校 | Teaching optimization method based on student attention |
| CN113967014A (en) * | 2021-12-22 | 2022-01-25 | 成都航空职业技术学院 | Student behavior analysis device, system and method based on big data |
| WO2022110917A1 (en) * | 2020-11-26 | 2022-06-02 | 亿咖通（湖北）科技有限公司 | Method for determining driving state of driver, computer storage medium, and electronic device |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101540090A (en) * | 2009-04-14 | 2009-09-23 | 华南理工大学 | Driver fatigue monitoring device based on multivariate information fusion and monitoring method thereof |
-
2012
- 2012-12-03 CN CN201210505976.5A patent/CN103020594B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101540090A (en) * | 2009-04-14 | 2009-09-23 | 华南理工大学 | Driver fatigue monitoring device based on multivariate information fusion and monitoring method thereof |
Non-Patent Citations (4)
| Title |
|---|
| 张希波等: "基于方向盘操作的驾驶人疲劳状态实时检测方法", 《清华大学学报（自然科学版）》 * |
| 成波等: "基于多源信息融合的驾驶人疲劳状态监测及预警方法研究", 《公路交通科技》 * |
| 李衡峰: "基于综合集成的驾驶疲劳识别", 《中国优秀硕博士学位论文全文数据库（硕士）信息科技辑》 * |
| 马添翼等: "基于面部表情特征的驾驶员疲劳状态识别", 《汽车安全与节能学报》 * |
Cited By (38)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103617421A (en) * | 2013-12-17 | 2014-03-05 | 上海电机学院 | Fatigue detecting method and system based on comprehensive video feature analysis |
| CN103950386A (en) * | 2014-04-07 | 2014-07-30 | 临颍县贝克电子科技有限公司 | System for comprehensively monitoring by using state features of automobile driver |
| CN104182338B (en) * | 2014-08-19 | 2017-05-24 | 苏州清研微视电子科技有限公司 | Fatigue driving early warning product detection accuracy test method |
| CN104484549A (en) * | 2014-11-06 | 2015-04-01 | 山东交通学院 | Multi-task driving safety state identification method based on visual attention mechanism of driver |
| CN105809893B (en) * | 2014-12-30 | 2019-07-30 | 北京奇虎科技有限公司 | A kind of anti-fatigue-driving system and method |
| CN105809893A (en) * | 2014-12-30 | 2016-07-27 | 北京奇虎科技有限公司 | System and method for preventing fatigue driving |
| CN107480695A (en) * | 2015-02-06 | 2017-12-15 | 安徽大学 | A kind of contour of object detection recognition method |
| CN104794438A (en) * | 2015-04-08 | 2015-07-22 | 天脉聚源（北京）教育科技有限公司 | Person state monitoring method and person state monitoring device |
| CN104850827A (en) * | 2015-04-23 | 2015-08-19 | 小米科技有限责任公司 | Fingerprint identification method and apparatus |
| WO2017040519A1 (en) * | 2015-08-31 | 2017-03-09 | Sri International | Method and system for monitoring driving behaviors |
| US10769459B2 (en) | 2015-08-31 | 2020-09-08 | Sri International | Method and system for monitoring driving behaviors |
| CN105354988A (en) * | 2015-12-11 | 2016-02-24 | 东北大学 | Driver fatigue driving detection system based on machine vision and detection method |
| CN105354988B (en) * | 2015-12-11 | 2018-02-27 | 东北大学 | A kind of driver tired driving detecting system and detection method based on machine vision |
| CN105632104A (en) * | 2016-03-18 | 2016-06-01 | 内蒙古大学 | Fatigue driving detection system and method |
| CN105632104B (en) * | 2016-03-18 | 2019-03-01 | 内蒙古大学 | A kind of fatigue driving detecting system and method |
| CN105931430A (en) * | 2016-05-25 | 2016-09-07 | 惠州华阳通用电子有限公司 | Alarm sensitivity detection method and apparatus for driver state early warning system |
| CN106407922A (en) * | 2016-09-08 | 2017-02-15 | 哈尔滨工程大学 | Online dictionary learning deformation model-based fatigue state recognition method |
| CN106355838A (en) * | 2016-10-28 | 2017-01-25 | 深圳市美通视讯科技有限公司 | Fatigue driving detection method and system |
| CN111295699B (en) * | 2017-08-31 | 2023-03-21 | 松下知识产权经营株式会社 | Assistance method, assistance system using the assistance method, and assistance device |
| CN111295699A (en) * | 2017-08-31 | 2020-06-16 | 松下知识产权经营株式会社 | Assistance method, assistance system using the assistance method, and assistance device |
| CN107844777B (en) * | 2017-11-16 | 2021-06-11 | 百度在线网络技术（北京）有限公司 | Method and apparatus for generating information |
| CN107844777A (en) * | 2017-11-16 | 2018-03-27 | 百度在线网络技术（北京）有限公司 | Method and apparatus for generating information |
| CN108108684B (en) * | 2017-12-15 | 2020-07-17 | 杭州电子科技大学 | Attention detection method integrating sight detection |
| CN108108684A (en) * | 2017-12-15 | 2018-06-01 | 杭州电子科技大学 | A kind of attention detection method for merging line-of-sight detection |
| CN110765807B (en) * | 2018-07-25 | 2024-04-05 | 斑马智行网络(香港)有限公司 | Driving behavior analysis and processing method, device, equipment and storage medium |
| CN110765807A (en) * | 2018-07-25 | 2020-02-07 | 阿里巴巴集团控股有限公司 | Driving behavior analysis method, driving behavior processing method, driving behavior analysis device, driving behavior processing device and storage medium |
| CN109192275A (en) * | 2018-08-06 | 2019-01-11 | 百度在线网络技术（北京）有限公司 | The determination method, apparatus and server of personage's state of mind |
| US11055512B2 (en) | 2018-08-06 | 2021-07-06 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method, apparatus and server for determining mental state of human |
| CN109543577A (en) * | 2018-11-09 | 2019-03-29 | 上海物联网有限公司 | A kind of fatigue driving detection method for early warning based on facial expression feature |
| CN109977761A (en) * | 2019-02-01 | 2019-07-05 | 重庆交互科技有限公司 | Fatigue driving awakening method and system based on facial state recognition |
| CN109925678A (en) * | 2019-03-01 | 2019-06-25 | 北京七鑫易维信息技术有限公司 | A kind of training method based on eye movement tracer technique, training device and equipment |
| CN110182050A (en) * | 2019-05-04 | 2019-08-30 | 王飞 | Fatigue driving intelligent controls speed limit parking device |
| CN110232327A (en) * | 2019-05-21 | 2019-09-13 | 浙江师范大学 | A kind of driving fatigue detection method based on trapezoidal concatenated convolutional neural network |
| CN111784973B (en) * | 2020-07-30 | 2021-12-14 | 广州敏视数码科技有限公司 | MDVR equipment integration fatigue detection method of fleet management platform |
| CN111784973A (en) * | 2020-07-30 | 2020-10-16 | 广州敏视数码科技有限公司 | MDVR equipment integration fatigue detection method of fleet management platform |
| WO2022110917A1 (en) * | 2020-11-26 | 2022-06-02 | 亿咖通（湖北）科技有限公司 | Method for determining driving state of driver, computer storage medium, and electronic device |
| CN112906633A (en) * | 2021-03-18 | 2021-06-04 | 南通师范高等专科学校 | Teaching optimization method based on student attention |
| CN113967014A (en) * | 2021-12-22 | 2022-01-25 | 成都航空职业技术学院 | Student behavior analysis device, system and method based on big data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103020594B (en) | 2016-01-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103020594B (en) | On-line study is utilized to eliminate the fatigue state detection method of driver individual difference | |
| Chirra et al. | Deep CNN: A Machine Learning Approach for Driver Drowsiness Detection Based on Eye State. | |
| Zhang et al. | Driver fatigue detection based on eye state recognition | |
| Tipprasert et al. | A method of driver’s eyes closure and yawning detection for drowsiness analysis by infrared camera | |
| Garcia et al. | Vision-based drowsiness detector for real driving conditions | |
| CN104183091A (en) | System for adjusting sensitivity of fatigue driving early warning system in self-adaptive mode | |
| EP3248179B1 (en) | Method and apparatus for recognising microsleep in a driver of a vehicle | |
| CN106250801A (en) | Based on Face datection and the fatigue detection method of human eye state identification | |
| CN108294759A (en) | A kind of Driver Fatigue Detection based on CNN Eye state recognitions | |
| CN104574817A (en) | Machine vision-based fatigue driving pre-warning system suitable for smart phone | |
| CN103340637A (en) | System and method for driver alertness intelligent monitoring based on fusion of eye movement and brain waves | |
| CN109229108A (en) | A kind of driving behavior safe evaluation method based on driving fingerprint | |
| CN109559481A (en) | Drive risk intelligent identification Method, device, computer equipment and storage medium | |
| CN106548132A (en) | The method for detecting fatigue driving of fusion eye state and heart rate detection | |
| CN112528843A (en) | Motor vehicle driver fatigue detection method fusing facial features | |
| CN103729646A (en) | Eye image validity detection method | |
| CN108446673A (en) | A kind of controller's giving fatigue pre-warning method based on face recognition | |
| Picot et al. | EOG-based drowsiness detection: Comparison between a fuzzy system and two supervised learning classifiers | |
| Rani et al. | Development of an Automated Tool for Driver Drowsiness Detection | |
| Bin et al. | A fatigue driving detection method based on multi facial features fusion | |
| Feldhütter et al. | A new approach for a real-time non-invasive fatigue assessment system for automated driving | |
| Wang et al. | A fatigue driving detection method based on deep learning and image processing | |
| Bajaj et al. | AI based novel approach to detect driver drowsiness | |
| Joseph et al. | Real time drowsiness detection using Viola jones & KLT | |
| Shaykha et al. | FEER: Non-intrusive facial expression and emotional recognition for driver's vigilance monitoring |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |