CN101872171B - Driver fatigue state recognition method and system based on information fusion - Google Patents

Driver fatigue state recognition method and system based on information fusion Download PDF

Info

Publication number
CN101872171B
CN101872171B CN2009100825662A CN200910082566A CN101872171B CN 101872171 B CN101872171 B CN 101872171B CN 2009100825662 A CN2009100825662 A CN 2009100825662A CN 200910082566 A CN200910082566 A CN 200910082566A CN 101872171 B CN101872171 B CN 101872171B
Authority
CN
China
Prior art keywords
confidence level
tired
parameter
driver
bearing circle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN2009100825662A
Other languages
Chinese (zh)
Other versions
CN101872171A (en
Inventor
宋正河
朱忠祥
谢斌
毛恩荣
张俊
成波
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Agricultural University
Original Assignee
China Agricultural University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Agricultural University filed Critical China Agricultural University
Priority to CN2009100825662A priority Critical patent/CN101872171B/en
Publication of CN101872171A publication Critical patent/CN101872171A/en
Application granted granted Critical
Publication of CN101872171B publication Critical patent/CN101872171B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The invention discloses driver fatigue state recognition method and system based on information fusion, wherein the method comprises the steps of: acquiring eye state information of a driver, operating state information of a steering wheel and track driving state information of a vehicle; processing the eye state information of the driver, the operating state information of the steering wheel and the track driving state information of the vehicle to obtain fatigue expression information of the driver; and carrying out weighting fusion processing on the fatigue expression information to obtain a fatigue membership degree corresponding to a drive fatigue grade, and obtaining the fatigue degree of the driver according to the fatigue membership degree. The invention improves the reliability and the accuracy of monitoring the driver fatigue state, can reduce the quantity of traffic accidents caused by the fatigue driving of the driver, ensures the road traffic safety, and has strong practicability because the acquisition of multi-source information does not cause the interference on the driving of the driver.

Description

Driver fatigue state recognition method and system based on information fusion
Technical field
The present invention relates to traffic engineering, particularly a kind of driver fatigue state recognition method and system based on information fusion.
Background technology
Rapid increase along with expanding economy and automobile quantity; The incidence of road traffic accident also increases gradually; And driver tired driving is a key factor that causes road traffic accident; The fatigue conditions of therefore, scientificlly and effectively monitoring the driver has great significance to guaranteeing traffic safety.
Monitoring to driver fatigue state in the prior art mainly is to adopt single source information monitoring, for example, judges through monitoring driver physiological situation whether the driver is in fatigue driving state.For a long time, EEG measuring or electrocardio measurement are described as " golden standard " of monitoring driver fatigue state.The ripple of the different frequency range in the brain wave has corresponding variation at the different times of driver fatigue; When tired in the early stage; Slow wave delta, alpha can change, and the belta ripple can increase when mid-term was tired, and delta, theta and alpha ripple have increase by a relatively large margin when extremely tired.The radio-frequency component HF and the low-frequency component LF of the HRV in the electrocardio index can change when fatigue.
But; The method of the single source information monitoring of this employing driver fatigue state remains in following technological deficiency in the prior art: on the one hand; There is the not high defective of accuracy, stability and reliability that information is single, discern in single source information; If should list source information inaccurately will make above-mentioned driver fatigue state recognition method lose efficacy, then be difficult to monitor the driving fatigue situation that obtains the driver, thereby make driver's driving have very big potential safety hazard; On the other hand, single source information measuring equipment (like the equipment of measure cerebral electricity or measure cardiac electrical equipment) can produce bigger interference to driver's driving, is not suitable for the monitoring of actual driver fatigue state, poor practicability.
Summary of the invention
The purpose of this invention is to provide a kind of driver fatigue state recognition method and system, effectively overcome prior art and be not suitable for that practical application, information are single, the accuracy and the not high defective of reliability of identification based on information fusion.
The invention provides a kind of driver fatigue state recognition method, comprising based on information fusion:
Step 1, collection driver's eye status information, bearing circle operation state information and track of vehicle transport condition information;
Step 2, the eye status information to said driver, bearing circle operation state information and track of vehicle transport condition information are handled; Acquisition driver's tired performance information, said tired performance information comprise that on average degree of opening, the longest time of closing one's eyes, bearing circle keep motionless time scale, angle variable quantity, the transversal displacement standard deviation of the snakelike transport condition of vehicle and the time scale that said transversal displacement is in the safe distance scope of the unexpected rotation of bearing circle to interior eyes closed degree of unit interval above 80% shared time scale, eyes;
Step 3, said tired performance information is carried out weighting fusion treatment obtain the tired degree of membership corresponding, and draw driver's tired grade according to said tired degree of membership with the driver fatigue grade.
The invention provides a kind of driver fatigue state recognition system, comprising based on information fusion:
Acquisition module is used to gather driver's eye status information, bearing circle operation state information and track of vehicle transport condition information;
Processing module; The eye status information, bearing circle operation state information and the track of vehicle transport condition information that are used for said driver are handled; Acquisition driver's tired performance information, said tired performance information comprise that on average degree of opening, the longest time of closing one's eyes, bearing circle keep motionless time scale, angle variable quantity, the transversal displacement standard deviation of the snakelike transport condition of vehicle and the time scale that said transversal displacement is in the safe distance scope of the unexpected rotation of bearing circle to interior eyes closed degree of unit interval above 80% shared time scale, eyes;
Fusion Module is used for that said tired performance information is carried out weighting fusion treatment and obtains the tired degree of membership corresponding with the driver fatigue grade, and draws driver's tired grade according to said tired degree of membership.
The invention provides a kind of driver fatigue state recognition method and system based on information fusion; At first gather driver's eye status information, bearing circle operation state information and track of vehicle transport condition information; Afterwards said driver's eye status information, bearing circle operation state information and track of vehicle transport condition information are handled; Obtain driver's tired performance information; At last said tired performance information is carried out weighting fusion treatment and obtain the tired degree of membership corresponding, and draw driver's tired grade according to said tired degree of membership with the driver fatigue grade.Because the present invention integrates the driving fatigue situation of estimating the driver through the multi-source information to reflection driver driving fatigue situation; Therefore the reliability and the accuracy of driver fatigue state identification have been improved; Can effectively reduce the traffic hazard quantity that driver tired driving causes, guarantee traffic safety.Compared with prior art, the driver fatigue state recognition method and the system implementation that the present invention is based on information fusion can not cause interference to driver's driving, have practical, be convenient to advantages such as enforcement, be with a wide range of applications.
Description of drawings
Fig. 1 is the process flow diagram that the present invention is based on driver fatigue state recognition method first embodiment of information fusion;
Fig. 2 is the process flow diagram that the present invention is based on driver fatigue state recognition method second embodiment of information fusion;
Fig. 3 is the structural representation that the present invention is based on driver fatigue state recognition system first embodiment of information fusion;
Fig. 4 is the structural representation that the present invention is based on driver fatigue state recognition system second embodiment of information fusion.
Embodiment
The inventor finds in realizing process of the present invention; Driver's fatigue conditions can be reflected in multiple aspect; Physiological status, manipulative behavior and vehicle-state and driver fatigue state all have certain correlativity; Such as, the eyelid movement in the facial state, head movement, eye sight line and facial expression all obvious variation can occur when fatigue state takes place the driver; Aspect trailer reversing, the lateral excursion of pedaling the power of stepping on and track of vehicle of the grip of the time domain of steering wheel angle and frequency-domain index, bearing circle, angular velocity, the speed of a motor vehicle, pedal etc. all have certain correlativity with driver's fatigue state.Based on this; The external symptom that the present invention grasps the key link when monitoring driver fatigue state; The information of eye state, bearing circle manipulation and track of vehicle three aspects is merged, be applied in the driver fatigue state monitoring, integrate the fatigue conditions of estimating the driver; Can when the supplemental characteristic fiduciary level is lower, reject it; Utilize the higher information of other fiduciary levels, thereby can remedy the deficiency of single source information monitoring, improve the accuracy rate stability of driver fatigue state monitoring.
Through accompanying drawing and embodiment, technical scheme of the present invention is done further detailed description below.
Fig. 1 is the process flow diagram that the present invention is based on driver fatigue state recognition method first embodiment of information fusion, and as shown in Figure 1, present embodiment comprises:
Step 101, collection driver's eye status information, bearing circle operation state information and track of vehicle transport condition information.
Collection driver's in this step eye status information can adopt the multiple IMAQ mode of prior art, for example photography or shooting.Photography is meant the eyes image of whenever gathering a driver at a distance from a setting-up time (as 1 minute), after shooting is meant continuous acquisition driver's eyes image, extracts rest image by a setting-up time (as 1 minute).The bearing circle operation state information of gathering the driver can adopt the multiple sensors mode of prior art, and scrambler or angular transducer for example are set on bearing circle.Collection vehicle track transport condition information can adopt the multiple metering system of prior art, for example adopts the lateral excursion distance of position-measurement device measuring vehicle apart from center line of road.Driver's eye parameter in the present embodiment can be to carry out statistical treatment through the camera collection eyes image to obtain; The bearing circle parameter can be that direction of passage dish scrambler is gathered instantaneous direction dish corner and carried out statistical treatment and obtain; The track of vehicle parameter can be to obtain apart from the process statistical treatment through the lateral excursion of Flame Image Process collection vehicle track apart from center line of road.
Step 102, the eye status information to the driver, bearing circle operation state information and track of vehicle transport condition information are handled, and obtain driver's tired performance information.
The eye status information that collects, bearing circle operation state information and track of vehicle transport condition information are carried out statistical treatment, obtain driver's tired performance information.This fatigue performance information comprises that on average degree of opening, the longest time of closing one's eyes, bearing circle keep motionless time scale, angle variable quantity, the transversal displacement standard deviation of the snakelike transport condition of vehicle and the time scale that said transversal displacement is in the safe distance scope of the unexpected rotation of bearing circle to interior eyes closed degree of unit interval above 80% shared time scale, eyes.
Step 103, tired performance information is carried out weighting fusion treatment obtain the tired degree of membership corresponding, and draw driver's tired grade according to tired degree of membership with the driver fatigue grade.
Present embodiment integrates the driving fatigue situation of estimating the driver through the multi-source information to reflection driver driving fatigue situation; Driver fatigue state monitoring reliability and accuracy have been improved; Can reduce the traffic hazard quantity that driver tired driving causes, guarantee traffic safety; And the driving of obtaining not the driver of this multi-source information causes interference, and is practical.
Fig. 2 is the process flow diagram that the present invention is based on driver fatigue state recognition method second embodiment of information fusion, and as shown in Figure 2, present embodiment comprises:
Step 201, obtain eye status information, bearing circle operation state information and the track of vehicle transport condition information of reflection driver fatigue state, and obtain tired performance information through handling;
In the present embodiment technical scheme; At first adopt the method for expert's video scoring; Video recording to the facial state of driver in the whole driving procedure is cut into per 1 minute independent video, and several experts are carried out the fatigue scoring that effective statistical treatment obtains whole driving procedure to per 1 minute scoring.Tired grade is divided into 4 grades here, promptly clear-headed, slight tired, moderate is tired and severe is tired.With above-mentioned expert's scoring is standard, confirms and obtain the information type of needed reflection driver fatigue state, comprises eye status information, bearing circle operation state information and track of vehicle transport condition information.On this basis; Use statistical methods such as correlation analysis, variance analysis to carry out statistical treatment through driver's eye status information, bearing circle operation state information and the track of vehicle transport condition information of sensor acquisition, extract with driver fatigue and estimate the strongest tired performance information of correlativity.For example; Driver's eye parameter is to carry out statistical treatment through the single frames degree of closing one's eyes that the eyes image processing of gathering is obtained to obtain; In the unit interval, this driver's eye parameter comprises on average degree of opening (can represent with ratio value) of the PERCLOS80 the strongest with tired correlativity (the unit interval eyes closed surpasses 80% shared time scale), the longest closing one's eyes the time (closed surpass 80% promptly be judged as close one's eyes) and eyes; The bearing circle parameter is to carry out statistical treatment through the instantaneous direction dish corner to the collection of bearing circle scrambler to obtain, and this bearing circle parameter comprises that said bearing circle keeps motionless time scale and the unexpected angle variable quantity that rotates of bearing circle in the unit interval; The track of vehicle parameter be track of vehicle through vehicle being exercised sensor acquisition apart from the lateral excursion of center line of road apart from obtaining through statistical treatment; This track of vehicle parameter comprises the transversal displacement standard deviation of the snakelike transport condition of vehicle and the time scale that transversal displacement is in the safe distance scope, and this safe distance is that transversal displacement is less than 100mm/s.
Step 202, the tired performance information input artificial neural network that will obtain obtain the elementary tired degree of membership in the different measuring cycle in corresponding different parameters space.
Eye state parameter in the tired performance information, bearing circle operation state parameter and track of vehicle transport condition parameter are imported the neural network model that trains respectively, obtain elementary evaluation result, promptly belong to the elementary degree of membership of each tired grade.Present embodiment can adopt three layers of BP feedforward network, the following formula of the model formation of these three layers of BP feedforward networks (1):
y k＝f 2(ω (2)，f 1(ω (1)，x k，b (1))，b (2)) (1)
Wherein, f 1(), f 2() is respectively latent layer of neural network and the neuronic transport function of output layer; ω (1), ω (2)Be respectively and connect input layer and latent layer, latent layer and the neuronic weight vector of output layer; b (1), b (2)Be respectively latent layer and the neuronic offset vector of output layer; x kBe the parameter in different parameters space, such as eye state parameter, bearing circle operation state parameter or track of vehicle transport condition parameter, y kElementary degree of membership for corresponding each tired grade.Concrete output layer y kExpression as shown in table 1 below:
The tired grade statement of table 1
Figure G2009100825662D00061
For example; As above first in the table 1 gone; 0.9,0.1,0.1,0.1 identify clear-headed to tired grade, slight tired, moderate is tired, severe is tired degree of membership respectively, when the clear-headed degree of membership of tired grade 0.9 was the highest, the corresponding tired grade of this output result was " regaining consciousness ".
Neural network in the present embodiment is classified according to driver's physiological characteristic and operating feature in advance; Utilize dissimilar drivers' training sample that the BP neural network of setting up is carried out off-line training; Weight and biasing are stored into notepad, when carrying out real-time judge, call in stored parameters; According to the thought of template matches, use corresponding driver's type training parameter to carry out online judgement; So parameter comprises the dissimilar parameters of mating different drivers respectively in this neural network model.
Step 203, the tired grade degree of membership that will pass through the different measuring cycle of neural network output are carried out multicycle D_S evidence theory and are merged, and draw the basic confidence level in corresponding different parameters space.
The measurement of sensor comprises a plurality of measuring periods; The degree of membership in different measuring cycle is carried out single-sensor multicycle D_S evidence theory merge, at first the elementary tired degree of membership with a plurality of measuring periods of neural network output is converted into the basic reliability distribution parameter as the important evidence of D_S evidence theory through normalization; Formula (2) below utilizing then merges the basic reliability distribution parameter of a plurality of measuring periods, just can draw the basic confidence level in different parameters space.This fusion formula of D_S evidence theory that merges a plurality of measuring periods is following:
M ( A i ) = c - 1 Σ I A j = A i Π 1 ≤ s ≤ n M s ( A i ) , i = 1,2 , . . . , k - - - ( 2 )
Wherein
c = 1 - Σ I A i = Φ Π 1 ≤ s ≤ n M s ( A i ) = Σ I A = φ Π 1 ≤ s ≤ n M s ( A i )
To be sensor distribute formula according to the cumulative measurement of n measuring period to the fusion posteriority confidence level of k proposition to following formula, wherein, establishes M 1(A i), M 2(A i) ... M n(A i) be sensor in each measuring period, distribute the posteriority confidence level distribution that obtains through continuous target situation and changeless priori confidence level, i=1,2 ..., k, M j(A i) be illustrated in j cycle proposition A iConfidence level distribute.Can draw the basic confidence level in different parameters space through above-mentioned formula 2, i.e. eye state parameter confidence level, bearing circle operation state parameter confidence level and track of vehicle transport condition parameter confidence level.
The fusion that neural network in process step 202～203 and D_S evidence theory have been accomplished the subsystem irrespective of size continues execution in step 204～step 205, carries out system-level fusion.
Step 204, definite fusion weight of tired performance information being carried out weighting fusion.
Eye state parameter, bearing circle operation state parameter and track of vehicle transport condition parameter and driver's fatigue state all has certain correlativity; But relevant degree is different, therefore also should be different with the corresponding fusion weight of above-mentioned three types of parameters.This fusion weight is to confirm through the parameter confidence level that obtains in the system-level fusion of Chang Quanchong zygote.
At first, Chang Quanchong utilizes that the normalized result of class separability index confirms in the pattern-recognition, and the class separability of big more this index of explanation of class separability index in the pattern-recognition is good more.Utilize the parameter in different parameters space that the separating capacity of fatigue is confirmed Chang Quanchong at system-level level.This Chang Quanchong representes that with J it confirms that method is to utilize following formula (3):
J = Σ i M Σ j ≠ i M ( μ i - μ j ) 2 σ i 2 + σ j 2 , - - - ( 3 )
Wherein, but J representes the classification capacity of parameter, but representes the classification capacity of eye state parameter such as the Chang Quanchong J of eye state parameter.μ i, μ jRepresent adjacent type average respectively, σ i, σ jBe adjacent type variance, M representes classification number altogether.Handle as Chang Quanchong through the J value being carried out normalization.Adjacent class is meant adjacent tired grade, is two adjacent type such as slight fatigue and moderate fatigue.Calculate in three parameter spaces the average or the variance of corresponding with adjacent type parameter respectively, can obtain μ i, μ j and σ i, σ j.Tired grade in the present embodiment is 4 grades, so M equals 4.Can obtain the Chang Quanchong separately of three types of corresponding parameters, i.e. eye state parameter Chang Quanchong, bearing circle operation state parameter Chang Quanchong and track of vehicle transport condition parameter Chang Quanchong through above-mentioned formula 3.
Then, confirm to merge weight through the parameter confidence level that obtains in the system-level fusion of Chang Quanchong zygote.Can preestablish one with reference to the confidence level upper limit with reference to the confidence level lower limit, will be set at 0.8 with reference to the confidence level upper limit in the present embodiment, will be 0.5 with reference to the confidence level lower limit set.The parameter and the parameter confidence level of each parameter space are reported to fusion center together.When parameter confidence level＜0.5, merging weight is zero, does not promptly merge this parameter space, and the parameter of this parameter space is rejected; When parameter confidence level＞0.8, do not become power this moment, use Chang Quanchong to merge; When the parameter confidence level is between the 0.5-0.8, calculate the fusion weight according to formula (4)-(9).This moment the parameter confidence level with merge the linear proportional relationship of weight, Chang Quan likens to and is scale-up factor, the relation between the weight as shown in the formula:
W 1 W 3 = w 13 × r 1 r 3 - - - ( 4 )
W 2 W 3 = w 23 × r 2 r 3 - - - ( 5 )
W 1+W 2+W 3＝1 (6)
W 1 = r 1 w 12 r 1 w 12 + r 2 + r 3 w 32 - - - ( 7 )
W 2 = r 2 r 1 w 12 + r 2 + w 32 r 3 - - - ( 8 )
W 3 = r 3 r 1 w 13 + r 2 w 23 + r 3 - - - ( 9 )
Wherein, W 1, W 2, W 3Represent that respectively the eye state parameter merges weight, bearing circle operation state parameter merges weight and track of vehicle transport condition parameter merges weight; r 1, r 2, r 3Represent eye state parameter confidence level, bearing circle operation state parameter confidence level and track of vehicle transport condition parameter confidence level respectively; w 12, w 23, w 13Represent the ratio, eye state parameter Chang Quanchong of the ratio, bearing circle operation state parameter Chang Quanchong of eye state parameter Chang Quanchong and bearing circle operation state parameter Chang Quanchong and track of vehicle transport condition parameter Chang Quanchong and the ratio of track of vehicle transport condition parameter Chang Quanchong respectively.
Step 205, self D_S that tired performance information is carried out variable weight merge, and obtain the tired degree of membership corresponding with the driver fatigue grade, and draw driver's tired grade according to said tired degree of membership.
Self D_S that eye state parameter, bearing circle operation state parameter and track of vehicle transport condition parameter is carried out weight according to the fusion weight of three types of different parameters that obtain in the step 204 merges; Promptly merge at the system-level multisensor D_S evidence theory that carries out the different parameters space; Obtain the tired degree of membership corresponding, and draw driver's tired grade according to this fatigue degree of membership with the driver fatigue grade.Formula (2) is still adopted in multisensor D_S evidence theory fusion in this step, just the data fusion of a plurality of measuring periods is changed into the data fusion of a plurality of parameter spaces.
Present embodiment integrates the driving fatigue situation of estimating the driver through the multi-source information to reflection driver driving fatigue situation; Tripartite surface information can more comprehensively be estimated fatigue; Can not make evaluation system paralyse if one-sided information loses efficacy yet, and still can obtain fatigue state, thereby improve driver fatigue state monitoring reliability and accuracy; Can reduce the traffic hazard quantity that driver tired driving causes, guarantee traffic safety; And the driving of obtaining not the driver of this multi-source information causes interference, and is practical.
Fig. 3 is the structural representation that the present invention is based on driver fatigue state recognition system first embodiment of information fusion; As shown in Figure 3; Present embodiment driver fatigue monitoring system can comprise acquisition module 301, processing module 302 and Fusion Module 303; Wherein, acquisition module 301 is used to gather driver's eye status information, bearing circle operation state information and track of vehicle transport condition information; Eye status information, bearing circle operation state information and track of vehicle transport condition information that processing module 302 is used for said driver are handled, and obtain driver's tired performance information; Fusion Module 303 is used for that said tired performance information is carried out weighting fusion treatment and obtains the tired degree of membership corresponding with the driver fatigue grade, and draws driver's tired grade according to said tired degree of membership.
Concrete; Eye status information, bearing circle operation state information and track of vehicle transport condition information that acquisition module 301 is gathered the reflection driver fatigue state; Processing module 302 is carried out analyzing and processing with the above-mentioned information that acquisition module 301 obtains; Obtain driver's tired performance information, this fatigue performance information comprises that eyes closed degree in the unit interval surpasses 80% shared time scale, on average close one's eyes time, the longest time of closing one's eyes, bearing circle keeps motionless time scale, angle variable quantity, the transversal displacement standard deviation of the snakelike transport condition of vehicle and the time scale that said transversal displacement is in the safe distance scope that bearing circle rotates suddenly.Fusion Module 303 carries out above-mentioned tired performance information can obtain the comprehensive tired degree of membership corresponding with tired grade after the weighting fusion treatment, and draws driver's tired grade according to this comprehensive tired degree of membership.
Present embodiment driver fatigue monitoring system integrates the driving fatigue situation of estimating the driver through the multi-source information to reflection driver driving fatigue situation; Driver fatigue state monitoring reliability and accuracy have been improved; Can reduce the traffic hazard quantity that driver tired driving causes, guarantee traffic safety; And the driving of obtaining not the driver of this multi-source information causes interference, and is practical.
Fig. 4 is the structural representation that the present invention is based on driver fatigue state recognition system second embodiment of information fusion; As shown in Figure 4; Present embodiment is on the basis of first embodiment; Fusion Module 303 can comprise subsystem irrespective of size integrated unit 3031 and system-level integrated unit 3032; Wherein, subsystem irrespective of size integrated unit 3031 is used for the eye status information of said tired performance information, bearing circle operation state information and track of vehicle transport condition information are carried out weighting fusion treatment respectively, obtains eye status information confidence level, bearing circle operation state information credibility and track of vehicle transport condition information credibility; System-level integrated unit 3032 confirms respectively that according to said eye status information confidence level, bearing circle operation state information credibility and track of vehicle transport condition information credibility the eye status information merges weight, bearing circle operation state information fusion weight and track of vehicle transport condition information fusion weight; And merge weight, bearing circle operation state information fusion weight and track of vehicle transport condition information fusion weight according to said eye status information said eye status information, bearing circle operation state information and track of vehicle transport condition information are carried out weighting fusion treatment, obtain driver's tired grade.
Further; Subsystem irrespective of size integrated unit 3031 may further include first subelement, second subelement; Wherein, First subelement is used for said eye status information, bearing circle operation state information and track of vehicle transport condition information are carried out the elementary tired degree of membership that weighting fusion obtains corresponding driver fatigue grade according to the neural network parameter qualified relation, and said elementary tired degree of membership comprises corresponding a plurality of said eye status informations, bearing circle operation state information and the elementary tired degree of membership in track of vehicle transport condition information measurement cycle; Second subelement is used for that said elementary tired degree of membership is carried out the normalization processing and is converted into the basic reliability distribution parameter; And the said a plurality of said eye status informations of correspondence, bearing circle operation state information and the basic reliability distribution parameter in track of vehicle transport condition information measurement cycle are carried out the D_S evidence theory merge, obtain eye state parameter confidence level, bearing circle operation state parameter confidence level and track of vehicle transport condition parameter confidence level respectively.System-level integrated unit 3032 comprises that weight confirms subelement, is used for confirming respectively that according to said eye state parameter confidence level, bearing circle operation state parameter confidence level and track of vehicle transport condition parameter confidence level the eye state parameter merges weight, bearing circle operation state parameter merges weight and track of vehicle transport condition parameter merges weight.
Concrete; On the basis of first embodiment; The process that 303 pairs of above-mentioned tired performance informations of Fusion Module carry out weighting fusion treatment can be divided into the subsystem irrespective of size and merge and two steps of system-level fusion, and neural network merges and the D_S evidence theory merges two steps and the fusion of subsystem irrespective of size can be divided into.The artificial neural network that first subelement in the subsystem irrespective of size integrated unit 3031 has trained eye state parameter, bearing circle operation state parameter and the input of track of vehicle transport condition parameter earlier; Can obtain the corresponding elementary tired degree of membership that belongs to the different fatigue grade of these three kinds of parameter spaces respectively, and be the elementary tired degree of membership in different measuring cycle.Second subelement carries out normalization with above-mentioned elementary tired degree of membership and handles the basic reliability distribution parameter that is converted into the corresponding different measuring cycle; Utilize the D_S evidence theory that the basic reliability distribution parameter of single space many measuring periods is merged then; Just can obtain the basic confidence level of every type of parameter space, i.e. eye state parameter confidence level, bearing circle operation state parameter confidence level and track of vehicle transport condition parameter confidence level.When 3032 pairs of three types of parameters of system-level integrated unit are carried out weighting fusion; Weight wherein confirms that subelement needs to confirm earlier the fusion weight of three types of parameters; This fusion weight is to change according to the basic confidence level that second subelement draws; Be to combine the confidence level of all kinds of parameters to confirm that comprehensively concrete definite method can be referring to method embodiment through Chang Quanchong.After the fusion weight was confirmed, 3032 of system-level integrated units can merge weight according to this three types of parameters are carried out weighting fusion and self variable weight D_S evidence theory fusion, obtain tired grade result.
Present embodiment driver fatigue monitoring system integrates the driving fatigue situation of estimating the driver through the multi-source information to reflection driver driving fatigue situation; Driver fatigue state monitoring reliability and accuracy have been improved; Can reduce the traffic hazard quantity that driver tired driving causes, guarantee traffic safety; And the driving of obtaining not the driver of this multi-source information causes interference, and is practical.
What should explain at last is: above embodiment is only in order to technical scheme of the present invention to be described but not limit it; Although the present invention has been carried out detailed explanation with reference to preferred embodiment; Those of ordinary skill in the art is to be understood that: it still can make amendment or be equal to replacement technical scheme of the present invention, also can not make amended technical scheme break away from the spirit and the scope of technical scheme of the present invention and these are revised or be equal to replacement.

Claims (6)

1. the driver fatigue state recognition method based on information fusion is characterized in that, comprising:
Step 1, collection driver's eye status information, bearing circle operation state information and track of vehicle transport condition information;
Step 2, the eye status information to said driver, bearing circle operation state information and track of vehicle transport condition information are handled; Acquisition driver's tired performance information, said tired performance information comprise that on average degree of opening, the longest time of closing one's eyes, bearing circle keep motionless time scale, angle variable quantity, the transversal displacement standard deviation of the snakelike transport condition of vehicle and the time scale that said transversal displacement is in the safe distance scope of the unexpected rotation of bearing circle to interior eyes closed degree of unit interval above 80% shared time scale, eyes;
Step 3, said tired performance information is carried out weighting fusion treatment obtain the tired degree of membership corresponding, and draw driver's tired grade according to said tired degree of membership with the driver fatigue grade;
Wherein, said step 3 comprises:
Step 31, eyes closed degree in the said unit interval is surpassed on average degree of opening and grow the time of closing one's eyes most and merge and obtain eye state parameter confidence level of 80% shared time scale, eyes; With said bearing circle keep motionless time scale and bearing circle suddenly the angle variable quantity of rotation merge and obtain bearing circle operation state parameter confidence level, the time scale that the transversal displacement standard deviation and the said transversal displacement of the snakelike transport condition of said vehicle is in the safe distance scope merges and obtains track of vehicle transport condition parameter confidence level;
Step 32, confirm that according to said eye state parameter confidence level the eye state parameter merges weight; Confirm that according to said bearing circle operation state parameter confidence level bearing circle operation state parameter merges weight, obtain track of vehicle transport condition parameter according to said track of vehicle transport condition parameter confidence level and merge weight; According to said eye state parameter fusion weight, bearing circle operation state parameter fusion weight and track of vehicle transport condition parameter fusion weight said tired performance information is carried out weighting fusion treatment and obtain the tired degree of membership corresponding, and draw driver's tired grade according to said tired degree of membership with the driver fatigue grade.
2. method according to claim 1 is characterized in that, said step 31 specifically comprises:
Step 311, said tired performance information is merged according to neural network model, obtain the elementary tired degree of membership of eye state parameter, the elementary tired degree of membership of bearing circle operation state parameter and the elementary tired degree of membership of track of vehicle transport condition parameter in the different measuring cycle of corresponding tired performance information respectively;
Step 312, the elementary tired degree of membership in different measuring cycle is converted into the basic reliability distribution parameter, and the basic reliability distribution parameter in said different measuring cycle merged according to D-S evidence theory model obtains eye state parameter confidence level, bearing circle operation state parameter confidence level and track of vehicle transport condition parameter confidence level respectively.
3. method according to claim 1 is characterized in that, the parameter confidence level comprises said eye state parameter confidence level, bearing circle operation state parameter confidence level and track of vehicle transport condition parameter confidence level, and said step 32 comprises:
Step 321, extract predefined with reference to the confidence level upper limit with reference to the confidence level lower limit;
Step 322, with said parameter confidence level with compare with reference to the confidence level upper limit with reference to the confidence level lower limit, if said parameter confidence level less than said with reference to the confidence level lower limit, then merging weight with said parameter confidence level corresponding parameters is zero; If with reference to the confidence level upper limit, then merge weight with said parameter confidence level corresponding parameters is to be used for the Chang Quanchong of identification parameter to the separating capacity of fatigue to said parameter confidence level greater than said; If said parameter confidence level with reference to the confidence level lower limit with between with reference to the confidence level upper limit, then merges weight and the linear proportional relationship of said parameter confidence level with said parameter confidence level corresponding parameters said.
4. the driver fatigue state recognition system based on information fusion is characterized in that, comprising:
Acquisition module is used to gather driver's eye status information, bearing circle operation state information and track of vehicle transport condition information;
Processing module; The eye status information, bearing circle operation state information and the track of vehicle transport condition information that are used for said driver are handled; Acquisition driver's tired performance information, said tired performance information comprise that on average degree of opening, the longest time of closing one's eyes, bearing circle keep motionless time scale, angle variable quantity, the transversal displacement standard deviation of the snakelike transport condition of vehicle and the time scale that said transversal displacement is in the safe distance scope of the unexpected rotation of bearing circle to interior eyes closed degree of unit interval above 80% shared time scale, eyes;
Fusion Module is used for that said tired performance information is carried out weighting fusion treatment and obtains the tired degree of membership corresponding with the driver fatigue grade, and draws driver's tired grade according to said tired degree of membership;
Wherein, said Fusion Module comprises:
Subsystem irrespective of size integrated unit; Be used for eyes closed degree in the said unit interval is surpassed on average degree of opening and grow the time of closing one's eyes most and merge and obtain eye state parameter confidence level of 80% shared time scale, eyes; With said bearing circle keep motionless time scale and bearing circle suddenly the angle variable quantity of rotation merge and obtain bearing circle operation state parameter confidence level, the time scale that the transversal displacement standard deviation and the said transversal displacement of the snakelike transport condition of said vehicle is in the safe distance scope merges and obtains track of vehicle transport condition parameter confidence level;
System-level integrated unit; Be used for basis and said eye state parameter confidence level, eye state parameter that bearing circle operation state parameter confidence level is relevant with track of vehicle transport condition parameter confidence level and merge weight, bearing circle operation state parameter and merge weight and track of vehicle transport condition parameter and merge weight and said tired performance information is carried out weighting fusion treatment obtain the tired degree of membership corresponding, and draw driver's tired grade according to said tired degree of membership with the driver fatigue grade.
5. system according to claim 4 is characterized in that, said subsystem irrespective of size integrated unit comprises:
First subelement; Be used for said tired performance information is merged according to neural network model, obtain the elementary tired degree of membership of eye state parameter, the elementary tired degree of membership of bearing circle operation state parameter and the elementary tired degree of membership of track of vehicle transport condition parameter in the different measuring cycle of corresponding tired performance information respectively;
Second subelement; Be used for the elementary tired degree of membership in different measuring cycle is converted into the basic reliability distribution parameter, and the basic reliability distribution parameter in said different measuring cycle merged according to D-S evidence theory model obtain eye state parameter confidence level, bearing circle operation state parameter confidence level and track of vehicle transport condition parameter confidence level respectively.
6. system according to claim 4 is characterized in that, said system-level integrated unit comprises:
Weight is confirmed subelement, is used for confirming respectively that according to said eye state parameter confidence level, bearing circle operation state parameter confidence level and track of vehicle transport condition parameter confidence level the eye state parameter merges weight, bearing circle operation state parameter merges weight and track of vehicle transport condition parameter merges weight.
CN2009100825662A 2009-04-24 2009-04-24 Driver fatigue state recognition method and system based on information fusion Expired - Fee Related CN101872171B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN2009100825662A CN101872171B (en) 2009-04-24 2009-04-24 Driver fatigue state recognition method and system based on information fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN2009100825662A CN101872171B (en) 2009-04-24 2009-04-24 Driver fatigue state recognition method and system based on information fusion

Publications (2)

Publication Number Publication Date
CN101872171A CN101872171A (en) 2010-10-27
CN101872171B true CN101872171B (en) 2012-06-27

Family

ID=42997091

Family Applications (1)

Application Number Title Priority Date Filing Date
CN2009100825662A Expired - Fee Related CN101872171B (en) 2009-04-24 2009-04-24 Driver fatigue state recognition method and system based on information fusion

Country Status (1)

Country Link
CN (1) CN101872171B (en)

Families Citing this family (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE112012006288B4 (en) * 2012-04-25 2020-09-03 Toyota Jidosha Kabushiki Kaisha Drift judging device
CN103473890B (en) * 2013-09-12 2015-09-16 合肥工业大学 Based on driver fatigue real-time monitoring system and the monitoring method of multi information
CN103578227B (en) * 2013-09-23 2015-10-07 吉林大学 Based on the method for detecting fatigue driving of GPS locating information
CN103989471B (en) * 2014-05-08 2015-11-04 东北大学 A kind of method for detecting fatigue driving based on electroencephalogram identification
DE102014214214A1 (en) * 2014-07-22 2016-01-28 Robert Bosch Gmbh Method and device for determining a degree of fatigue of a driver of a vehicle and vehicle
CN104391569A (en) * 2014-10-15 2015-03-04 东南大学 Brain-machine interface system based on cognition and emotional state multi-mode perception
CN104952209A (en) * 2015-04-30 2015-09-30 广州视声光电有限公司 Driving prewarning method and device
JP6399311B2 (en) * 2015-05-07 2018-10-03 スズキ株式会社 Dozing detection device
CN105160818B (en) * 2015-08-26 2019-10-29 厦门雅迅网络股份有限公司 It is a kind of based on preceding method for detecting fatigue driving and device to risk of collision
CN105139070B (en) * 2015-08-27 2018-02-02 南京信息工程大学 fatigue driving evaluation method based on artificial neural network and evidence theory
CN105303829A (en) * 2015-09-11 2016-02-03 深圳市乐驰互联技术有限公司 Vehicle driver emotion recognition method and device
JP6666705B2 (en) * 2015-09-28 2020-03-18 デルタ工業株式会社 Biological condition estimation device, biological condition estimation method, and computer program
CN105320966A (en) * 2015-10-30 2016-02-10 东软集团股份有限公司 Vehicle driving state recognition method and apparatus
CN105261153A (en) * 2015-11-03 2016-01-20 北京奇虎科技有限公司 Vehicle running monitoring method and device
DE102015225109A1 (en) * 2015-12-14 2017-06-14 Robert Bosch Gmbh A method and apparatus for classifying eye opening data of at least one eye of an occupant of a vehicle and method and apparatus for detecting drowsiness and / or instantaneous sleep of an occupant of a vehicle
CN105740847A (en) * 2016-03-02 2016-07-06 同济大学 Fatigue grade discrimination algorithm based on driver eye portion identification and vehicle driving track
CN106236047A (en) * 2016-09-05 2016-12-21 合肥飞鸟信息技术有限公司 The control method of driver fatigue monitoring system
CN106355837A (en) * 2016-09-09 2017-01-25 常州大学 Fatigue driving monitoring method on basis of mobile phone
CN106251583B (en) * 2016-09-30 2018-09-25 江苏筑磊电子科技有限公司 Fatigue driving discrimination method based on driving behavior and eye movement characteristics
CN106580349B (en) * 2016-12-07 2020-01-21 中国民用航空总局第二研究所 Controller fatigue detection method and device and controller fatigue response method and device
CN107233103B (en) * 2017-05-27 2020-11-20 西南交通大学 High-speed rail dispatcher fatigue state evaluation method and system
CN107379967A (en) * 2017-07-14 2017-11-24 南京信息工程大学 A kind of Drunken driving measure and control device based on multi-sensor information fusion
CN107844777B (en) * 2017-11-16 2021-06-11 百度在线网络技术（北京）有限公司 Method and apparatus for generating information
CN108108766B (en) * 2017-12-28 2021-10-29 东南大学 Driving behavior identification method and system based on multi-sensor data fusion
JP6714036B2 (en) * 2018-04-25 2020-06-24 株式会社日立物流 Management support system
CN110641467A (en) * 2018-06-25 2020-01-03 广州汽车集团股份有限公司 Vehicle distance control method and device of adaptive cruise system
CN110177299B (en) * 2018-08-30 2021-04-30 深圳市野生动物园有限公司 Video playing speed adjusting platform
CN109584507B (en) * 2018-11-12 2020-11-13 深圳佑驾创新科技有限公司 Driving behavior monitoring method, device, system, vehicle and storage medium
DE102018221063A1 (en) 2018-12-05 2020-06-10 Volkswagen Aktiengesellschaft Configuration of a control system for an at least partially autonomous motor vehicle
CN110143202A (en) * 2019-04-09 2019-08-20 南京交通职业技术学院 A kind of dangerous driving identification and method for early warning and system
CN110371133A (en) * 2019-06-28 2019-10-25 神龙汽车有限公司 Fatigue of automobile driver detection system and method
CN112233276B (en) * 2020-10-13 2022-04-29 重庆科技学院 Steering wheel corner statistical characteristic fusion method for fatigue state recognition
CN114820692B (en) * 2022-06-29 2023-07-07 珠海视熙科技有限公司 State analysis method, device, storage medium and terminal for tracking target
CN115366909B (en) * 2022-10-21 2023-04-07 四川省公路规划勘察设计研究院有限公司 Dynamic early warning method and device for driver accidents in long and large longitudinal slope section and electronic equipment
CN116682264B (en) * 2023-07-11 2023-12-26 营口港信科技有限公司 Active safety prevention and control system for port vehicle

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1407916A2 (en) * 2002-10-11 2004-04-14 Audi Ag Vehicle
CN1830389A (en) * 2006-04-21 2006-09-13 太原理工大学 Device for monitoring fatigue driving state and its method
CN101089906A (en) * 2006-06-14 2007-12-19 吴玉沈 Active vehicle driver fatigue driving alarm device and method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1407916A2 (en) * 2002-10-11 2004-04-14 Audi Ag Vehicle
CN1830389A (en) * 2006-04-21 2006-09-13 太原理工大学 Device for monitoring fatigue driving state and its method
CN101089906A (en) * 2006-06-14 2007-12-19 吴玉沈 Active vehicle driver fatigue driving alarm device and method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
李埃荣.《基于多传感器数据融合的驾驶状态监控研究》.《中国优秀硕士学位论文全文数据库（电子期刊）》.2008,(第4期), *

Also Published As

Publication number Publication date
CN101872171A (en) 2010-10-27

Similar Documents

Publication Publication Date Title
CN101872171B (en) Driver fatigue state recognition method and system based on information fusion
US7389178B2 (en) System and method for vehicle driver behavior analysis and evaluation
Das et al. Differentiating alcohol-induced driving behavior using steering wheel signals
US20070005404A1 (en) System and method for providing driving insurance
CN109000940A (en) A kind of rolling stock exception axis temperature diagnostic method and system
CN109177982A (en) Consider the vehicle driving Hazard degree assessment method of driving style
CN103824420A (en) Fatigue driving identification system based on heart rate variability non-contact measuring
CN110509983A (en) A kind of steering-by-wire road feel feedback device suitable for different drive demands
CN104794855A (en) Driver's attention comprehensive assessment system
CA2611408A1 (en) System and method for displaying a driving profile
CN105139070A (en) Fatigue driving evaluation method based on artificial nerve network and evidence theory
CN105261151A (en) High-grade highway driver fatigue state detection method based on operation behavior characteristics
CN109229108A (en) A kind of driving behavior safe evaluation method based on driving fingerprint
Peng et al. Intelligent method for identifying driving risk based on V2V multisource big data
CN106952361A (en) A kind of efficient vehicle running state identifying system
CN112991685A (en) Traffic system risk assessment and early warning method considering fatigue state influence of driver
Liu et al. A review of driver fatigue detection: Progress and prospect
Cheng et al. Quantitative evaluation methodology for chassis-domain dynamics performance of automated vehicles
CN113647956A (en) Automatic driving automobile comfort level test evaluation method and system based on automatic driving robot
Barendswaard et al. A classification method for driver trajectories during curve-negotiation
CN108537161A (en) A kind of driving of view-based access control model characteristic is divert one's attention detection method
CN110154893B (en) Automobile safe driving early warning method based on driver characteristics
CN110047167A (en) Vehicle zig zag number equivalent statistics method
CN112660141A (en) Method for identifying driver driving distraction behavior through driving behavior data
Chen et al. Driving-data-driven platform of driving behavior spectrum for vehicle networks

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
C17 Cessation of patent right
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20120627

Termination date: 20130424