CN1680779A - Fatigue monitoring method and device for driver - Google Patents

Abstract

A method for monitoring fatigue strength of driver includes shining driver eye with infrared ray to obtain multiimage of different retina image at the same time, carrying out difference processing for collected images to obtain pupil images, using kalmen filter to trace pupil in real - time to obtain pupil characteristic parameter, processing the parameter to obtain maximum value of pupil size and real - time coroclisis percentage of pupil, calculating out PERCLOS value f and using BP network sorter to judge fatigue strength of driver based on obtained value f. The monitoring device is also disclosed.

Description

Driver fatigue monitoring method and device

Technical field

The present invention relates to Communication and Transportation Engineering, refer in particular to a kind of driver fatigue monitoring method and device.

Background technology

At present mainly understand its behavior state, for safe driving provides necessary supplementary based on monitoring driving person's mouth state at the identification monitoring technology of driver's eye fatigue characteristic.Pertinent literature has Shi Shuming, Jin Bensheng, Wang Rongben, child soldiers are bright, the 34th the 2nd phase of volume of Jilin University's journal (engineering version), in April, 2004, " based on driver's mouth condition detection method of machine vision " since the driver normal driving, speak and (drowsiness) the three kinds of states of yawning under the mouth stretching degree certain difference is arranged.According to these characteristics, the author utilizes the Fisher sorter to extract the profile and the position of lip, the geometric properties that utilizes lip region then is as eigenwert, the composition characteristic vector, as the input of three layers of BP neural network, with normal driving, speak and (drowsiness) the three kinds of different state of mind of yawning as output.

But, because the restriction of method itself, at first, be subjected to the influence of illumination brightness etc., can not satisfy round-the-clock requirement, secondly, be to discern at the single-frame images of mouth in background technology, because fatigue characteristic is not obvious, other action easy and driver is obscured, and discrimination is not high.

Summary of the invention

At above-mentioned deficiency, the present invention proposes driver fatigue monitoring method and device at driver's eye fatigue characteristic.Utilize the irradiation of Infrared, satisfied round-the-clock requirement, can not influence the driver again simultaneously and drive normally driver's eye; Adopt PERCLOS (Percentage of Eyelid Closure Over the Pupil Over Time) index as criterion, and employing BP network classifier assisting as evaluation criterion, further improve the accuracy of estimating, remedy the limitation of experimental data, thereby can better adapt to different crowd.

Realize the method for the technical scheme of above-mentioned purpose based on infrared light supply, difference image, KALMAN wave filter, and design system prototype, test shows, can satisfy requirement real-time, round-the-clock, high discrimination fully.

Main technical schemes:

1. image acquisition:

Utilize the irradiation of Infrared, obtain several by a plurality of CMOS cameras and have only the different multiple image of retinal images at synchronization to driver's eye.

The device of realizing said method is mainly by infrared light supply, the CMOS camera, and control main board and corresponding software are partly formed.It utilizes the camera of 2 separation, 90 ° of intersections.When image through a beam splitter, be divided into 2 the bundle enter respectively in the camera lens of 2 cameras, then, 2 camera lenses obtain corresponding infrared image with the wave filter of 850nm and 950nm wavelength respectively.The result just obtains 2 width of cloth and has only 2 different width of cloth images of retinal images at synchronization.

2. Flame Image Process, real-time follow-up:

The picture signal that collects is carried out difference processing by image processing program built-in in the control main board, obtains pupil image.Utilize the auxiliary Kalman wave filter of neural network that pupil is carried out the real-time follow-up prediction simultaneously.

3. calculate coupling:

The characteristic parameter of the pupil that obtains is transferred to control module and is handled, and obtains the maximal value of pupil size and real-time coreclisis number percent by statistical treatment, calculates PERCLOS value f, judges driver's degree of fatigue then.

PERCLOS is meant that the eyes closed time accounts for the percent of a certain special time, and the P80 of PERCLOS (time that the eyes closed degree surpasses 80% or more in the unit interval accounts for the number percent of T.T.) and the correlativity of driving fatigue degree are best.

As long as measure t ₁～t ₄Value just can be calculated f:

$f=\frac{t_3-t_2}{t_4-{\mathrm{<figure-callout\; id="1"\; label="t"\; filenames="A20051003777100081.png"\; state="\{\{state\}\}">t</figure-callout>}}_1}$

Wherein, f is that eyes closed surpasses the percent that time of 80% accounts for a certain special time.

4. tired the evaluation:

The evaluation criterion that system adopts is the P80 standard of PERCLOS (Percentage of Eyelid Closure Over the Pupil OverTime).And utilize BP network classifier auxiliary as evaluation criterion simultaneously.BP network based on region geometry feature neural network algorithm is a 3-tier architecture, and input layer has 4 neurons, represents the eigenwert t among the PERCLOS respectively ₁～t ₄Hidden layer has 10 neurons, and output layer has 3 neurons, represents the 3 kinds of different conditions of the eigenwert f among the PERCLOS, and the transport function of hidden layer is the Sigmoid function.The output vector of network is Y ₁=[1,0,0], Y ₂=[0,1,0], Y ₃=[0,0,1].X wherein ₁～X ₄Represent t ₁～t ₄, Y ₁Represent the f value less than normal, Y ₂Represent the suitable Y of f value ₃Represent the f value bigger than normal.

The invention has the beneficial effects as follows:

1. adopted the method for machine vision to come driver's eye is followed the tracks of, monitored, avoided direct Body contact with the driver;

Thereby 2. at present by the research that detects pupil monitoring fatigue, adopted a kind of PERCLOS method best with the Pearson correlativity;

3. utilize infrared imaging, improved the applicability of device greatly, satisfy under any driving situation monitoring requirement driver status;

4. utilize the auxiliary Kalman wave filter of neural network that the eye characteristic parameter of gathering is handled, can be good at realizing tracking and prediction, solve the problem of identification eye under the situation that driver's head rocks effectively driver's eye;

5. system has adopted the high CMOS image sensor of integrated level and based on the control main board of dsp processor, be convenient to car in original circuit integrated;

6. BP network classifier auxiliary as evaluation criterion can further be improved the accuracy of evaluation, remedied the limitation of experimental data, thereby can better adapt to different crowd.

Description of drawings

Fig. 1 device is formed and the testing process block diagram

The structural drawing of Fig. 2 PERCLOS camera

Fig. 3 BP network classifier structural drawing

The measuring principle synoptic diagram of Fig. 4 PERCLOS value f

The 1-fan; The 2-control main board; The 3-950nm filter; The 4-950nm filter; The 5-optical splitter

Embodiment

As shown in the figure, present embodiment is mainly by infrared light supply, the CMOS camera, and control main board and corresponding software are partly formed.Wherein camera is installed in the place ahead of driver, is as the criterion with the visual field that does not influence the driver.Camera adopts CMOS complementary metal oxide semiconductor (CMOS) (Complementary Metal-Oxide-Semiconductor) as sensor.

Consider applicability, utilize the basic physiological characteristics of human eye, promptly retina to the infrared light of different wave length can volume reflection difference.At the 850nm wavelength, can reflect 90% incident light, can only reflect 40% incident light at the 950nm retina.Under the situation of same illumination, 2 cameras are measured the image of human eye simultaneously, and one is the image of 850nm wavelength, another is the image of 950nm, the result of 2 width of cloth image subtractions just only stays the image of amphiblestroid position, and then analyzes amphiblestroid size and position.

The identical image in order to access 2 width of cloth light sources with different wavelengths utilizes the camera of 2 separation, 90 ° of intersections.When image through a beam splitter, be divided into 2 the bundle enter respectively in the camera lens of 2 cameras, then, 2 camera lenses obtain corresponding infrared image with the wave filter of 850nm and 950nm wavelength respectively.The result just obtains 2 width of cloth and has only 2 different width of cloth images of retinal images at synchronization.The structure of camera is seen Fig. 2.

Because what adopt is infrared light supply, can not have influence on driver's driver behavior on the one hand, on the other hand, can satisfy round-the-clock requirement effectively.

The picture signal that collects is carried out difference processing by image processing program built-in in the control main board, obtains pupil image.Utilize the auxiliary Kalman wave filter of neural network that pupil is carried out the real-time follow-up prediction simultaneously.

The characteristic parameter of the pupil that obtains is transferred to control module and is handled, and obtains the maximal value of pupil size and real-time coreclisis number percent by statistical treatment, calculates PERCLOS value f, judges driver's degree of fatigue then.Measuring principle such as Fig. 4 of PERCLOS value f.

As long as measure t ₁～t ₄Value just can be calculated f:

$f=\frac{t_3-t_2}{t_4-{\mathrm{<figure-callout\; id="1"\; label="t"\; filenames="A20051003777100081.png"\; state="\{\{state\}\}">t</figure-callout>}}_1}$

Wherein, f is that eyes closed surpasses the percent that time of 80% accounts for a certain special time.

The evaluation criterion that system adopts is the P80 standard of PERCLOS (Percentage of Eyelid Closure Over the Pupil OverTime).And utilize BP network classifier auxiliary as evaluation criterion simultaneously.BP network based on region geometry feature neural network algorithm is a 3-tier architecture, and input layer has 4 neurons, represents the eigenwert t among the PERCLOS respectively ₁～t ₄Hidden layer has 10 neurons, and output layer has 3 neurons, represents the 3 kinds of different conditions of the eigenwert f among the PERCLOS, and the transport function of hidden layer is the Sigmoid function.The output vector of network is Y ₁=[1,0,0], Y ₂=[0,1,0], Y ₃=[0,0,1].X wherein ₁~ X ₄Represent t ₁～t ₄, Y ₁Represent the f value less than normal, Y ₂Represent the suitable Y of f value ₃Represent the f value bigger than normal, the structure of this neural network as shown in Figure 3.

The system acquisition wavelength is the infrared image of 850nm/950nm and the difference image of eye.

Claims (2)

1. a driver fatigue monitoring method is characterized in that: utilize the irradiation of Infrared to driver's eye, obtain several by a plurality of CMOS cameras and have only the different image of retinal images at synchronization; The picture signal that collects is carried out difference processing by image processing program built-in in the control main board, obtains pupil image; Utilize the auxiliary Kalman wave filter of neural network that pupil is carried out the real-time follow-up prediction simultaneously; The characteristic parameter of the pupil that obtains is handled by control module, obtain the maximal value of pupil size and real-time coreclisis number percent by statistical treatment, calculate PERCLOS value f, and utilize the supplementary means of BP network classifier simultaneously as evaluation criterion, judge driver's degree of fatigue.

2. realize the device of the described driver fatigue monitoring method of claim 1, it is characterized in that: device is made of the CMOS camera of 2 separation, 90 ° of intersections infrared light supply, CMOS camera, control main board, be installed in the place ahead of driver, the CMOS camera through signal wire links to each other with mainboard.

Images