CN104881956A - Fatigue driving early warning system - Google Patents

Fatigue driving early warning system Download PDF

Info

Publication number
CN104881956A
CN104881956A CN201510335822.XA CN201510335822A CN104881956A CN 104881956 A CN104881956 A CN 104881956A CN 201510335822 A CN201510335822 A CN 201510335822A CN 104881956 A CN104881956 A CN 104881956A
Authority
CN
China
Prior art keywords
driver
fatigue
image
system
face
Prior art date
Application number
CN201510335822.XA
Other languages
Chinese (zh)
Inventor
徐美华
沈华明
王琪
郭爱英
沈东阳
Original Assignee
上海大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海大学 filed Critical 上海大学
Priority to CN201510335822.XA priority Critical patent/CN104881956A/en
Publication of CN104881956A publication Critical patent/CN104881956A/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00624Recognising scenes, i.e. recognition of a whole field of perception; recognising scene-specific objects
    • G06K9/00832Recognising scenes inside a vehicle, e.g. related to occupancy, driver state, inner lighting conditions
    • G06K9/00845Recognising the driver's state or behaviour, e.g. attention, drowsiness
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00Alarms responsive to a single specified undesired or abnormal operating condition and not elsewhere provided for
    • G08B21/02Alarms for ensuring the safety of persons
    • G08B21/06Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

The invention discloses a fatigue driving early warning system and a method thereof. The system comprises an embedded XC7Z020 processor 1, an infrared camera 2, an SD card 3, and a display 4. The embedded XC7Z020 processor 1 is used for fatigue detection of drivers, to determine whether the drivers are in a fatigued driving state. If yes, the system gives out an alarm message. The infrared camera 2 is used to acquire facial images of the drivers. The SD card 3 is used to store an embedded Linux system and FPGA configuration files, and when the system is initialized, a system is directly loaded and started. The display 4 is used to display faces of the drivers in real time, and process detection time and alarm messages. The system can satisfy real-time requirement of the fatigue driving early warning system, and response time is relatively short, so as to detect whether the driver is in a fatigued state. The system is high in detection reliability, and low in manufacturing cost.

Description

A kind of driver fatigue monitor system

Technical field

The present invention relates to physiology, control the related disciplines such as, recognition of face, embedded system, belong to automotive electronics and technical field of image processing, especially a kind of driver fatigue monitor system.

Background technology

In recent years, along with popularizing of automobile, road traffic system is growing, and meanwhile, traffic hazard also increases increasingly.According to the statistics made by the departments concerned, in current especially big traffic injuries and deaths accident, that causes due to driver tired driving accounts for 40%.Be in the driver of fatigue driving state, drive function and progressively decline, very easily traffic hazard occurs, constitute serious threat to safe driving, therefore, driver drowsy early warning system has just become an important research direction in automobile active safety system.

The fatigue detecting of driver is a very complicated problem, and early stage research, mainly from medical angle, is undertaken by medical device.From the eighties in 20th century, in order to really solve the test problems of practical automobile driver driving fatigue, the many experts in the world and scholar have carried out useful exploration, and propose much brand-new thought and method.Finally determine that PERCLOS method is as the tired assessment method of real-time driver by Highway Administration of the United States Federal.To the nineties in 20th century, the research of driver fatigue measuring method has had very large progress, and many countries have started the development and research of driving fatigue vehicle electronics measurement mechanism.

At present, the general thinking of Driver Fatigue Detection is: first determine the feature that can reflect driver fatigue that will choose, as electroencephalogram (EEG), frequency of wink etc., then feature extraction is carried out, the standard that the characteristic sum that last basis is extracted presets identifies, determines whether tired generation.The detection of brain wave is the most accurately, but will extract brain wave must contact human body, can cause interference to driving.Therefore, that seeks a kind of practicality the vehicular driver fatigue detection device of large-scale production can become the joint research direction of domestic and international researcher.

Existing application number is the patent of CN201310752652, its system includes DSP master control borad, video capture circuit plate, CCD camera, first DSP controls ccd video camera shooting driver original image, after DSP control panel receives the original image of collection, start driver fatigue detection algorithm, and carry out the judgement of illumination power by the character that optical illumination dark condition affects according to the Clustering features of the colour of skin, after selecting corresponding treatment measures to complete the location of human eye area adaptively by DSP mainboard, again in conjunction with the face feature of face, region growing and morphology operations accurately locate eye position, finally utilize the relative distance of upper palpebra inferior to calculate PERCLOS value and judge that whether driver is tired.But in the algorithm that said system detects at driver fatigue, undertaken judging the strong and weak face feature in conjunction with face of illumination, region growing and morphology operations location human eye by the character that optical illumination dark condition affects by the Clustering features of the colour of skin, its reliability is lower.This system uses TMS320DM642DSP master control borad, and cost is high, and its price, more than 1500 yuans, is not suitable for large-scale production.

In September, 2012, Peking University's journal (natural science edition) 719-726 page intermediate range was as medium " the vehicle-mounted embedded type driver fatigue monitor system [ J ] based on Adaboost method " delivered, system uses infrared light supply and infrared camera to obtain the video image of driver, by the face of Face datection location driver, then human eye area is extracted, with the Adaboost algorithm based on Haar feature, human eye closure state is judged, based on the corresponding early warning mechanism of PERCLOS standard formulation, judgement is carried out and early warning to potential fatigue driving.But this system is that 19 frames are per second in DSP platform process frame per second, and cannot meet the requirement of driver fatigue monitor system real-time, the response time is long, can not detect that driver is in fatigue driving state in time.

Summary of the invention

In order to overcome the deficiency that prior art exists, the invention provides a kind of driver fatigue monitor system, this system can not only meet the requirement of driver fatigue monitor system real-time, response time is shorter, can detect that driver is in fatigue driving state in time, and the reliability detected is high, low cost of manufacture.

In order to achieve the above object, the present invention adopts following technical solution:

The invention provides a kind of driver fatigue monitor system, comprise the processor 1 of embedded X C7Z020, infrared camera 2, SD card 3, display 4, wherein,

The processor 1 of described embedded X C7Z020, for carrying out fatigue detecting to driver, judges whether driver is in fatigue driving state, if be in fatigue driving state, then sends out warning message;

Described infrared camera 2 is for gathering driver's face-image;

Described SD card 3, for depositing Embedded linux system and FPGA configuration file, directly loads start up system during system initialization;

Described display 4 is for showing driver's face in real time and processing detection time and warning message;

The processor 1 of described embedded X C7Z020 comprises image capture module 11, image pre-processing module 12, image processing module 13, fatigue warning module 14;

Driver's face-image that image capture module 11 catches for obtaining described infrared camera;

Image pre-processing module 12 carries out gradation conversion after the driver got face-image is reduced in size half, obtains gray-scale map, then carries out integrogram computing to gray-scale map, obtains integrogram;

Image processing module 13 is for the calculating carrying out Haar eigenwert through pretreated gray-scale map, by Adaboost algorithm successively locating human face position, position of human eye, the method of support vector machine is adopted to detect the closure state of driver's eyes in current slot, judge whether driver is in fatigue driving state according to PERCLOS standard, to fatigue warning module 14 transmission information;

The information that fatigue warning module 14 is transmitted for receiving image processing module 13, if the information of transmission is in fatigue driving state for driver, then gives the alarm, and reminds human pilot, and sends current face's image and warning message to display 4; Otherwise send current face's image to display 4 equally, do not send warning message.

As preferred a kind of scheme: described image processing module 13 comprises eye recognition unit 131, driver fatigue detecting unit 132, wherein,

Described eye recognition unit 131, for the computing carrying out Haar eigenwert through pretreated gray level image, by Adaboost algorithm to the location completing human eye, uses the method for support vector machine to detect the closure state of driver's eyes;

Described driver fatigue detecting unit 132 is for judging whether driver is in fatigue driving state, according to PERCLOS standard, the frame number calculating driver's eyes closed in setting-up time section accounts for the ratio of the totalframes in this period, if the ratio of the totalframes that the frame number that driver's eyes closes accounted in this period is greater than 0.68, then judge that driver is in fatigue driving state, otherwise be abnormal driving state, and this information is sent to fatigue warning module 14.

A kind of fatigue driving method for early warning of the present invention, the method adopts driver fatigue monitor system to carry out fatigue driving early warning, the steps include:

(1). first use the face-image of the infrared camera collection driver with infrared light supply, pre-service is carried out to reduce the quantity that the later stage needs pixel to be processed to the image captured, accelerates the speed of later image scanning;

(2). then detect driver's position of human eye by the Adaboost method based on Haar feature, judge driver's eyes closure state by the method for support vector machine;

(3). then, the frequency of wink of the eyes in a period of time is observed by PERCLOS method, judge whether driver is in fatigue driving state according to frequency of wink, if detect that driver is in fatigue driving state, system will give the alarm to remind human pilot, and display shows current face's image in real time and processes detection time, warning information; Otherwise display shows current face's image in real time and processes detection time, does not send warning message;

(4). finally under based on the embedded X C7Z020 processor of ARM, realize detection algorithm, specific as follows:

(4-1). gather driver's face-image by infrared camera, after being reduced in size half, unification is converted to gray-scale map, and carries out the computing of integrogram;

(4-2). eye recognition: first, the pretreated image that step (4-1) obtains is carried out the extraction of Haar feature, then Haar eigenwert is sent into the face location having used and located driver in self-built infrared face storehouse and the trained Adaboost cascade classifier of MIT face database;

(4-3). the image of driver's face location step (4-2) obtained uses the mode detecting double eye to carry out the extraction of Haar feature, similarly use trained Adaboost cascade classifier to navigate to the position of human eye, then use the method for support vector machine to judge the closure state of human eye;

(4-4). the threshold value of setting fatigue driving, calculate the PERCLOS value in a period of time, the PERCLOS value calculated compares with the threshold value of setting fatigue driving, if the PERCLOS value in this period is greater than the threshold value of setting fatigue driving, then judge the fatigue driving of driver, send a warning, otherwise be abnormal driving state, do not send a warning.

The beneficial effect that a kind of driver fatigue monitor system of the present invention has mainly contains: the present invention is when detecting driver fatigue, and average frame per second reaches 25 frames/second, accelerates the response time of system alarm, can early warning in time; Use Adaboost cascade classifier and support vector machine, adopt the mode detecting double eye to carry out Haar feature extraction location, its verification and measurement ratio is higher and bit error rate is less, low cost of manufacture.

Accompanying drawing explanation

Fig. 1 is the total frame diagram of embodiment of the present invention system

Fig. 2 is the fatigue driving detection algorithm outline flowchart of the embodiment of the present invention

Fig. 3 is the process flow diagram of the eye recognition part of the embodiment of the present invention

Fig. 4 is the PERCLOS measuring principle figure of the embodiment of the present invention.

Embodiment

Below in conjunction with accompanying drawing, the invention will be further described.

With reference to Fig. 1 ~ Fig. 4, a kind of driver fatigue monitor system, comprise: embedded X C7Z020 processor 1, infrared camera 2, SD card 3, display 4, the processor 1 of described embedded X C7Z020 is for carrying out fatigue detecting to driver, judge whether driver is in fatigue driving state, if be in fatigue driving state, then send out warning message; Described infrared camera 2 is for gathering driver's face-image; Described SD card 3, for depositing embedded system and FPGA configuration file, when system initialization, directly loads start up system; Described display 3 is for showing driver's face, process detection time and warning information in real time.

Fig. 1 is the total frame diagram of system, and wherein software section realizes by C language, and whole hardware components contains embedded X C7Z020 processor 1, infrared camera 2, SD card 3, display 4 four parts.

Hardware components:

(1) image capture module 11: receive camera shooting by USB interface, obtain driver's face image, achieve the function of image acquisition.

(2) image and processing module 12 and image processing module 13 adopt the XC7Z020 processor of XILINX company, coordinate the Installed System Memory of DDR3, described embedded X C7Z020 processor 1 comprises ARM-cotexA9 kernel and artix7 type FPGA, the linux operating system of matching embedded type forms an image processing system, realizes real-time analysis and the process of image.The algorithm of key step human eye detection of the present invention and analysis of fatigue part realizes in this part.

(3) fatigue warning module 14: if detect that driver is in fatigue driving state, uses embedded X C7Z020 processor 1 to control buzzer warning, and transmits facial image and warning message by FPCA to display screen 4.

(4) infrared camera 2: the infrared light photograph of wavelength 850mm, is connected with processor by USB interface.

(5) SD card 3:SD high speed storing card, deposits embedded system and FPGA configuration file, and system directly loads from SD card and starts.

(6) display 4: adopt the AD7511 display chip driven with FPGA, FPGA connection handling device and AD7511, display chip realizes image display.

Software section:

Fig. 2 illustrates the outline flowchart of fatigue driving detection algorithm:

In the fatigue driving of driver is detected, pre-service is the first step, the image difference opposite sex obtained under different photoenvironments due to infrared camera is larger, and the noise that more or less can exist in image in various degree, therefore, in order to ensure good detection and Identification effect, the pre-service of image must be carried out;

First in order to reduce the feature quantity that later image is extracted, accelerate the speed of image scanning, the image of 640*480 image acquisition obtained is reduced in size half, is then converted into gray-scale map according to following formula by unified for image:

Gray=0.3R+0.59G+0.11B

Wherein, R represents red component, and G represents green component, and B represents blue component.

In order to the Haar eigenwert of selected sample in image can be obtained, gray-scale map is carried out to the computing of integrogram, the every bit of integrogram the gray-scale value of rectangular area obtained by following formula:

Wherein, represent point the gray-scale value at place.

After the every bit traversal in gray-scale map, obtained the gray-scale value of arbitrary rectangular area in gray-scale map by following formula:

Wherein, I(x, y) be every bit (x, the y) upper left side of gray-scale map gray-scale value sum a little.

Second step, the whole human eye detection flow process of human eye detection as shown in Figure 3.By the extraction carrying out Haar feature through pretreated image obtained in step one, then the Haar eigenwert obtained is inputed to use in self-built infrared face storehouse and the trained face Adaboost sorter of MIT face database and judge, sample is after the classification of Adaboost sorter, if when face successfully being detected, return sample effective marker, then according to the facial image coordinate setting face location returned, otherwise return step one.

After quick position face location, enter the human eye localization process stage, detect above-mentioned the extraction that the sample image of face carries out Haar feature.Need owing to adopting the algorithm of Haar feature to scan view picture image to be detected, image is less or scan box is larger, so institute is consuming time also shorter, so the present invention adopts the mode detecting double eye to carry out the extraction of Haar feature, then the Haar eigenwert obtained is input in Adaboost sorter and detects sample, if human eye detected, then return effective marker, system location human eye; Otherwise return step one.

After obtaining above-mentioned eye image, choose the threshold value of 0.5, binary conversion treatment is carried out to people's eye portion of image.Then using the pixel of binaryzation as sample characteristics, after using support vector machine training, then the closure state of human eye to be classified.In support vector machine, use Nonlinear Classifier, choose Radial basis kernel function , its kernel function expression formula is:

Wherein, ,

Wherein, for Lagrange multiplier, for the mark of training sample, for threshold value, exp is the function of e , for input amendment, for radial parameter center, for width parameter, =0.01;

3rd step, fatigue detecting definition driver its eyes closed degree time scale be more than or equal to shared by 80% within the time period is fatigue driving, namely, driver's eyes is adopted to close the P80 standard of degree PERCLOS, judge whether driver is in fatigue driving state, its formula detecting fatigue driving is:

As shown in Figure 4, in the time N of sampling, when the eye closing frame number detected exceedes 80% of totalframes, be judged as fatigue state.And when driver is in fatigue state, during nictation, the time of eyes closed all more than 1 second, therefore can be set to 1 second T detection time.Experimental result of the present invention shows, if frame number of closing one's eyes in 1 second is more than 17 frames, PERCLOS value is greater than 0.68, judges that driver is in fatigue driving state.

Claims (2)

1. a driver fatigue monitor system, is characterized in that, this system comprises processor (1), infrared camera (2), SD card (3), the display (4) of embedded X C7Z020, wherein,
The processor (1) of described embedded X C7Z020, for carrying out fatigue detecting to driver, judges whether driver is in fatigue driving state, if be in fatigue driving state, then sends out warning message;
Described infrared camera (2) is for gathering driver's face-image;
Described SD card (3), for depositing Embedded linux system and FPGA configuration file, directly loads start up system during system initialization;
Described display (4) is for showing driver's face in real time and processing detection time and warning message;
The processor (1) of described embedded X C7Z020 comprises image capture module (11), image pre-processing module (12), image processing module (13), fatigue warning module (14);
Driver's face-image that image capture module (11) catches for obtaining described infrared camera;
Image pre-processing module (12) carries out gradation conversion after the driver got face-image is reduced in size half, obtains gray-scale map, then carries out integrogram computing to gray-scale map, obtains integrogram;
Image processing module (13) is for the calculating carrying out Haar eigenwert through pretreated gray-scale map, by Adaboost algorithm successively locating human face position, position of human eye, the method of support vector machine is adopted to detect the closure state of driver's eyes in current slot, judge whether driver is in fatigue driving state according to PERCLOS standard, to fatigue warning module (14) transmission information;
The information that fatigue warning module (14) is transmitted for receiving image processing module (13), if the information of transmission is in fatigue driving state for driver, then give the alarm, remind human pilot, and send current face's image and warning message to display (4); Otherwise send current face's image to display (4) equally, do not send warning message.
2. according to a kind of driver fatigue monitor system described in claim 1, it is characterized in that: described image processing module (13) comprises eye recognition unit 131, driver fatigue detecting unit 132, wherein,
Described eye recognition unit 131, for the computing carrying out Haar eigenwert through pretreated gray level image, by Adaboost algorithm to the location completing human eye, uses the method for support vector machine to detect the closure state of driver's eyes;
Described driver fatigue detecting unit 132 is for judging whether driver is in fatigue driving state, according to PERCLOS standard, the frame number calculating driver's eyes closed in setting-up time section accounts for the ratio of the totalframes in this period, if the ratio of the totalframes that the frame number that driver's eyes closes accounted in this period is greater than 0.68, then judge that driver is in fatigue driving state, otherwise be abnormal driving state, and this information is sent to fatigue warning module (14).
CN201510335822.XA 2015-06-17 2015-06-17 Fatigue driving early warning system CN104881956A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510335822.XA CN104881956A (en) 2015-06-17 2015-06-17 Fatigue driving early warning system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510335822.XA CN104881956A (en) 2015-06-17 2015-06-17 Fatigue driving early warning system

Publications (1)

Publication Number Publication Date
CN104881956A true CN104881956A (en) 2015-09-02

Family

ID=53949433

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510335822.XA CN104881956A (en) 2015-06-17 2015-06-17 Fatigue driving early warning system

Country Status (1)

Country Link
CN (1) CN104881956A (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101599207A (en) * 2009-05-06 2009-12-09 深圳市汉华安道科技有限责任公司 Fatigue driving detection device and automobile
CN102085099A (en) * 2011-02-11 2011-06-08 北京中星微电子有限公司 Method and device for detecting fatigue driving
CN104240446A (en) * 2014-09-26 2014-12-24 长春工业大学 Fatigue driving warning system on basis of human face recognition
US20150042477A1 (en) * 2009-03-30 2015-02-12 Tobii Technology Ab Eye closure detection using structured illumination

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150042477A1 (en) * 2009-03-30 2015-02-12 Tobii Technology Ab Eye closure detection using structured illumination
CN101599207A (en) * 2009-05-06 2009-12-09 深圳市汉华安道科技有限责任公司 Fatigue driving detection device and automobile
CN102085099A (en) * 2011-02-11 2011-06-08 北京中星微电子有限公司 Method and device for detecting fatigue driving
CN104240446A (en) * 2014-09-26 2014-12-24 长春工业大学 Fatigue driving warning system on basis of human face recognition

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
程中如等: "基于Adaboost方法的车载嵌入式疲劳驾驶预警系统", 《北京大学学报(自然科学版)》 *

Similar Documents

Publication Publication Date Title
US8744642B2 (en) Driver identification based on face data
Horng et al. Driver fatigue detection based on eye tracking and dynamic template matching
Guo et al. Pedestrian detection for intelligent transportation systems combining AdaBoost algorithm and support vector machine
CN104200192B (en) Driver's gaze detection system
US20060204042A1 (en) System and method for determining eye closure state
EP1732028B1 (en) System and method for detecting an eye
CN101593425B (en) Machine vision based fatigue driving monitoring method and system
Devi et al. Driver fatigue detection based on eye tracking
Dong et al. Fatigue detection based on the distance of eyelid
Mbouna et al. Visual analysis of eye state and head pose for driver alertness monitoring
Jo et al. Vision-based method for detecting driver drowsiness and distraction in driver monitoring system
EP1589485A2 (en) Object tracking and eye state identification method
Eriksson et al. Driver fatigue: a vision-based approach to automatic diagnosis
CN101950355B (en) Method for detecting fatigue state of driver based on digital video
Lenskiy et al. Driver’s eye blinking detection using novel color and texture segmentation algorithms
CN101639894B (en) Method for detecting train driver behavior and fatigue state on line and detection system thereof
CN101032405A (en) Safe driving auxiliary device based on omnidirectional computer vision
EP1701288A1 (en) System and method of detecting eye closure based on edge lines
CN101540090A (en) Driver fatigue monitoring device based on multivariate information fusion and monitoring method thereof
Dasgupta et al. A vision-based system for monitoring the loss of attention in automotive drivers
CN101090482A (en) Driver fatigue monitoring system and method based on image process and information mixing technology
CN105654753A (en) Intelligent vehicle-mounted safe driving assistance method and system
CN102841354B (en) Vision protection implementation method of electronic equipment with display screen
CN103400471B (en) A kind of driver tired driving detection system and detection method
CN103150560B (en) The implementation method that a kind of automobile intelligent safety is driven

Legal Events

Date Code Title Description
PB01 Publication
C06 Publication
SE01 Entry into force of request for substantive examination
EXSB Decision made by sipo to initiate substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20150902

WD01 Invention patent application deemed withdrawn after publication