CN101599207A - A kind of fatigue driving detection device and automobile - Google Patents

A kind of fatigue driving detection device and automobile Download PDF

Info

Publication number
CN101599207A
CN101599207A CNA200910107179XA CN200910107179A CN101599207A CN 101599207 A CN101599207 A CN 101599207A CN A200910107179X A CNA200910107179X A CN A200910107179XA CN 200910107179 A CN200910107179 A CN 200910107179A CN 101599207 A CN101599207 A CN 101599207A
Authority
CN
China
Prior art keywords
image
human eye
module
face
fatigue
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CNA200910107179XA
Other languages
Chinese (zh)
Inventor
吴泽俊
程如中
赵勇
卿云立
王强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Safdao Technology Corp Ltd
Original Assignee
Shenzhen Safdao Technology Corp Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Safdao Technology Corp Ltd filed Critical Shenzhen Safdao Technology Corp Ltd
Priority to CNA200910107179XA priority Critical patent/CN101599207A/en
Publication of CN101599207A publication Critical patent/CN101599207A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Image Processing (AREA)

Abstract

The invention discloses a kind of fatigue driving detection device and automobile, comprise image capture module, image processing module and alarm module, described image processing module receives the vision signal of image capture module output, face characteristic sorter location human face region according to training in advance, human eye feature sorter location human eye area according to training in advance, in human eye area, obtain iris image, analyze the eyes closed state based on iris image, eyes closed state and fatigue criteria are compared, and the output alarm control command is given described alarm module when judging that the driver is in fatigue state, and described alarm module response controlling alarm order is also reported to the police.The present invention has reduced the influence of individuality difference to testing result, has improved the accuracy of tired judgement, has better practicability.Present embodiment adopts embedded system simultaneously, and volume is little, and is easy to use.

Description

A kind of fatigue driving detection device and automobile
[technical field]
The present invention relates to the active safety technical field of vehicle, relate in particular to the detection technique field of fatigue driving.
[background technology]
Fatigue driving is one of important hidden danger of current traffic safety.When the driver is in fatigue state, in various degree decline is all arranged, traffic hazard takes place easily to the perception of surrounding environment, situation judgement with to the ability of controlling of vehicle.Therefore, research and develop high performance driver fatigue state and detect in real time and early warning technology, can effectively reduce the hidden danger that fatigue driving brings, ensure driver's personal safety and peripheral related personnel's purpose of safety thereby reach.
The driver fatigue detection system is meant, by Che Nei and the outer optional equipment of car information such as driver's physiological signal, driver's physiological reaction feature, driver's operation behavior or vehicle-state are sampled, can single a kind of information be assessed also and can use reliable tired model to judge whether the driver is in fatigue state at last to multiple parametric synthesis assessment.If the driver is in fatigue driving state, system sends alarm message reminding driver precarious position or directly by the controlled vehicle-mounted electrical interface vehicle is controlled, thereby reduces owing to fatigue driving produces the traffic hazard incidence.
At present, fatigue driving detection device and correlation technique are main according to following three kinds of parameter indexs:
1 physical signs: the cycle that breathing pulse, brain wave, frequency of wink, pupil are stared, the grip of hand;
2 morphological indexs: body posture, head position, face shape, eye folding condition;
3 vehicle indexs: the automobile operating parameter of reflection driver state.
Various system and methods all are to realize information acquisition by setting up related sensor with the above-mentioned parameter index.In all parameter indexs, brain wave is the most stable, accurate to the judgement of fatigue state.Because will be at the lead of head-mount acquired signal, from being promoted and being that this mode is unactual on the popular angle of being accepted.So under the prerequisite of accuracy and stability, " practicality ", " ease for use " are to consider a whether key criterion of success of fatigue driving detection device.
[summary of the invention]
The main technical problem to be solved in the present invention is, a kind of fatigue driving detection device is provided, and this device can be eliminated individuality difference, has improved the accuracy of tired judgement, and is practical, and volume is little and easy to use.
According to an aspect of the present invention, a kind of fatigue driving detection device is provided, comprise image capture module, image processing module and alarm module, described image processing module receives the vision signal of image capture module output, face characteristic sorter location human face region according to training in advance, human eye feature sorter location human eye area according to training in advance, in human eye area, obtain iris image, analyze the eyes closed state based on iris image, eyes closed state and fatigue criteria are compared, judge whether the driver is in fatigue state, and the output alarm control command is given described alarm module when judging that the driver is in fatigue state, and described alarm module response controlling alarm order is also reported to the police.
According to a further aspect in the invention, provide a kind of automobile, comprise above-mentioned fatigue driving detection device.
The present invention detects people's face and human eye respectively by adopting people's face sorter and human eye sorter, reduced of the influence of individuality difference to testing result, reduced the influence of illumination and human face posture to testing result, the accuracy of driver's eye location and the accuracy that eyes closed is judged have been improved, thereby improved the accuracy of tired judgement, had better practicability.Present embodiment adopts embedded system simultaneously, and volume is little, and is easy to use.
[description of drawings]
Fig. 1 is the structural representation of an embodiment of the present invention;
The human eye locating effect synoptic diagram of Fig. 2 for showing;
Fig. 3 is the process flow diagram of the another kind of embodiment of the present invention;
Fig. 4 is the structural representation of a kind of instantiation of the present invention.
[embodiment]
In conjunction with the accompanying drawings the present invention is described in further detail below by embodiment.
Please refer to Fig. 1, fatigue driving detection device comprises image capture module 10, image processing module 20 and alarm module 30.Image processing module 20 receives the vision signal of image capture module 10 outputs, face characteristic sorter location human face region according to training in advance, human eye feature sorter location human eye area according to training in advance, in human eye area, obtain iris image, analyze the eyes closed state based on iris image, eyes closed state and fatigue criteria are compared, judge whether the driver is in fatigue state, and the output alarm control command is given described alarm module 30 when judging that the driver is in fatigue state, and described alarm module 30 response controlling alarm orders are also reported to the police.
Wherein, image capture module 10 generally includes camera, for example CCD infrared camera or CMOS camera.
In an example of present embodiment, image processing module 20 comprises codec 21 (for example audio/video encoding/decoding chip) and microprocessor 23 (for example DSP microprocessor), the input end of codec 21 is connected with the output terminal of image capture module 10, and the output terminal of codec 21 is connected with microprocessor 23.Microprocessor 23 is used for image is handled and judged whether the driver is in fatigue state, output alarm control command when judging that the driver is in fatigue state, and alarm module 30 response controlling alarm orders are reported to the police accordingly.
Codec 21 receives the analog video signal of image capture module 10 outputs, signal is carried out A/D conversion and decoding, the image that also needs in some cases to gather carries out the output format conversion, for example converts the UYVY format-pattern that obtains to yuv format, to meet the requirement of database format.
In a kind of example of present embodiment, microprocessor 23 receives the view data of codec 21 outputs, image is carried out gray processing and normalized, determine that people's face detects area-of-interest, during first two field picture after the image of gathering is the camera initialization, seeker's face and locate human face region on full images, during image after the image of gathering is first frame, detect area-of-interest as people's face after extracting the human face region of former frame image and setting the expansion of numerical value, and the face characteristic that writes down in the face characteristic sorter according to training in advance analyzes the image that described people's face detects area-of-interest, thus the location human face region.
When the face of people from location, locate human face region according to the face characteristic sorter of training in advance.Be producer's face tagsort device, adopt the AdaBoost algorithm that a large amount of people's face samples is detected, the AdaBoost algorithm uses a kind of training algorithm based on the Harr-like feature to train face characteristic sorter file (haarcascade_faces.xml), the training on a large amount of people's face sample data bases of this sorter is come out, and has stronger universality.After reading every frame video image, come analysis image information according to the people's face Harr feature that is write down in the face characteristic sorter file, and then adopt AdaBoost algorithm and relevant people face tagsort device that image is carried out pattern-recognition, the human face region in the image is demarcated.
Adopt image gray processing and histogram normalization that image is carried out pre-service, provide reliable image information for people's face detects.Gray processing can transform into the single channel gray level image by the triple channel image of YUV with original image; The normalized purpose of histogram is to strengthen the gray level image brightness contrast, by histogram transformation the gray scale spacing of image is drawn back, thereby increases contrast, makes that image detail is clear, feature outstanding, reduces the unequal and interference that causes of brightness simultaneously.
Determine earlier that before the human face region of location people's face detects area-of-interest, only seeker's face in more among a small circle reduces the Flame Image Process area, thereby reduces data processing amount.
During first two field picture after the image of gathering is the camera initialization, seeker's face on full images, and according to face characteristic sorter location human face region, during image after the image of gathering is first frame, determine that based on the human face region of former frame image people's face detects area-of-interest, for example the human face region of former frame image is outwards detected area-of-interest as people's face after 1 centimetre of the expansion.
In a further embodiment, microprocessor 23 also detects people's face area-of-interest before the human face region of location image dwindles according to preset proportion, further reduces data processing amount.
For improving the detection speed of human eye, microprocessor is also determined the human eye detection area-of-interest in human face region, the image of the human eye feature analysis human eye detection area-of-interest that writes down in the human eye feature sorter according to training in advance then, thereby location human eye area.
Can adopt multiple scheme to determine the human eye detection area-of-interest.
The purpose of human eye detection is accurately to navigate to human eye area, the influence of as far as possible removing eyebrow and hair.Human eye detection adopts with people's face and detects the same method.For making human eye tagsort device, adopt the AdaBoost algorithm that a large amount of human eye samples is detected, use trains human eye feature sorter file based on the training algorithm of Harr-like feature, and the training on a large amount of human eye sample data bases of this sorter is come out, and has stronger universality.
Microprocessor 23 carries out the brightness enhancing with the image of human eye area after obtaining human eye area, and adopts the binary conversion treatment of adaptive threshold to obtain iris image.
In a kind of example of present embodiment, alarm module 30 comprises lamp (for example LED lamp), when microprocessor 23 judges that the driver is in fatigue state, control warning module adopts the light mode to report to the police, and in such cases, microprocessor 23 is electrically connected with alarm module 30, when needs are reported to the police, microprocessor 23 is exported control signals to alarm module 30, or exports control signals by GPIO (being general input and output) to alarm module 30, and control LED lamp is lighted or glimmered.
In another kind of example, alarm module 30 comprises loudspeaker, when microprocessor 23 judges that the driver is in fatigue state, control warning module adopts voice mode to report to the police, and in such cases, codec 21 is electrically connected with alarm module 30, when needs are reported to the police, microprocessor 23 outputs to codec 21 with voice signal, outputs to alarm module 30 after codec 21 is handled, and voice signal is amplified the rear drive loudspeaker sound.
Adopt AdaBoost detection algorithm and face characteristic sorter, human eye feature sorter to locate people's face and human eye respectively in the present embodiment, has stronger universality, reduced of the influence of individuality difference to testing result, it is less that while AdaBoost algorithm is influenced by illumination and human face posture, accuracy in detection is higher, therefore improve the accuracy of people's face and human eye location, thereby also improved the accuracy that driver's fatigue state is judged, had very strong practicality.Present embodiment adopts embedded system simultaneously, and volume is little, and is easy to use.
Also adopt the scheme of determining people's face and human eye detection area-of-interest earlier to reduce the treatment capacity of view data in the present embodiment, the processing time of every two field picture is reduced, the number of image frames of handling in unit interval increases, and helps the industrialization of fatigue driving monitoring technology.
In another embodiment, as shown in Figure 1, fatigue driving detection device also comprises display module 40, described microprocessor 23 is connected with display module 40, display module 40 can be LCD (LCD), microprocessor 23 is exported to display module 40 with video image and is shown, while detected human eye area of mark on video image, i.e. and the image of display module 40 demonstrations has the human eye area telltale mark.Also can further show detected human face region, as shown in Figure 2, the detected human face region of square frame 1 expression, square frame 2 is a human eye zone location mark, represents detected human eye area, people's face or human eye area telltale mark also can adopt other mark in addition.This human eye area telltale mark is the position of microprocessor 23 according to the detected human eye area of setting of algorithm, might not be consistent with the human eye area in the image of reality, therefore when the driver is with the camera initialization after, see when human eye area in the image of human eye area telltale mark and reality is inconsistent, can adjust the position or the direction of camera, so that camera is aimed at people's face.
In another embodiment, at the detection scheme at night, image capture module comprises infrared camera, infrared camera is used to be arranged on operator seat the place ahead, take driver's face image, image capture module also comprises the infrared light supply that is distributed in described camera both sides, and described camera is provided with infrared filter.The infrared filter of the specific wavelength that is provided with on the camera is realized the stability of different light rays condition hypograph, and present embodiment adopts the 940nm infrared light supply and passes through the mode of 940nm filter plate filtering, realizes the stability of image in daytime.
In some embodiments, fatigue driving detection device is memory module 50 also, and as shown in Figure 1, memory module 50 is connected with microprocessor 23, is specifically designed to the image/video information in nearest a period of time of storage, the image of storage can be read to be presented on the display module.
Be illustrated in figure 3 as the processing flow chart of a kind of specific embodiment of the present invention, may further comprise the steps:
Step 301, the initialization camera.
Step 302, initialization face characteristic classifier parameters, initialisation image positional information.
Step 303 is obtained image by the camera that is arranged on driver the place ahead, and with the picture format of image transitions for being fit to, if for example the image of Pai Sheing is the UYVY format-pattern, then converts yuv format to.
Step 304 determines that according to the testing result of previous frame people's face detects area-of-interest.
Step 305 is single channel gray level image and normalized with the YUV image transitions, then image is dwindled according to preset proportion, for example image is narrowed down to 1/2 of original image.Image is dwindled the further data processing amount that reduced.
Step 306 adopts AdaBoost detection algorithm and face characteristic sorter location human face region, and execution in step 307 when detecting people's face, execution in step 308 when detecting less than people's face.
Step 307 is determined the human eye detection area-of-interest according to human face region.
The location of human face region provides base image can for the location of human eye, can also verify the tram of human eye area simultaneously.After determining people's face position, can save the expense of directly on entire image, seeking human eye greatly, reduce algorithm execution time, improve efficiency of algorithm in location human eye area on the human face region.
Adopt the simple algorithm can very fast definite human eye detection area-of-interest, detect human eye then in less human eye detection area-of-interest, this makes the accuracy and the efficient that detect human eye higher.
Execution in step 309 behind definite human eye detection area-of-interest.
Step 308, the initialisation image location parameter is from full images search human face region.
Step 309, the human eye Haar feature that writes down in the use human eye tagsort device detects the human eye detection area-of-interest and locatees, thereby obtains the human eye area of mating most.
By the AdaBoost algorithm, after reading every frame video image, subrange in the selected human face region is as the human eye detection area-of-interest, in the human eye detection area-of-interest, carry out the image information analysis according to the human eye Harr feature that is write down in the human eye feature sorter file, thereby accurately locate human eye area.
If successfully detected human eye area at the human eye detection area-of-interest, then execution in step 311.
When present frame can't pass through human eye feature sorter matching detection to human eye area, then execution in step 303, detected the next frame image.When not detecting human eye if detected people's face, then execution in step 310, and the position of human eye that utilizes previous frame to determine extracts human eye area, if successfully extract human eye area, then execution in step 311, if still can not extract human eye area, then execution in step 303, detect the next frame image.
By above-mentioned steps, image processing speed is handled a two field picture from several seconds of prior art and has been brought up to and can handle 12 two field pictures in 1 second.
Step 311 is with the brightness enhancing of eyes image, outstanding human eye iris region.
Step 312, image Gauss smoothly removes noise.Gauss's denoising is noise and the interference that exists for removal of images, and level and smooth profile.
Step 313, the binary conversion treatment of adaptive threshold.When vehicle is in the environment with dark light, infrared camera may be taken less than clearly people's face or human eye, adopting two value transforms of adaptive threshold is in order to obtain the information of human eye iris, for example adopt the OTSU image segmentation of adaptive threshold that eye image is handled, dynamically determine the threshold value of bianry image, utilize the gray scale of eye image obviously to be different from the feature (particularly under the infrared light radiation situation) of other people face position, make in the eye image gray scale more than or equal to the part display white of threshold value, gray scale shows black less than the part of threshold value in the eye image, pass through rim detection, adaptive binary Images Processing obtains the human eye fitted ellipse, thereby elimination eyebrow, pouch, the canthus is for the interference of image, obtain the bianry image and the marginal information thereof of iris, comprise the palpebra inferior point, iris center etc.
The binary conversion treatment of adaptive threshold can change by self-adaptation light, as particular surroundingss such as night, tunnels, therefore can access the clearly more demarcated iris image of profile through after the binary conversion treatment of adaptive threshold.
Step 314 finds the bianry image largest contours, does last preparation for detecting human eye state.In this step, can also negate by image (and white become black, black becomes white) give prominence to the profile of iris.
By above step, can more exactly driver's eye information be extracted.The adaptability of this scheme is stronger, cooperates the infrared light supply of infrared camera the right and left, can obtain human eye state information under high light, environment such as dim, thus the requirement of self-adaptation varying environment.
Step 315 is extracted the iris profile and is judged the eyes closed state according to improved PERCLOS.According to the iris profile of human eye, the last palpebra inferior point that calculates eyes reaches the distance that goes up between palpebra inferior, analyzes the eyes closed state, comprises closed degree and closure time etc., and each parameter value and fatigue criteria are compared.
In the prior art, fatigue criteria adopts the PERCLOS standard usually.The PERCLOS standard is to judge object with the pupil, because it is bigger that the imaging effect of pupil is influenced by camera resolution, and under the infrared light radiation situation, be easy to generate noise around the pupil, fatigue criteria adopts improved PERCLOS standard in the present embodiment, promptly with the iris is to judge object, and the zone that iris is covered by last palpebra inferior is as the eyes closed zone, calculate closed degree and closure time thus, thereby judge fatigue state.
Last palpebra inferior point is meant by the central axis of the above-below direction at iris center and the intersection point of iris profile.Detect the distance that can calculate behind the palpebra inferior point between the two.By detecting, eyes are normally opened the distance between last palpebra inferior point state under as can be known, and distance between the last palpebra inferior point of the real-time detection of calculating and eyes are normally opened the ratio of the distance between the last palpebra inferior point under the state, i.e. eyes closed degree as can be known.When eyes closed degree during, think eyes closed less than setting threshold.For example, under the normal condition, the visible iris of human eye is about 80%, thinks eyes closed when the eyes closed degree is 80% (being that visible iris is 20%).Add up the T.T. of eyes closed in a period of time and/or driving the once middle shared time of eyes closed of nictation.
The fatigue criteria of present embodiment is for to compare the critical value of eyes closed number percent f and setting, as eyes closed number percent f during greater than critical value, thinks that the driver is in fatigue state.Eyes closed number percent f accounts for the number percent that the eyes closed degree is less than or equal to time of 20% for eyes closed degree in nictation once the process more than or equal to time of 80%, that is:
f = t 3 - t 2 t 4 - t 1 × 100 %
Wherein, t1, t4 are that eyes open is the time point of 80% (be that visible iris is 80%, the eyes closed degree is 20%), and t2, t3 are that eyes open is 20% time point.F then represents the driver more near fatigue state more near 1.If selected 0.8 for judging whether critical value of driver fatigue, promptly eyes closed number percent f thinks that greater than 0.8 o'clock the driver is in fatigue state.
According to above-mentioned fatigue criteria, eyes are closures or open in each two field picture if detect, then only need to detect the distance that goes up between the palpebra inferior point and eyes and whether normally open the ratio of the distance between the last palpebra inferior point under the state more than or equal to 80%, if, judge that then eyes are in the state of opening in this two field picture, otherwise judge that eyes are in closure state in this two field picture.In the unit interval, if the consecutive image frame number of eyes closed, judges then that the driver is in fatigue state at this moment more than or equal to 80%.For example, if calculate according to per second 10 frames, if the visible iris of continuous 8 frames be 20% or below, then the driver is in fatigue state.
If judge that the driver is in fatigue state, then execution in step 316, send alerting signal, for example remind the driver by modes such as sound or light flash.
In the foregoing description, the technical scheme of determining people's face detection area-of-interest and human eye detection area-of-interest before detecting people's face and human eye earlier adopts people's face sorter and human eye sorter to detect among the embodiment of people's face and human eye except can be applicable to, and can also be applied in by alternate manner and detect among the embodiment of people's face and human eye.
Be illustrated in figure 4 as the structure that realizes a kind of concrete device of the present invention, this device comprises image capture module, image processing module, memory module, alarm module and output module.Below each module is described in detail.
1, image processing module
Microprocessor in the image processing module comprises DSP embedded (digital signal processor) and DDR2, NOR Flash (NOR type flash memory) and clock, reset circuit etc.DSP not only will handle a large amount of view data as Main Processor Unit, carries out intelligent decision in real time, also will doublely do MCU work, control peripheral each chip.DDR2 is an internal memory, and NOR Flash is a program storage.Each system power-up or when resetting, DSP carries out bios program in the inner ROM and the basic element of character and register are carried out initialization then the program in the program storage is called in DDR2, begin to carry out with regard to the start address that jumps to the last program of DDR2 after having transferred, finish the BootLoader process of system, the level state of the relevant pin that system can be by being provided with DSP is selected different BootMode.
Codec in the image processing module comprises video and audio coding and SDRAM.Video and audio coding mainly is to be responsible for video decode work, can support the video acquisition of multichannel simultaneously, and it can also do certain Video processing operation to image, as PIP, POP etc., also can have the audio coding decoding function concurrently, is responsible for the collection and the output of sound.SDRAM is as image or sound buffer memory in these processes, handles operation to support powerful lot of data.In the audio frequency input process, microphone is responsible for the collection of sound.In the output procedure, need a power amplifier to come smaller voice signal is amplified, reach enough power to drive the bigger load loudspeaker of back.
For example video and audio coding can adopt the audio/video encoding/decoding chip.Adopt multi-channel video codec chip framework, in car the collection of driver's image information, can gather the video information in orientation such as the inside and outside or front and back of multichannel car simultaneously.For example can adopt TW2835 audio/video decoding chip, but be not limited to this kind chip.TW2835 can gather 4 tunnel video informations at most.
2, memory module
In system, except internal memory DDR2, program storage NOR Flash and view data buffer unit SDRAM, also added a memory module to realize the storage of compressed image, be specifically designed to the storage image/video information of nearest a few minutes, also can be used for depositing number voice information, and on PC, realize reading of storage data by 485 serial ports.Memory module includes but not limited to NAND type flash memory, hard disk or SD card.
3, display module
Mainly be made up of LCD, because the LCD inside of choosing has the LCD driving circuit, do not need to write display driving software, the simulating signal that the image that only needs to show becomes forms such as PAL/NTSC is defeated by LCD and can be carried out image and show.
4, alarm module
The part of reporting to the police mainly is audible alarm unit and light warning unit, and the audible alarm unit comprises power amplifier and loudspeaker (or loudspeaker).The light warning unit mainly comprises light emitting diode (LED).Will make the loudspeaker sounding if microprocessor judges goes out driver fatigue, LED is lighted or glimmer.In addition, the sound of warning is to select, and can send the chimes of doom of different stage according to the difference of degree of fatigue.
5, image capture module
Image capture module is used to take the camera of driver's face image except that comprising, can also comprise a plurality of outer cameras of car that are arranged on, and can link to each other with image processing module by the video input/output interface.
6, interface
Can comprise a plurality of interfaces, except that the video IO interface, also can comprise other interface, the CAN-BUS interface that for example debug the J-TAG interface, connect 485 serial ports of PC, communicates with vehicle-mounted ECU, GPIO interface etc. that can external alarm.Wherein, JATG interface and CAN bus interface and RS485 bus serial ports belong to internal interface.
1) JATG interface
DSP is debugged by jtag interface at the debug phase PC.The present invention adopts the jtag interface circuit on the development board, is convenient to come debug system with original development environment.
2) CAN bus and RS485 bus.
The CAN bus be used for automobile on other electronic unit interface of carrying out communication, it is the more common interfaces of automotive electronics parts (ECU).RS485 is used for and computer or other spare interfaces that has the equipment of interface to carry out communication.
3) exterior I/O interface
Pass through I 2The I of C bus extended chip PCF8574APWRG4 and DSP 2The C bus links to each other, thereby has expanded exterior I/O parallel interface of one 8, can communicate by letter with the LPT device of outside very easily, and this is that spare interface as system uses.System also can directly draw 6 tunnel outside IO interface by pull-up resistor from the GPIO pin of DSP in addition, can be used for outside input and output and interrupts input.
The operational process of present embodiment is as follows:
Camera photographs image information, the simulating signal of output PAL/NSTL form, interface is RCA, Video Decoder obtains analog picture signal from the RCA interface, and carry out A/D and change, decode, through after a series of processing, the image information of eight yuv formats of output is given dsp processor, and DSP will do three things this moment:
Image is done intellectual analysis, judge whether fatigue,, light alarm lamp simultaneously if fatigue is reported to the police to sound horn with regard to output alarm signal.When DSP did Intelligent treatment, DSP exported to the image information that the demonstration of LCD screen photographs after also will doing certain processing (as: adding a frame etc. at the eyes place) to image.This effect mainly is to be convenient to adjust camera to aim at people's face.Video segment when DSP will fatigue driving take place deposits into NAND Flash (NAND type flash memory).
Mainly be the situation of video stream above, in addition, also have audio information stream and control information flow etc.DSP deposits digital audio-frequency data among the NOR Flash in, as the voice output signal of later warning usefulness.Actual storage is the sound original waveform file that does not add specific coding and compression among the NOR FLASH.This process is after system installs, and formally is applied to finish before the intelligent decision.DSP just calls alarm module after judging driver fatigue, and the sound waveform data that soon before had been stored among the NOR Flash are passed through I 2The C bus is given the audio/video encoding/decoding chip, the audio/video encoding/decoding chip with data preparation after, the power amplifier of rear end is exported in conversion through D/A then, drives sound horn by power amplifier and reports to the police.
Control information flow is fewer, mainly is that DSP passes through I 2The C bus is controlled the audio/video encoding/decoding chip, as initialization, and configuration etc.Be exactly read-write control to each storer (as DDR2, NAND, NOR) etc. in addition.
The foregoing description can be applicable on the automobile, whether is in fatigue state by monitoring driving person, and carries out warning reminding when driver fatigue, thereby has improved the security of car steering.
Above content be in conjunction with concrete embodiment to further describing that the present invention did, can not assert that concrete enforcement of the present invention is confined to these explanations.For the general technical staff of the technical field of the invention, without departing from the inventive concept of the premise, can also make some simple deduction or replace, all should be considered as belonging to protection scope of the present invention.

Claims (10)

1. fatigue driving detection device, comprise image capture module, image processing module and alarm module, it is characterized in that, described image processing module receives the vision signal of image capture module output, face characteristic sorter location human face region according to training in advance, human eye feature sorter location human eye area according to training in advance, in human eye area, obtain iris image, analyze the eyes closed state based on iris image, eyes closed state and fatigue criteria are compared, judge whether the driver is in fatigue state, and the output alarm control command is given described alarm module when judging that the driver is in fatigue state, and described alarm module response controlling alarm order is also reported to the police.
2. fatigue driving detection device as claimed in claim 1, it is characterized in that, described image processing module comprises codec and microprocessor, the input end of described codec is connected with the output terminal of described image capture module, the output terminal of described codec is connected with described microprocessor, described microprocessor and/or codec are connected with alarm module, and described microprocessor is used for when judging that the driver is in fatigue state the photoelectric alarm signal exported to alarm module or gives alarm module by codec output sound alerting signal.
3. fatigue driving detection device as claimed in claim 2 is characterized in that, described codec receives the video analog signal of image capture module output, carries out A/D conversion and decoding.
4. as claim 2 or 3 described fatigue driving detection devices, it is characterized in that, described microprocessor receives the view data of codec output, and image carried out gray processing and normalized, determine that people's face detects area-of-interest, during first two field picture after the image of gathering is the camera initialization, seeker's face and locate human face region on full images, during image after the image of gathering is first frame, detect area-of-interest as people's face after extracting the human face region of former frame image and setting the expansion of numerical value, and the face characteristic that writes down in the face characteristic sorter according to training in advance analyzes the image that described people's face detects area-of-interest, thus the location human face region.
5. fatigue driving detection device as claimed in claim 4 is characterized in that, described microprocessor detects people's face area-of-interest before the human face region of location image dwindles according to preset proportion.
6. fatigue driving detection device as claimed in claim 4, it is characterized in that, described microprocessor is also determined the human eye detection area-of-interest in human face region, the image of the human eye feature analysis human eye detection area-of-interest that writes down in the human eye feature sorter according to training in advance then, thereby location human eye area.
7. fatigue driving detection device as claimed in claim 6 is characterized in that, described microprocessor carries out the brightness enhancing with the image of human eye area after obtaining human eye area, and adopts the binary conversion treatment of adaptive threshold to obtain iris image.
8. fatigue driving detection device as claimed in claim 6 is characterized in that, also comprises display module, and described microprocessor is connected with display module, is used for outputing to display module with having the specifically labelled vedio data of human eye area.
9. as each described fatigue driving detection device in the claim 1 to 8, it is characterized in that, described image capture module comprises the infrared light supply that is used to be arranged on the infrared camera of operator seat the place ahead, shooting driver face image and is distributed in described camera both sides, and described camera is provided with infrared filter.
10. an automobile is characterized in that comprising each described fatigue driving detection device in the claim 1 to 9.
CNA200910107179XA 2009-05-06 2009-05-06 A kind of fatigue driving detection device and automobile Pending CN101599207A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CNA200910107179XA CN101599207A (en) 2009-05-06 2009-05-06 A kind of fatigue driving detection device and automobile

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CNA200910107179XA CN101599207A (en) 2009-05-06 2009-05-06 A kind of fatigue driving detection device and automobile

Publications (1)

Publication Number Publication Date
CN101599207A true CN101599207A (en) 2009-12-09

Family

ID=41420636

Family Applications (1)

Application Number Title Priority Date Filing Date
CNA200910107179XA Pending CN101599207A (en) 2009-05-06 2009-05-06 A kind of fatigue driving detection device and automobile

Country Status (1)

Country Link
CN (1) CN101599207A (en)

Cited By (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101803928A (en) * 2010-03-05 2010-08-18 北京智安邦科技有限公司 Video-based driver fatigue detection device
CN102208125A (en) * 2010-03-30 2011-10-05 深圳市赛格导航科技股份有限公司 Fatigue driving monitoring system and method thereof
CN102289660A (en) * 2011-07-26 2011-12-21 华南理工大学 Method for detecting illegal driving behavior based on hand gesture tracking
CN102542729A (en) * 2012-01-11 2012-07-04 广西卫通汽车信息技术有限公司 Method, device and system for monitoring retinal fatigue of driver
CN102707801A (en) * 2012-05-07 2012-10-03 广东好帮手电子科技股份有限公司 Vehicle-mounted recognition control system and control method thereof
CN102785574A (en) * 2012-07-17 2012-11-21 东莞市泰斗微电子科技有限公司 Fatigue or drunk-driving detection and control method and corresponding system
US8411901B2 (en) 2009-12-29 2013-04-02 Industrial Technology Research Institute Method and system for calculating image recognition rate and embedded image processing system
CN103035140A (en) * 2012-12-24 2013-04-10 吉林大学 Automobile driver dangerous driving behavior correction and evaluation system
CN103259971A (en) * 2012-02-16 2013-08-21 由田信息技术(上海)有限公司 Eye control device in vehicle and method for eye control
CN103366510A (en) * 2013-07-02 2013-10-23 惠州Tcl移动通信有限公司 Mobile phone and safe driving method based on camera of mobile phone
CN103390300A (en) * 2012-11-23 2013-11-13 深圳市捷顺科技实业股份有限公司 Parking lot management system
CN104077568A (en) * 2014-06-23 2014-10-01 北京理工大学珠海学院 High-accuracy driver behavior recognition and monitoring method and system
CN104183091A (en) * 2014-08-14 2014-12-03 苏州清研微视电子科技有限公司 System for adjusting sensitivity of fatigue driving early warning system in self-adaptive mode
CN104240446A (en) * 2014-09-26 2014-12-24 长春工业大学 Fatigue driving warning system on basis of human face recognition
CN104346897A (en) * 2013-07-23 2015-02-11 深圳市赛格导航科技股份有限公司 Vehicle-mounted terminal eyeball identification alarm system and method
CN104340118A (en) * 2013-08-01 2015-02-11 昆山研达电脑科技有限公司 Traffic offence prompting device and implementing method thereof for drivers
CN104574821A (en) * 2015-01-13 2015-04-29 上海优思通信科技有限公司 Driving safety prompting device and method
CN104574820A (en) * 2015-01-09 2015-04-29 安徽清新互联信息科技有限公司 Fatigue drive detecting method based on eye features
CN104760538A (en) * 2015-04-29 2015-07-08 深圳市保千里电子有限公司 Fatigue driving preventing massage device and using method thereof
CN104794776A (en) * 2015-04-17 2015-07-22 巫立斌 Driving recorder based on Internet of Things
CN104881956A (en) * 2015-06-17 2015-09-02 上海大学 Fatigue driving early warning system
CN105354986A (en) * 2015-11-12 2016-02-24 熊强 Driving state monitoring system and method for automobile driver
CN105679351A (en) * 2015-06-06 2016-06-15 孔霞 Intelligent music play system based on network communication
CN105678961A (en) * 2016-03-22 2016-06-15 重庆长安汽车股份有限公司 Anti-fatigue driving automobile safety warning system and warning method
CN106257549A (en) * 2015-06-22 2016-12-28 罗伯特·博世有限公司 For providing eyes to open the method and apparatus of reference levels of distance
CN106448265A (en) * 2016-10-27 2017-02-22 广州微牌智能科技有限公司 Collecting method and device of driver's driving behavior data
CN106671880A (en) * 2016-12-30 2017-05-17 天津云视科技发展有限公司 Vehicle operation state diagnostic system applying cloud computing and artificial intelligence technology
CN106897659A (en) * 2015-12-18 2017-06-27 腾讯科技(深圳)有限公司 The recognition methods of blink motion and device
CN107292278A (en) * 2017-06-30 2017-10-24 哈尔滨理工大学 A kind of face identification device and its recognition methods based on Adaboost algorithm
CN107330378A (en) * 2017-06-09 2017-11-07 湖北天业云商网络科技有限公司 A kind of driving behavior detecting system based on embedded image processing
CN107650913A (en) * 2016-07-26 2018-02-02 罗伯特·博世有限公司 Method and apparatus for the duration of the state of necessity of the driver that determines vehicle
CN108022256A (en) * 2017-12-25 2018-05-11 虹德科技(杭州)有限公司 A kind of video intelligent analyzer
CN108682158A (en) * 2018-03-23 2018-10-19 深圳融易保科技有限公司 Video analysis and scoring method for early warning and system
CN108682157A (en) * 2018-03-23 2018-10-19 深圳融易保科技有限公司 Video analysis and method for early warning and system
CN109299641A (en) * 2018-04-19 2019-02-01 中国铁路总公司 A kind of train dispatcher's fatigue monitoring image adaptive Processing Algorithm
CN110641467A (en) * 2018-06-25 2020-01-03 广州汽车集团股份有限公司 Vehicle distance control method and device of adaptive cruise system
CN111152653A (en) * 2018-11-07 2020-05-15 行为科技(北京)有限公司 Fatigue driving detection method based on multi-information fusion
CN111461206A (en) * 2020-03-30 2020-07-28 华南理工大学 Electroencephalogram-based fatigue detection method with steering wheel embedded with electroencephalogram sensor
CN111563435A (en) * 2020-04-28 2020-08-21 深圳市优必选科技股份有限公司 Sleep state detection method and device for user
CN111657889A (en) * 2020-06-03 2020-09-15 大连理工大学 Non-contact driver fatigue detection method based on millimeter wave radar
CN111657973A (en) * 2020-07-09 2020-09-15 海南科技职业大学 Fatigue degree detecting system based on artificial intelligence
CN111959393A (en) * 2020-05-11 2020-11-20 黄建松 Warning device and warning method for unsafe driving
CN112168636A (en) * 2020-09-24 2021-01-05 华人运通(上海)云计算科技有限公司 Method, device, equipment and system for improving vision in vehicle
CN113316805A (en) * 2019-01-17 2021-08-27 俊戈连接公司 Method and system for monitoring a person using infrared and visible light
WO2022142332A1 (en) * 2020-12-31 2022-07-07 上海商汤临港智能科技有限公司 Alert threshold adjustment method, apparatus, electronic device, and storage medium

Cited By (62)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8411901B2 (en) 2009-12-29 2013-04-02 Industrial Technology Research Institute Method and system for calculating image recognition rate and embedded image processing system
CN101803928A (en) * 2010-03-05 2010-08-18 北京智安邦科技有限公司 Video-based driver fatigue detection device
CN102208125A (en) * 2010-03-30 2011-10-05 深圳市赛格导航科技股份有限公司 Fatigue driving monitoring system and method thereof
CN102208125B (en) * 2010-03-30 2015-02-25 深圳市赛格导航科技股份有限公司 Fatigue driving monitoring system and method thereof
CN102289660B (en) * 2011-07-26 2013-07-03 华南理工大学 Method for detecting illegal driving behavior based on hand gesture tracking
CN102289660A (en) * 2011-07-26 2011-12-21 华南理工大学 Method for detecting illegal driving behavior based on hand gesture tracking
CN102542729A (en) * 2012-01-11 2012-07-04 广西卫通汽车信息技术有限公司 Method, device and system for monitoring retinal fatigue of driver
CN103259971A (en) * 2012-02-16 2013-08-21 由田信息技术(上海)有限公司 Eye control device in vehicle and method for eye control
CN102707801B (en) * 2012-05-07 2015-08-19 广东好帮手电子科技股份有限公司 Vehicle-mounted control system for identifying and control method thereof
CN102707801A (en) * 2012-05-07 2012-10-03 广东好帮手电子科技股份有限公司 Vehicle-mounted recognition control system and control method thereof
CN102785574A (en) * 2012-07-17 2012-11-21 东莞市泰斗微电子科技有限公司 Fatigue or drunk-driving detection and control method and corresponding system
CN102785574B (en) * 2012-07-17 2015-04-15 泰斗微电子科技有限公司 Fatigue or drunk-driving detection and control method and corresponding system
CN103390300A (en) * 2012-11-23 2013-11-13 深圳市捷顺科技实业股份有限公司 Parking lot management system
CN103035140B (en) * 2012-12-24 2014-09-10 吉林大学 Automobile driver dangerous driving behavior correction and evaluation system
CN103035140A (en) * 2012-12-24 2013-04-10 吉林大学 Automobile driver dangerous driving behavior correction and evaluation system
CN103366510B (en) * 2013-07-02 2016-02-24 惠州Tcl移动通信有限公司 A kind of mobile phone and the safe travelling method based on its camera
CN103366510A (en) * 2013-07-02 2013-10-23 惠州Tcl移动通信有限公司 Mobile phone and safe driving method based on camera of mobile phone
CN104346897B (en) * 2013-07-23 2017-02-08 深圳市赛格导航科技股份有限公司 Vehicle-mounted terminal eyeball identification alarm system and method
CN104346897A (en) * 2013-07-23 2015-02-11 深圳市赛格导航科技股份有限公司 Vehicle-mounted terminal eyeball identification alarm system and method
CN104340118A (en) * 2013-08-01 2015-02-11 昆山研达电脑科技有限公司 Traffic offence prompting device and implementing method thereof for drivers
CN104077568A (en) * 2014-06-23 2014-10-01 北京理工大学珠海学院 High-accuracy driver behavior recognition and monitoring method and system
CN104183091A (en) * 2014-08-14 2014-12-03 苏州清研微视电子科技有限公司 System for adjusting sensitivity of fatigue driving early warning system in self-adaptive mode
CN104240446A (en) * 2014-09-26 2014-12-24 长春工业大学 Fatigue driving warning system on basis of human face recognition
CN104574820A (en) * 2015-01-09 2015-04-29 安徽清新互联信息科技有限公司 Fatigue drive detecting method based on eye features
CN104574821B (en) * 2015-01-13 2017-05-24 上海优思通信科技有限公司 Driving safety prompting device and method
CN104574821A (en) * 2015-01-13 2015-04-29 上海优思通信科技有限公司 Driving safety prompting device and method
CN104794776A (en) * 2015-04-17 2015-07-22 巫立斌 Driving recorder based on Internet of Things
CN104794776B (en) * 2015-04-17 2017-08-25 青岛龙凯信息科技有限公司 A kind of drive recorder based on Internet of Things
CN104760538A (en) * 2015-04-29 2015-07-08 深圳市保千里电子有限公司 Fatigue driving preventing massage device and using method thereof
CN105679351B (en) * 2015-06-06 2018-08-10 北京聚能鼎力科技股份有限公司 Music intelligence play system based on network communication
CN105679351A (en) * 2015-06-06 2016-06-15 孔霞 Intelligent music play system based on network communication
CN104881956A (en) * 2015-06-17 2015-09-02 上海大学 Fatigue driving early warning system
CN106257549A (en) * 2015-06-22 2016-12-28 罗伯特·博世有限公司 For providing eyes to open the method and apparatus of reference levels of distance
CN106257549B (en) * 2015-06-22 2020-08-11 罗伯特·博世有限公司 Method and device for providing a reference level for the eye opening distance
CN105354986B (en) * 2015-11-12 2017-12-01 熊强 Driver's driving condition supervision system and method
CN105354986A (en) * 2015-11-12 2016-02-24 熊强 Driving state monitoring system and method for automobile driver
CN106897659A (en) * 2015-12-18 2017-06-27 腾讯科技(深圳)有限公司 The recognition methods of blink motion and device
CN106897659B (en) * 2015-12-18 2019-05-24 腾讯科技(深圳)有限公司 The recognition methods of blink movement and device
CN105678961B (en) * 2016-03-22 2017-12-29 重庆长安汽车股份有限公司 The automobile safety alarm system and alarming method for power of a kind of anti-fatigue-driving
CN105678961A (en) * 2016-03-22 2016-06-15 重庆长安汽车股份有限公司 Anti-fatigue driving automobile safety warning system and warning method
CN107650913A (en) * 2016-07-26 2018-02-02 罗伯特·博世有限公司 Method and apparatus for the duration of the state of necessity of the driver that determines vehicle
CN106448265A (en) * 2016-10-27 2017-02-22 广州微牌智能科技有限公司 Collecting method and device of driver's driving behavior data
CN106671880A (en) * 2016-12-30 2017-05-17 天津云视科技发展有限公司 Vehicle operation state diagnostic system applying cloud computing and artificial intelligence technology
CN107330378A (en) * 2017-06-09 2017-11-07 湖北天业云商网络科技有限公司 A kind of driving behavior detecting system based on embedded image processing
CN107292278A (en) * 2017-06-30 2017-10-24 哈尔滨理工大学 A kind of face identification device and its recognition methods based on Adaboost algorithm
CN108022256A (en) * 2017-12-25 2018-05-11 虹德科技(杭州)有限公司 A kind of video intelligent analyzer
CN108682158A (en) * 2018-03-23 2018-10-19 深圳融易保科技有限公司 Video analysis and scoring method for early warning and system
CN108682157A (en) * 2018-03-23 2018-10-19 深圳融易保科技有限公司 Video analysis and method for early warning and system
CN109299641A (en) * 2018-04-19 2019-02-01 中国铁路总公司 A kind of train dispatcher's fatigue monitoring image adaptive Processing Algorithm
CN109299641B (en) * 2018-04-19 2020-10-16 中国铁路总公司 Train dispatcher fatigue monitoring image adaptive processing algorithm
CN110641467A (en) * 2018-06-25 2020-01-03 广州汽车集团股份有限公司 Vehicle distance control method and device of adaptive cruise system
CN111152653A (en) * 2018-11-07 2020-05-15 行为科技(北京)有限公司 Fatigue driving detection method based on multi-information fusion
CN113316805A (en) * 2019-01-17 2021-08-27 俊戈连接公司 Method and system for monitoring a person using infrared and visible light
CN111461206A (en) * 2020-03-30 2020-07-28 华南理工大学 Electroencephalogram-based fatigue detection method with steering wheel embedded with electroencephalogram sensor
CN111461206B (en) * 2020-03-30 2023-05-26 华南理工大学 Electroencephalogram-based fatigue detection method for steering wheel embedded electroencephalogram sensor
CN111563435A (en) * 2020-04-28 2020-08-21 深圳市优必选科技股份有限公司 Sleep state detection method and device for user
CN111959393A (en) * 2020-05-11 2020-11-20 黄建松 Warning device and warning method for unsafe driving
CN111657889A (en) * 2020-06-03 2020-09-15 大连理工大学 Non-contact driver fatigue detection method based on millimeter wave radar
CN111657973A (en) * 2020-07-09 2020-09-15 海南科技职业大学 Fatigue degree detecting system based on artificial intelligence
CN112168636A (en) * 2020-09-24 2021-01-05 华人运通(上海)云计算科技有限公司 Method, device, equipment and system for improving vision in vehicle
CN112168636B (en) * 2020-09-24 2023-02-03 华人运通(上海)云计算科技有限公司 In-vehicle vision improving device, equipment and system
WO2022142332A1 (en) * 2020-12-31 2022-07-07 上海商汤临港智能科技有限公司 Alert threshold adjustment method, apparatus, electronic device, and storage medium

Similar Documents

Publication Publication Date Title
CN101599207A (en) A kind of fatigue driving detection device and automobile
CN101593425B (en) Machine vision based fatigue driving monitoring method and system
CN102263937B (en) Driver's driving behavior monitoring device and monitoring method based on video detection
CN202142160U (en) Fatigue driving early warning system
CN104183091B (en) System for adjusting sensitivity of fatigue driving early warning system in self-adaptive mode
CN102707801B (en) Vehicle-mounted control system for identifying and control method thereof
CN101574260B (en) Vehicle-mounted fatigue early warning device and method thereof
CN102881116A (en) System and method for pre-warning of fatigue driving
WO2020237664A1 (en) Driving prompt method, driving state detection method and computing device
WO2019127274A1 (en) Alarm method and device for criminal activity, storage medium and server
CN106218405A (en) Fatigue driving monitoring method and cloud server
CN101639894A (en) Method for detecting train driver behavior and fatigue state on line and detection system thereof
CN101950355A (en) Method for detecting fatigue state of driver based on digital video
CN103824420A (en) Fatigue driving identification system based on heart rate variability non-contact measuring
CN102752458A (en) Driver fatigue detection mobile phone and unit
CN106467057B (en) The method, apparatus and system of lane departure warning
CN110575163A (en) Method and device for detecting driver distraction
CN101877051A (en) Driver attention state monitoring method and device
CN101587544A (en) Automotive on-vehicle antitracking device based on computer vision
CN204890010U (en) Driver system of testing and assessing
Milan DRIVER’S DROWSINESS DETECTION
CN110264670A (en) Based on passenger stock tired driver driving condition analytical equipment
CN116386277A (en) Fatigue driving detection method and device, electronic equipment and medium
Suryawanshi et al. Driver drowsiness detection system based on lbp and haar algorithm
CN201927155U (en) Vehicle-mounted information intelligent processing system

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C12 Rejection of a patent application after its publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20091209