CN104224204A - Driver fatigue detection system on basis of infrared detection technology - Google Patents
Driver fatigue detection system on basis of infrared detection technology Download PDFInfo
- Publication number
- CN104224204A CN104224204A CN201410157686.5A CN201410157686A CN104224204A CN 104224204 A CN104224204 A CN 104224204A CN 201410157686 A CN201410157686 A CN 201410157686A CN 104224204 A CN104224204 A CN 104224204A
- Authority
- CN
- China
- Prior art keywords
- eyes
- infrared
- driver
- image
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The invention relates to a driver fatigue detection system and belongs to the technical field of detection equipment. The driver fatigue detection system comprises a CCD (charge coupled device) camera connected with a DSP (digital signal processor) through a video input decoder, and detection images are subjected to mode identification and processing. An infrared light source of the CCD camera is composed of two groups of infrared diodes, and the infrared diodes are uniformly distributed on a same ring of the same plane and same axis and are optionally switched. The infrared diodes emit infrared light of 850nm and 950nm in wavelength. The driver fatigue detection system is characterized in that judgment of lip syncing is added on the basis of human eye area judgment, so that probability of judgment missing or mistakes is decreased. By a method for differentiating odd-even frames of the images, positions and characteristics of eyes and lips can be accurately detected, and fatigue degree of a driver can be accurately detected. By the aid of a Kalman filter and a Mean-shift algorithm, the positions and the characteristics of the eyes and the lips are continuously detected; by a short-time tracking strategy, the eyes and the lips are tracked, and prediction is realized.
Description
Technical field
The present invention relates to a kind of Study in Driver Fatigue State Surveillance System, belong to checkout equipment technical field.
Background technology
Current driver fatigue detect delay method can be divided into two large classes:
A) from driver's unique characteristics, physiological parameter feature or the visual signature of driver is obtained by certain equipment, utilize driver different with the feature mode of fatigue state at normal condition, adopt corresponding mode identification technology to classify to differentiate, thus detect whether there be tired generation;
B) indirectly judge whether driver produces fatigue according to the behavior expression of vehicle.In this kind of technology, vehicle various parameters are in the process of moving obtained by sensor, according to the abnormal conditions in vehicle travel process, as vehicle whether exceed road mark line, whether speed exceeds the speed limit, whether distance between vehicle too near, judges whether driver has tired generation.
Current fatigue detecting system is many, and what effectiveness comparison was good has following several:
A) DAS of Australian International University of Japan exploitation, commercially obtains use at present.Use the tracing of human eye system monitoring driver be arranged on fascia, can monitoring driving person show, the grip feedback of utilization orientation dish, feeds back to driver by road surface tracing deviation simultaneously.Road surface tracking equipment object detects vehicle to nuzzle up suddenly the unlawful practice such as pavement marker or road edge, and this system provides warning when exception being detected, and has the braking function of steering wheel.The Seat vibration alerts simultaneously adopted is relevant with the degree that road side is departed from, and corrects deviated route to encourage them.
B) European Union completed AWAKE engineering in 2004.The significant condition adopted comprises eyelid movement, grip changes and road surface is followed the tracks of, and the braking action such as apply the brakes and steering wheel position, these methods combining are got up to resist traffic risk.
C) the Copilot monitoring system of Carnegie Mellon university exploitation.Measure eyelid movement by Perelose, monitoring system is little and handy, provides a kind of effective research tool.
D) the FaeeLAB III system monitoring driving behavior of Seeing machine seminar exploitation, can detect the situations such as tired and distraction.Obtain video image by a pair video camera, from left to right coupling draws the three-dimensional position of each feature.Adopt Least-squares minimization method positioning head three-dimensional position, FaceLAB software parallel process eye-gaze data, iris center, location, determines eye gaze direction according to eye gaze vector, calculates eyes and opens and frequency of wink, monitoring eyelid.FaceLAB is proved to be very effective in drive simulating, and has good effect, even if driver is with sunglasses also can detect on low light, head grand movement and visual direction are followed the tracks of.Can find out, system utilizes the comprehensive characteristics such as the behavior expression of various visual signature and vehicle to go to detect whether have tired generation, and some type of alarm of also employing simultaneously had reminds driver to drive with caution.At home this respect is related to fewer at present, only have a small amount of achievement in research.
Problems existing and development trend
Due to driver's individual variation and the driving environment difference such as light, road surface large, current fatigue detecting algorithm is substantially all based on drive simulating environment, next step will develop on true driving environment, make fatigue detecting technology be widely applied to commercial field, reduce fatigue driving and the vehicle accident that causes.No matter be from driver's unique characteristics or from vehicle behavior aspect, the characteristic type that can directly obtain is limited, and the surface characteristic data of extracting directly is many and have redundancy.Therefore to excavate fatigue characteristic on the one hand, extract with the signal processing method of advanced person and can characterize tired characteristic parameter, signal fused processing method to be adopted on the other hand, multiple fatigue characteristic parameter is combined driver fatigue situation is detected, overcome the impact such as space, illumination, improve real-time, the accuracy of detection algorithm.Study in Driver Fatigue State Surveillance System should have tired decision making function.Because different people has different tired performance characteristics, fatigue detecting system should have intelligence, has self study, inference function.At the initial stage of driving, system is trained system according to obtaining the related data of driver, can show that the fatigue characteristic of driver selects his detection method the most applicable, can adapt to each driver.
Current public fatigue driving data set compares shortage, detection system method of evaluating performance disunity, is difficult to carry out Quantitative Comparison to the performance of various fatigue detecting technology.In order to quantitatively, more various detection technique objectively, promote the development of fatigue detection method, need to strengthen the construction to fatigue driving vedio data storehouse, integrated system evaluating method and specification.
Fatigue detecting equipment mainly concentrates on the detection to driver eye both at home and abroad at present, does not have to realize the judgement to nozzle type, therefore easily occurs the situation failed to judge and misjudge; In addition, fatigue driving equipment can only detect eye position and eye feature in real time both at home and abroad at present, can not realize the forecast function to eye position and eye feature, therefore not realize forecast function to the fatigue state of driver; Fatigue detecting equipment takes the method for odd even two two field picture pixel comparison to detect eye position and feature both at home and abroad at present, and detection speed is slow, and accuracy of detection is low.
Summary of the invention
The object of the invention is to the deficiency solving the existence of above-mentioned prior art, a kind of Study in Driver Fatigue State Surveillance System based on infrared detection technology is provided.
Native system, by comparing analysis to various fatigue detection method, proposes the Study in Driver Fatigue State Surveillance System design of the information such as a set of comprehensive driver's eyes fatigue characteristic, head pose, countenance feature, driver's direction of gaze.Have the feature of notable difference based on the reflectance of human eye to two kinds of wavelength (850nm/950nm) Infrared, system adopts infrared light supply to gather image, takes the method for odd even two two field picture being carried out difference, accurately detects eyes.Kalman filter and Mean-shift algorithm is utilized to take to adopt continuous detecting, in short-term tracking strategy, eye position is followed the tracks of, predicts, analyze the information of eyes and calculate the countenance feature that eye feature, head pose and mouth states can show, by tired to these feature-set either shallow, moderate is tired and overtired criterion provide the warning of different stage, realizes the function of fatigue detecting.
A kind of Study in Driver Fatigue State Surveillance System based on infrared detection technology of the present invention, include the CCD photographic head based on infrared detection technology, CCD photographic head is connected to digital signal processor DSP infrared detection image by video input decoder, pattern recognition and process are carried out to detected image, digital signal processor is connected with alarm, by judging that the opening and closing of driver's eyes judges that whether driver is tired, and further can by judging whether the eigenvalue of eyes and face and positional value reach setting numerical value, system reminds driver to take a good rest and driving safety by alarm generation alarm sound.
The above-mentioned CCD photographic head based on infrared detection technology, infrared light supply is made up of two groups of inner and outer ring infrared diodes, is evenly distributed on an annulus of isoplanar concentric, freely switches between the two, and the limited view scope of video camera is reduced;
Described two groups of infrared diodes launch the infrared light of 850nm wavelength, 950nm wavelength respectively, realize the detection of infrared detection technology to eyes.
(1) infrared detection technology is to the principle of eye detection
Human eye is different to the infrared light reflection range degree of different wave length, and when 850nm wavelength, retina can reflect the incident illumination of 90%, when 950nm, retina can only reflect the incident illumination of 40%, and within the scope of 880 ± 80nm, other parts of face are basically identical for infrared degree of reflection.The infrared image obtained with 950nm infrared irradiation respectively face of 850nm wavelength carries out Difference Calculation, can accurately detect eyes.The optimum size being arranged on the light-emitting diodes pipe ring of lens surface can only be an empirical value.If a circle diode of outside is opened will produce dark pupil effect, a circle diode of the inside is opened and will be produced bright pupil effect.
(2) positioning analysis of eyes and tracing algorithm
A) detection of eyes
Complete the detection of eyes based on above-mentioned infrared detection principle, after Infrared irradiation face, utilize CCD camera collection facial video image.Collect the reflected image Difference Calculation of two wavelength, can eyes be detected.Due to effect of noise, the image that difference obtains needs to carry out pretreatment, first rectangular histogram equalization process is carried out, the gray level many to number of pixels in image carries out broadening, the gray level few to number of pixels is reduced, the form that the histogram transformation of original image uniformly distributes, be then converted into bianry image.
B) eye tracks algorithm
Mean-Shift algorithm and Kalman filter are combined, adopts continuous detecting, in short-term tracking strategy, when namely Kalman filter detects human eye, directly adopt its result detected, record present image and position of human eye simultaneously; If can't detect, just with the image recorded for the last time and position of human eye, Mean-Shift algorithm is initialized, and use Mean-Shift algorithm to match search human eye area on present image, if Kalman filter still can't detect in successive image, then carry out the tracking of Mean-Shift algorithm always.
(3) eyelid movement parameter
After people enters fatigue, eyes there will be open close slow, eye rotation frequency lowers, eyelid has closed trend, the feature such as One's eyesight is restrained.Extract the characteristic information of eyelid, utilize eyes closed PERCLOS average time algorithm and the average closure speed algorithm of eyes to calculate the fatigue data of driver.
PERCLOS principle
Time scale shared when PERCLOS refers to eyes closed within the regular hour.
P70, P80, EYEMEA (EM) three kinds of modules are had in specific experiment:
P70: the percentage of time of eyes closed area more than 70%;
P80: the percentage of time of eyes closed area more than 80%, this index is the most frequently used; Wherein P80 is considered to the degree of fatigue that can reflect people.
Fig. 5 is the measuring principle figure of PERCLOS value.In figure, curve is an eyes closed and opens in process degree of opening curve over time, can obtain the closed of required certain degree of eyes measured according to this curve or open the lasting time, thus calculate PERCLOS value.T in figure
1for eyes open the time of closed 20% completely; t
2for eyes open the time of closed 80% completely; t
3for eyes open the time of opening 20% completely next time; t
4for eyes open the time of opening 80% completely next time.By measuring t
1to t
4value just can calculate the value of PERCLOS:
In formula, the percentage rate of f a certain special time shared by the eyes closed time
For P80 metering system, as PERCLOS value f>0.15, think that driver is in fatigue state.(4) human face posture analysis
The facial pose of driver utilizes the positional information of eyes to make assessment
Δ x=x in formula
2-x
1, Δ y=y
2-y
1, (x
2, y
2) be the coordinate position of left and right eyes.
When face is partial to left direction, then θ > 8 °;
When face is at dead ahead, then | θ |≤8 °;
When face is partial to right direction, then θ <-8 °;
(5) eye gaze Direction estimation and tracking (see figure 6)
The information of watching attentively of people comprises the direction of gaze of face and the direction of gaze of eyes, and compared to the direction of gaze of eyes, the direction of gaze visual field of face is larger, and therefore face is watched attentively for a long time and watch attentively as broad sense, eyes are watched attentively for a long time and watched attentively as narrow sense.Consider these two in design and watch information attentively.Broad sense is watched attentively and is calculated through head pose information, and the eye pupil geometric parameter that narrow sense is watched attentively through video camera shooting calculates.Because the shape of pupil and direction change along with head pose change, therefore, broad sense is watched information attentively and is finally also calculated with the geometric parameter of pupil.If Δ x and Δ y is the displacement deflection parameter of pupil-reflective effect; R is the ratio of pupillary reflex image ellipse major axis and minor axis; θ is oval deflection; g
x, g
yit is the coordinate function of pupillary reflex light image.Δ x and Δ y reflects the functional relationship between reflected image and through hole, is that narrow sense watches information attentively.R to represent in anti-phase plane that face rotates, when face at dead ahead time, ratio is 1, and when face moves up and down time, ratio becomes large or diminishes.Angle θ is facial movement direction in camera objective optical axial plane, (g
x, g
y) pupil center-of-mass coordinate in plane.
Obtain above six parameters, utilize extensive recurrent neural networks to eye gaze directional structure vectorical structure mathematical function, what calculate driver watches information attentively, if driver is to certain direction fixation time time-out, then judges fatigue state.
(6) countenance analysis
Utilize the status information of mouth to describe countenance feature.Eyes detected, the face area of people is also determined thereupon.Face is divided into upper and lower two parts by the eyes coordinates position detected, then adopt iterative threshold values selection algorithm to carry out threshold values process to the latter half of face, thus make up little due to lip and colour of skin difference and in the binary image that causes, can not get the problem of face integrity profile.The image of shooting is converted to gray level image, carries out automatization's threshold values process, can obtain the profile information in face region to threshold values.
Because the gray value of lip is more shallow than face, thus there will be hole region after binary conversion treatment, also there will be the impact of " cavity " and the noise spot caused in nostril simultaneously.After obtaining the profile information in face region, image is carried out denoising, utilize minimum enclosed rectangle indicate face produce the profile of hole region, finally utilize classical Harris corner detection approach to search the corners of the mouth.The present invention detects angle point and carries out in original gray level image, and only detects face region.The specific algorithm of Corner Detection is as follows:
R＝det(M)-k(tr
2(M))
In formula
Wherein I
u(x, y), I
v(x, y), I
uvthe partial derivative of the gray scale that (x, y) is respectively picture point (x, y) place in u and v direction and second-order mixed partial derivative; K is constant; The mark that tr (M) is Metzler matrix.
The ratio of the upper lower lip height that the degree that face opens is opened by face and left and right corners of the mouth width is determined.Utilize the yawning frequency computation part driver's fatigue degree of face.
The present invention has following innovative point
1, on the basis judging human eye area, add again the judgement to nozzle type, reduce the probability of failing to judge or misjudging.
2, odd even two two field picture is carried out the method for difference, accurately detect eyes, the position of nozzle type and feature, realize the accurate detection to driver's fatigue degree.
3, native system adopts Kalman filter and Mean-shift algorithm, realizes continuous detecting, and adopts tracking strategy in short-term, follow the tracks of, and realize forecast function to eyes and nozzle type position the position of eyes and nozzle type and feature.
Accompanying drawing explanation
Fig. 1: the principle schematic of bright pupil (above) and dark pupil (below) effect;
Fig. 2: pupil effect image schematic diagram;
Fig. 3: infrared light supply arranges schematic diagram;
Fig. 4: the image schematic diagram after image (c) the difference binaryzation of the dark pupil of image (b) of (a) bright pupil;
Fig. 5: PERCLOS principle schematic;
Fig. 6: the coordinate relation schematic diagram between the bright pupil of (a) bright pupil image (b) pupil iridescent image (c)-reflective;
Fig. 7: the fundamental diagram of Study in Driver Fatigue State Surveillance System of the present invention;
Fig. 8: the hardware configuration schematic diagram of Study in Driver Fatigue State Surveillance System of the present invention.
Detailed description of the invention
Referring to accompanying drawing, provide the specific embodiment of the invention, the present invention will be further described.
Embodiment 1
The present embodiment is based on the Study in Driver Fatigue State Surveillance System of infrared detection technology, include the vehicle-borne CCD photographic head 1 based on infrared detection technology, CCD photographic head 1 absorbs the facial image of driver continuously, and ceaselessly follow the tracks of the motion of driver's eyes, CCD photographic head is connected to digital signal processor 5 infrared detection image by video input decoder 4, pattern recognition and process are carried out to detected image, by judging that the opening and closing of driver's eyes judges that whether driver is tired, setting numerical value whether is reached further by the eigenvalue and positional value that judge eyes and face, alarm sound is there is and reminds driver to take a good rest and driving safety in system by alarm 6.
The above-mentioned CCD photographic head 1 based on infrared detection technology, infrared light supply is made up of first group of internal ring infrared diode 2 and second group of outer shroud infrared diode 3, be evenly distributed on an annulus of isoplanar concentric, freely switch between the two, the limited view scope of video camera is reduced;
Described two groups of infrared diodes launch the infrared light of 850nm wavelength, 950nm wavelength respectively, realize the detection of infrared detection technology to eyes.
Based on the CCD photographic head of infrared detection technology, detected image is sent to digital signal processor DM642, native system software adopts MATLAB development environment to develop fatigue driving instrument inspection software, by simulink software conversionization C code, on the computer of CCS development environment, C code write emulator 510, and conversion turns in assembler language write hardware development plate DM642 further, pattern recognition and process are carried out to detected image, when the eigenvalue of eyes and face and positional value reach setting numerical value, alarm sound will be there is and remind driver to take a good rest and driving safety in system.
The full name TMS320DM642 of DM642, it is fixed DSP up-to-date in TI company C6000 series DSP, its core is C6416 type high-performance digital signal processor, there is extremely strong handling property, the motility of height and programmability, simultaneously peripherally be integrated with equipment and the interfaces such as very complete audio frequency, video and network service, be specially adapted to machine vision, medical imaging, Network Video Surveillance, digital broadcasting and the high-speed dsp application such as consumer electronics product based on digital video/image procossing.Take TVP5150 as video input decoder, be that audio frequency inputs Acquisition Circuit with PCM1801, the multi-channel video capturing being core processor with TMS320DM642 type DSP is held concurrently and is compressed processing PCI, is applied to and builds high stability, high robust and multimedia digital monitoring system.
TMS320DM642 adopts second filial generation high-performance, the advanced DSP core of very long instruction word veloci T1.2 structure and the parallel mechanism of enhancing, and periphery is integrated with very complete audio frequency, video and network communication interface.
Concrete technical solution is analyzed as follows.
1, technical scheme
(1) infrared detection technology is to the principle of eye detection
Human eye is different to the infrared light reflection range degree of different wave length, and when 850nm wavelength, retina can reflect the incident illumination of 90%, when 950nm, retina can only reflect the incident illumination of 40%, and within the scope of 880 ± 80nm, other parts of face are basically identical for infrared degree of reflection.The infrared image obtained with 950nm infrared irradiation respectively face of 850nm wavelength carries out Difference Calculation, can accurately detect eyes.The optimum size being arranged on the light-emitting diodes pipe ring of lens surface can only be an empirical value.If a circle diode of outside is opened will produce dark pupil effect, a circle diode of the inside is opened and will be produced bright pupil effect.(see Fig. 1,2,3)
(2) positioning analysis of eyes and tracing algorithm
A) detection of eyes
Complete the detection of eyes based on above-mentioned infrared detection principle, after Infrared irradiation face, utilize CCD to gather facial video image.Ccd video camera operation principle is based on active infrared camera technique, when the light sensor in video camera detects that visible ray is not enough to reach the requirement gathering image, control module in video camera carries out floor light by automatically opening solid luminescent infrared lamp, within the scope that the spectrum of infrared-emitting diode infrared lamp can be experienced in common CCD camera, and due to human eye insensitive to infrared light, so the illumination not enough impact on gathering video and bringing when namely thermal camera can eliminate night running, the normal traveling of driver can not be disturbed again.
Collect the reflected image Difference Calculation of two wavelength, can eyes be detected.Due to effect of noise, the image that difference obtains needs to carry out pretreatment, first rectangular histogram equalization process is carried out, the gray level many to number of pixels in image carries out broadening, the gray level few to number of pixels is reduced, the form that the histogram transformation of original image uniformly distributes, be then converted into bianry image.(see figure 4)
B) eye tracks algorithm
Mean-Shift algorithm and Kalman filter combine by the present invention, adopt continuous detecting, in short-term tracking strategy, when namely Kalman filter detects human eye, directly adopt its result detected, record present image and position of human eye simultaneously; If can't detect, just with the image recorded for the last time and position of human eye, Mean-Shift algorithm is initialized, and use Mean-Shift algorithm to match search human eye area on present image, if Kalman filter still can't detect in successive image, then carry out the tracking of Mean-Shift algorithm always.
(3) eyelid movement parameter
After people enters fatigue, eyes there will be open close slow, eye rotation frequency lowers, eyelid has closed trend, the feature such as One's eyesight is restrained.Extract the characteristic information of eyelid, utilize eyes closed PERCLOS average time algorithm and the average closure speed algorithm of eyes to calculate the fatigue data of driver.
PERCLOS principle
Time scale shared when PERCLOS refers to eyes closed within the regular hour.
P70, P80, EYEMEA (EM) three kinds of modules are had in specific experiment:
P70: the percentage of time of eyes closed area more than 70%;
P80: the percentage of time of eyes closed area more than 80%, this index is the most frequently used; Wherein P80 is considered to the degree of fatigue that can reflect people.
Fig. 5 is the measuring principle figure of PERCLOS value.In figure, curve is an eyes closed and opens in process degree of opening curve over time, can obtain the closed of required certain degree of eyes measured according to this curve or open the lasting time, thus calculate PERCLOS value.T in figure
1for eyes open the time of closed 20% completely; t
2for eyes open the time of closed 80% completely; t
3for eyes open the time of opening 20% completely next time; t
4for eyes open the time of opening 80% completely next time.By measuring t
1to t
4value just can calculate the value of PERCLOS:
In formula, the percentage rate (see figure 5) of f a certain special time shared by the eyes closed time
For P80 metering system, as PERCLOS value f>0.15, think that driver is in fatigue state.
(4) human face posture analysis
The facial pose of driver utilizes the positional information of eyes to make assessment
Δ x=x in formula
2-x
1, Δ y=y
2-y
1, (x
2, y
2) be the coordinate position of left and right eyes.
When face is partial to left direction, then θ > 8 °;
When face is at dead ahead, then | θ |≤8 °;
When face is partial to right direction, then θ <-8 °;
(5) eye gaze Direction estimation and tracking (see figure 6)
The information of watching attentively of people comprises the direction of gaze of face and the direction of gaze of eyes, and compared to the direction of gaze of eyes, the direction of gaze visual field of face is larger, and therefore face is watched attentively for a long time and watch attentively as broad sense, eyes are watched attentively for a long time and watched attentively as narrow sense.Consider these two in design and watch information attentively.Broad sense is watched attentively and is calculated through head pose information, and the eye pupil geometric parameter that narrow sense is watched attentively through video camera shooting calculates.Because the shape of pupil and direction change along with head pose change, therefore, broad sense is watched information attentively and is finally also calculated with the geometric parameter of pupil.If Δ x and Δ y is the displacement deflection parameter of pupil-reflective effect; R is the ratio of pupillary reflex image ellipse major axis and minor axis; θ is oval deflection; g
x, g
yit is the coordinate function of pupillary reflex light image.Δ x and Δ y reflects the functional relationship between reflected image and through hole, is that narrow sense watches information attentively.R to represent in anti-phase plane that face rotates, when face at dead ahead time, ratio is 1, and when face moves up and down time, ratio becomes large or diminishes.Angle θ is facial movement direction in camera objective optical axial plane, (g
x, g
y) pupil center-of-mass coordinate in plane.
Obtain above six parameters, utilize extensive recurrent neural networks to eye gaze directional structure vectorical structure mathematical function, what calculate driver watches information attentively, if driver is to certain direction fixation time time-out, then judges fatigue state.
(6) countenance analysis
The present invention utilizes the status information of mouth to describe countenance feature.Eyes detected, the face area of people is also determined thereupon.Face is divided into upper and lower two parts by the eyes coordinates position detected, then adopt iterative threshold values selection algorithm to carry out threshold values process to the latter half of face, thus make up little due to lip and colour of skin difference and in the binary image that causes, can not get the problem of face integrity profile.The image of shooting is converted to gray level image, carries out automatization's threshold values process, can obtain the profile information in face region to threshold values.
Because the gray value of lip is more shallow than face, thus there will be hole region after binary conversion treatment, also there will be the impact of " cavity " and the noise spot caused in nostril simultaneously.After obtaining the profile information in face region, image is carried out denoising, utilize minimum enclosed rectangle indicate face produce the profile of hole region, finally utilize classical Harris corner detection approach to search the corners of the mouth.The present invention detects angle point and carries out in original gray level image, and only detects face region.The specific algorithm of Corner Detection is as follows:
R＝det(M)-k(tr
2(M))
In formula
Wherein I
u(x, y), I
v(x, y), I
uvthe partial derivative of the gray scale that (x, y) is respectively picture point (x, y) place in u and v direction and second-order mixed partial derivative; K is constant; The mark that tr (M) is Metzler matrix.
The ratio of the upper lower lip height that the degree that face opens is opened by face and left and right corners of the mouth width is determined.Utilize the yawning frequency computation part driver's fatigue degree of face.
2. the feasibility of technical scheme is discussed
Pattern recognition is now very ripe, and project adopts the algorithm based on pattern recognition to carry out system design, algorithmically has certain reliability.First carry out coding from Arithmetic of Face Image Recognition via in the design and build visual detecting system, simplify experiment difficulty; Video camera is directly accessed computer, after detecting and tracking information has been tested, can be transplanted in dsp chip easily, there is feasibility and maturity.
(1). the infrared beams of specific wavelength, along the irradiation optical axis face of video camera, can obtain bright pupil effect.If but departed from optical axis, the light of reflection could enter video camera and will cause dark pupil effect.Infrared light supply is arranged along the optical axis of lens in theory just can bright pupil effect.But in fact this point is difficult to, infrared light supply can limit the visual field of photographic head.The present invention is made up of infrared light supply two groups of infrared diodes, be evenly distributed on an annulus of isoplanar concentric, freely switch between the two, and the limited view scope of video camera is reduced.
(2). the current detection to driver fatigue is mainly based on the contact measurement method of physiological driver's signal, driver behavior behavior, vehicle-state.Although higher based on the bio-signal acquisition accuracy of driver, affect the driving of driver; Based on vehicle-state measurement, there is unreliability and non-operability.By comparison, the contactless detection device based on face characteristic does not have Body contact with driver, measure in real time, does not have nocuity radiation to driver.In the design by the efficient combination of multiple fatigue detection method, realize comprehensive detection to information such as driver ocular characteristics, head pose, countenance feature, eye gaze directions, finally the fatigue state that the fatigue of driver makes different brackets is divided and makes corresponding early warning judgement.
(3). utilize iterative threshold values selection method to carry out threshold values process in second region of face, well improve unintelligible, the connective bad defect of corners of the mouth profile that traditional binarization method causes.
The object of the invention and meaning:
1. developing rapidly along with Modern Traffic, vehicle accident is more and more serious, and fatigue driving has become the key factor of traffic safety hidden danger.Under the pressure of security consideration, fatigue driving causes the attention of common people, and various countries' research worker is devoted to research in this respect one after another, and achieves certain achievement.And China's research starting is in this regard late, research of the present invention can advance China's research and development in this regard.
2. the tired measuring technique of multiple features is under the support of infrared light supply, extract the eye feature of driver accurately, detect driver's eyelid movement characteristic parameter (blink speed, the time that opens and closes eyes, pupil geometric properties), head pose, eye gaze directional information, thus calculate fatigue strength; The characteristic parameter extracting driver's face in addition measures the countenance feature of driver face under fatigue state.The fatigue detecting of driver is carried out to the strategy of combination features detection, the fatigue conditions of driver is made and accurately detects in real time, huge meaning will be brought for traffic safety.
Claims (2)
1. the Study in Driver Fatigue State Surveillance System based on infrared detection technology, it is characterized in that including the CCD photographic head (1) based on infrared detection technology, CCD photographic head is connected to by video input decoder (4) digital signal processor (5) detected image being carried out to pattern recognition and process infrared detection image, and digital signal processor (5) is connected with alarm (6);
The described CCD photographic head based on infrared detection technology, infrared light supply is made up of first group of internal ring infrared diode (2) and second group of outer shroud infrared diode (3), is evenly distributed on an annulus of isoplanar concentric.
2., according to the Study in Driver Fatigue State Surveillance System based on infrared detection technology a kind of described in claim 1, it is characterized in that described two groups of infrared diodes launch the infrared light of 850nm wavelength, 950nm wavelength respectively, realize the detection of infrared detection technology to eyes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410157686.5A CN104224204B (en) | 2013-12-24 | 2014-04-18 | A kind of Study in Driver Fatigue State Surveillance System based on infrared detection technology |
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2013107218205 | 2013-12-24 | ||
| CN201310721820 | 2013-12-24 | ||
| CN201310721820.5 | 2013-12-24 | ||
| CN201410157686.5A CN104224204B (en) | 2013-12-24 | 2014-04-18 | A kind of Study in Driver Fatigue State Surveillance System based on infrared detection technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104224204A true CN104224204A (en) | 2014-12-24 |
| CN104224204B CN104224204B (en) | 2016-09-07 |
Family
ID=52213747
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410157686.5A Active CN104224204B (en) | 2013-12-24 | 2014-04-18 | A kind of Study in Driver Fatigue State Surveillance System based on infrared detection technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104224204B (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106355838A (en) * | 2016-10-28 | 2017-01-25 | 深圳市美通视讯科技有限公司 | Fatigue driving detection method and system |
| CN106548600A (en) * | 2016-12-06 | 2017-03-29 | 张家港全智电子科技有限公司 | Real-time pupil tracing system and fatigue state monitoring method |
| CN106814838A (en) * | 2015-11-30 | 2017-06-09 | 腾讯科技（深圳）有限公司 | The method and apparatus of terminal auto sleep |
| CN107194381A (en) * | 2017-07-06 | 2017-09-22 | 重庆邮电大学 | Driver status monitoring system based on Kinect |
| CN108886587A (en) * | 2016-03-29 | 2018-11-23 | 罗伯特·博世有限公司 | For observing the imaging sensor of the multi-functional observation video camera of vehicle driver, image sensor apparatus, observation video camera and method for generating infrared image and intensity image |
| CN109799610A (en) * | 2017-11-16 | 2019-05-24 | Aptiv技术有限公司 | Eye tracking apparatus |
| CN109948509A (en) * | 2019-03-11 | 2019-06-28 | 成都旷视金智科技有限公司 | Obj State monitoring method, device and electronic equipment |
| CN110062937A (en) * | 2016-12-28 | 2019-07-26 | 本田技研工业株式会社 | Information providing system |
| CN110191317A (en) * | 2019-05-21 | 2019-08-30 | 重庆工程学院 | A kind of electronic monitoring and control system based on image recognition |
| CN110638474A (en) * | 2019-09-25 | 2020-01-03 | 中控智慧科技股份有限公司 | Method, system and equipment for detecting driving state and readable storage medium |
| CN111382694A (en) * | 2020-03-06 | 2020-07-07 | 杭州宇泛智能科技有限公司 | Face recognition method and device and electronic equipment |
| CN111680546A (en) * | 2020-04-26 | 2020-09-18 | 北京三快在线科技有限公司 | Attention detection method, attention detection device, electronic equipment and storage medium |
| CN111985328A (en) * | 2020-07-16 | 2020-11-24 | 西安理工大学 | Unsafe driving behavior detection and early warning method based on facial feature analysis |
| CN112494045A (en) * | 2020-12-15 | 2021-03-16 | 南通路远科技信息有限公司 | Driver fatigue detection method and device |
| US11132543B2 (en) | 2016-12-28 | 2021-09-28 | Nvidia Corporation | Unconstrained appearance-based gaze estimation |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5769085A (en) * | 1993-01-06 | 1998-06-23 | Mitsubishi Jidosha Kogyo Kabushiki Kaisha | Apparatus for detecting awareness of a vehicle driver and method thereof |
| US20040005083A1 (en) * | 2002-03-26 | 2004-01-08 | Kikuo Fujimura | Real-time eye detection and tracking under various light conditions |
| CN1680779A (en) * | 2005-02-04 | 2005-10-12 | 江苏大学 | Fatigue monitoring method and device for driver |
| CN102393989A (en) * | 2011-07-28 | 2012-03-28 | 山西智济电子科技有限公司 | Real-time monitoring system of driver working state |
| CN203885510U (en) * | 2013-12-24 | 2014-10-22 | 烟台通用照明有限公司 | Driver fatigue detection system based on infrared detection technology |
-
2014
- 2014-04-18 CN CN201410157686.5A patent/CN104224204B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5769085A (en) * | 1993-01-06 | 1998-06-23 | Mitsubishi Jidosha Kogyo Kabushiki Kaisha | Apparatus for detecting awareness of a vehicle driver and method thereof |
| US20040005083A1 (en) * | 2002-03-26 | 2004-01-08 | Kikuo Fujimura | Real-time eye detection and tracking under various light conditions |
| CN1680779A (en) * | 2005-02-04 | 2005-10-12 | 江苏大学 | Fatigue monitoring method and device for driver |
| CN102393989A (en) * | 2011-07-28 | 2012-03-28 | 山西智济电子科技有限公司 | Real-time monitoring system of driver working state |
| CN203885510U (en) * | 2013-12-24 | 2014-10-22 | 烟台通用照明有限公司 | Driver fatigue detection system based on infrared detection technology |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106814838B (en) * | 2015-11-30 | 2020-12-04 | 腾讯科技（深圳）有限公司 | Method and device for terminal automatic dormancy |
| CN106814838A (en) * | 2015-11-30 | 2017-06-09 | 腾讯科技（深圳）有限公司 | The method and apparatus of terminal auto sleep |
| CN108886587A (en) * | 2016-03-29 | 2018-11-23 | 罗伯特·博世有限公司 | For observing the imaging sensor of the multi-functional observation video camera of vehicle driver, image sensor apparatus, observation video camera and method for generating infrared image and intensity image |
| CN108886587B (en) * | 2016-03-29 | 2022-02-01 | 罗伯特·博世有限公司 | Image sensor device and method for generating infrared image and intensity image |
| CN106355838A (en) * | 2016-10-28 | 2017-01-25 | 深圳市美通视讯科技有限公司 | Fatigue driving detection method and system |
| CN106548600A (en) * | 2016-12-06 | 2017-03-29 | 张家港全智电子科技有限公司 | Real-time pupil tracing system and fatigue state monitoring method |
| CN110062937A (en) * | 2016-12-28 | 2019-07-26 | 本田技研工业株式会社 | Information providing system |
| US11132543B2 (en) | 2016-12-28 | 2021-09-28 | Nvidia Corporation | Unconstrained appearance-based gaze estimation |
| CN107194381A (en) * | 2017-07-06 | 2017-09-22 | 重庆邮电大学 | Driver status monitoring system based on Kinect |
| CN109799610A (en) * | 2017-11-16 | 2019-05-24 | Aptiv技术有限公司 | Eye tracking apparatus |
| CN109948509A (en) * | 2019-03-11 | 2019-06-28 | 成都旷视金智科技有限公司 | Obj State monitoring method, device and electronic equipment |
| CN110191317A (en) * | 2019-05-21 | 2019-08-30 | 重庆工程学院 | A kind of electronic monitoring and control system based on image recognition |
| CN110191317B (en) * | 2019-05-21 | 2020-02-11 | 重庆工程学院 | Electronic monitoring system based on image recognition |
| CN110638474A (en) * | 2019-09-25 | 2020-01-03 | 中控智慧科技股份有限公司 | Method, system and equipment for detecting driving state and readable storage medium |
| CN111382694A (en) * | 2020-03-06 | 2020-07-07 | 杭州宇泛智能科技有限公司 | Face recognition method and device and electronic equipment |
| CN111680546A (en) * | 2020-04-26 | 2020-09-18 | 北京三快在线科技有限公司 | Attention detection method, attention detection device, electronic equipment and storage medium |
| CN111985328A (en) * | 2020-07-16 | 2020-11-24 | 西安理工大学 | Unsafe driving behavior detection and early warning method based on facial feature analysis |
| CN112494045A (en) * | 2020-12-15 | 2021-03-16 | 南通路远科技信息有限公司 | Driver fatigue detection method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104224204B (en) | 2016-09-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104224204B (en) | A kind of Study in Driver Fatigue State Surveillance System based on infrared detection technology | |
| CN104013414B (en) | A kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone | |
| EP1320830B1 (en) | Facial image processing system | |
| Jo et al. | Vision-based method for detecting driver drowsiness and distraction in driver monitoring system | |
| CN102324166B (en) | Fatigue driving detection method and device | |
| CN102054163B (en) | Method for testing driver fatigue based on monocular vision | |
| Batista | A drowsiness and point of attention monitoring system for driver vigilance | |
| CN105286802B (en) | Driver Fatigue Detection based on video information | |
| CN105719431A (en) | Fatigue driving detection system | |
| CN104574817A (en) | Machine vision-based fatigue driving pre-warning system suitable for smart phone | |
| CN103714659B (en) | Fatigue driving identification system based on double-spectrum fusion | |
| CN101950355A (en) | Method for detecting fatigue state of driver based on digital video | |
| Flores et al. | Driver drowsiness detection system under infrared illumination for an intelligent vehicle | |
| CN106682603A (en) | Real time driver fatigue warning system based on multi-source information fusion | |
| CN101425136A (en) | Method and device for acquiring partial binary mode characteristic of video image | |
| Tang et al. | Real-time image-based driver fatigue detection and monitoring system for monitoring driver vigilance | |
| CN102085099A (en) | Method and device for detecting fatigue driving | |
| CN104881956A (en) | Fatigue driving early warning system | |
| CN203885510U (en) | Driver fatigue detection system based on infrared detection technology | |
| Bergasa et al. | Visual monitoring of driver inattention | |
| CN203677103U (en) | CCD (Charge Coupled Device) camera based on infrared detection technology | |
| Kajiwara | Driver-condition detection using a thermal imaging camera and neural networks | |
| Lang et al. | The study of driver fatigue monitor algorithm combined PERCLOS and AECS | |
| Avizzano et al. | Real-time embedded vision system for the watchfulness analysis of train drivers | |
| Akshay et al. | Drowsy driver detection using eye-tracking through machine learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20190129 Address after: 264006 No. 69 Changjiang Road, Yantai Development Zone, Shandong Province Patentee after: YANTAI TAIMING LIGHTING CO., LTD. Address before: 264006 No. 69 Changjiang Road, Yantai Development Zone, Shandong Province Patentee before: YANTAI GENERAL LIGHTING CO., LTD. |
|
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: 264006 No. 69 Changjiang Road, Yantai Development Zone, Shandong, China Patentee after: Taiming Intelligent Technology Development Co., Ltd Address before: 264006 No. 69 Changjiang Road, Yantai Development Zone, Shandong Province Patentee before: YANTAI TAIMING LIGHTING Co.,Ltd. |