CN103839379A - Automobile and driver fatigue early warning detecting method and system for automobile - Google Patents

Abstract

The invention discloses an automobile and a driver fatigue early warning detecting method and system for the automobile. The driver fatigue early warning detecting system comprises a video collecting module, a facial recognition module, an eye detecting module and a judging module, wherein the video collecting module is used for collecting image information of passengers inside the automobile, the facial recognition module is used for processing the image information of the passengers inside the automobile to obtain facial characteristic parameters of a driver and obtaining facial position information of the driver according to the facial characteristic parameters of the driver, the eye detecting module is used for obtaining eye area information of the driver according to the facial position information of the driver and obtaining eye characteristic parameters of the driver according to the eye area information of the driver to detect eye characteristic information of the driver, and the judging module is used for judging whether the driver is in the fatigue state or not according to the eye characteristic information of the driver. According to the driver fatigue early warning detecting system, the driver inside the automobile can be automatically recognized, fatigue processes and movements of different drivers can be studied, the fatigued characteristics of the driver are memorized, and whether the driver is in the fatigue state or not is judged.

Description

Automobile and for its driver fatigue early-warning detection method and system

Technical field

The present invention relates to automobile technical field, particularly a kind of driver fatigue early warning detection system, the automobile with this system and a kind of driver fatigue early-warning detection method.

Background technology

At present, Related product based on eye state on market is according to perclos(Percentage of EyeIidCIosure over the PupiI, over Time mostly, tolerance fatigue/sleepy physical quantity) standard, account for the number percent of a period of time as the measurement index of physiological fatigue by the catacleisis time.Utilize facial recognition techniques location eyes, nose and corners of the mouth position, eyes, nose and corners of the mouth position are combined, again according to obtaining driver's notice direction to the tracking of eyeball, and by judging whether driver's notice disperses to report to the police, thereby can realize the object that prevents driver tired driving.

But, in correlation technique only in certain area, a people's face being judged, but generally in automobile, can not only have people of driver, so just require the condition of first carrying of fatigue judgement very strict, therefore can not effectively reach the effect of early warning, and there will be erroneous judgement, can not meet user's needs.

Summary of the invention

Object of the present invention is intended at least solve one of above-mentioned technological deficiency.

For this reason, first object of the present invention is to propose a kind of driver fatigue early warning detection system of the state of mind can identify driver in car detecting device and drive time.

The second object of the present invention is to propose a kind of automobile with above-mentioned driver fatigue early warning detection system.The 3rd object of the present invention is to propose a kind of driver fatigue early-warning detection method.

For achieving the above object, a kind of driver fatigue early warning detection system that first aspect present invention embodiment proposes, comprising: video acquisition module, for gathering passenger's image information; Facial recognition modules, described facial recognition modules processes to obtain driver's face characteristic parameter to described passenger's image information by gray level co-occurrence matrixes, color space transformation and skin color modeling, and according to the face positional information of driver described in described driver's face characteristic gain of parameter; Human eye detection module, for obtain described driver's ocular information according to described driver's face positional information, and according to the eye characteristic parameter of driver described in described driver's ocular acquisition of information to detect described driver's eye feature information; Judge module, for comparing to judge that according to described driver's eye feature information and default eye feature threshold value whether described driver is in fatigue state.

According to the driver fatigue early warning detection system of the embodiment of the present invention, gather passenger's image information by video acquisition module, and by facial recognition modules to passenger's image information analyzing and processing to identify driver's face characteristic parameter, then obtain driver's face positional information, human eye detection module is finally obtained driver's eye characteristic parameter to detect driver's eye feature information according to driver's face positional information, last judge module judges that according to driver's eye feature information whether driver is in fatigue state.Therefore, the driver fatigue early warning detection system of the embodiment of the present invention can be identified driver in car automatically, and can learn different drivers' fatigue process and action, and memory driver's fatigue characteristic, whether judge driver in fatigue state, thereby realize, the absorbed state of mind of driver is carried out to early warning, guarantee driving safety, meet user's needs, and reduce False Rate.

According to one embodiment of present invention, described facial recognition modules processes to obtain described passenger's face texture characteristic information to described passenger's image information according to described gray level co-occurrence matrixes, and identify described driver according to described passenger's face texture characteristic information.

According to one embodiment of present invention, described human eye detection module carries out according to described driver's face positional information that histogram equalization is processed and subregion processes to lock described driver's ocular, and described driver's ocular is carried out to connected domain processing to obtain described driver's eye position, and extract described driver's eye characteristic parameter according to described driver's eye position.

According to one embodiment of present invention, described judge module judges that according to perclos standard and BP neural network algorithm whether described driver is in described fatigue state.

According to one embodiment of present invention, described driver fatigue early warning detection system also comprises: warning module, described in described driver is during in described fatigue state, warning module sends alarm.

A kind of automobile that second aspect present invention embodiment proposes comprises above-mentioned driver fatigue early warning detection system.

According to the automobile of the embodiment of the present invention, can automatically identify driver in car, and can learn different drivers' fatigue process and action, and memory driver's fatigue characteristic, whether judge driver in fatigue state, thereby realize, the absorbed state of mind of driver is carried out to early warning, guarantee driving safety, meet user's needs.

For achieving the above object, a kind of driver fatigue early-warning detection method for automobile that third aspect present invention embodiment proposes, comprises the following steps: S1, collection passenger's image information; S2, by gray level co-occurrence matrixes, color space transformation and skin color modeling, described passenger's image information is processed to obtain driver's face characteristic parameter, and according to the face positional information of driver described in described driver's face characteristic gain of parameter; S3, obtains described driver's ocular information according to described driver's face positional information, and according to the eye characteristic parameter of driver described in described driver's ocular acquisition of information to detect described driver's eye feature information; S4, compares to judge that according to described driver's eye feature information and default eye feature threshold value whether described driver is in fatigue state.

According to the driver fatigue early-warning detection method for automobile of the embodiment of the present invention, by gathering passenger's image information, and to passenger's image information analyzing and processing to identify driver's face characteristic parameter, then obtain driver's face positional information, finally obtain driver's eye characteristic parameter according to driver's face positional information to detect driver's eye feature information, finally judge that according to driver's eye feature information whether driver is in fatigue state.Therefore, the driver fatigue early-warning detection method for automobile of the embodiment of the present invention can be identified driver in car automatically, and can learn different drivers' fatigue process and action, and memory driver's fatigue characteristic, whether judge driver in fatigue state, thereby realize, the absorbed state of mind of driver is carried out to early warning, guarantee driving safety, and can reduce False Rate.

According to one embodiment of present invention, in step S2, according to described gray level co-occurrence matrixes, described passenger's image information is processed to obtain described passenger's face texture characteristic information, and identify described driver according to described passenger's face texture characteristic information.

According to one embodiment of present invention, in step S3, carry out according to described driver's face positional information that histogram equalization is processed and subregion processes to lock described driver's ocular, and described driver's ocular is carried out to connected domain processing to obtain described driver's eye position, and extract described driver's eye characteristic parameter according to described driver's eye position.

According to one embodiment of present invention, in step S4, judge that according to perclos standard and BP neural network algorithm whether described driver is in described fatigue state.

The aspect that the present invention is additional and advantage in the following description part provide, and part will become obviously from the following description, or recognize by practice of the present invention.

Accompanying drawing explanation

The present invention above-mentioned and/or additional aspect and advantage will become from the following description of the accompanying drawings of embodiments obviously and easily and understand, wherein:

Fig. 1 is according to the block diagram of the driver fatigue early warning detection system of the embodiment of the present invention;

Fig. 2 A is the schematic diagram that textural characteristics face complexion area is cut apart;

Fig. 2 B is by the schematic diagram of textural characteristics face positioning result;

Fig. 3 is the process flow diagram of tentatively realizing the division in people's face skin skin region and other regions according to textural characteristics;

Fig. 4 A is the schematic diagram of face hair zones segmentation result;

Fig. 4 B is the schematic diagram that adopts the accurate positioning result of color space skin color modeling method face;

Fig. 5 is the process flow diagram of distinguishing area of skin color and hair zones according to colour of skin space according to an embodiment of the invention and accurately locating face;

Fig. 6 is face " three front yards " standard schematic diagram;

Fig. 7 is image denoising and face subregion (1/2 and 1/3) locking flow figure;

Fig. 8 is the algorithm flow chart of ocular locking;

Fig. 9 A is the schematic diagram of facial image situation in night;

Fig. 9 B is the schematic diagram of facial image situation on daytime;

Figure 10 is not bending moment calculation flow chart of seven of HUShi;

Figure 11 is BP neural network basic structure schematic diagram;

Figure 12 is the integration process flow diagram of integrated part;

Figure 13 is the top level diagram of driver fatigue early warning detection system internal module according to an embodiment of the invention; And

Figure 14 is according to the process flow diagram of the driver fatigue early-warning detection method for automobile of the embodiment of the present invention.

Embodiment

Describe embodiments of the invention below in detail, the example of described embodiment is shown in the drawings, and wherein same or similar label represents same or similar element or has the element of identical or similar functions from start to finish.Be exemplary below by the embodiment being described with reference to the drawings, only for explaining the present invention, and can not be interpreted as limitation of the present invention.

Disclosing below provides many different embodiment or example to be used for realizing different structure of the present invention.Of the present invention open in order to simplify, hereinafter the parts to specific examples and setting are described.Certainly, they are only example, and object does not lie in restriction the present invention.In addition, the present invention can be in different examples repeat reference numerals and/or letter.This repetition is in order to simplify and object clearly, itself do not indicate the relation between discussed various embodiment and/or setting.In addition, the various specific technique the invention provides and the example of material, but those of ordinary skills can recognize the property of can be applicable to of other techniques and/or the use of other materials.In addition, First Characteristic described below Second Characteristic it " on " structure can comprise that the first and second Characteristics creations are the direct embodiment of contact, also can comprise the embodiment of other Characteristics creation between the first and second features, such the first and second features may not be direct contacts.

In description of the invention, it should be noted that, unless otherwise prescribed and limit, term " installation ", " being connected ", " connection " should be interpreted broadly, for example, can be mechanical connection or electrical connection, also can be the connection of two element internals, can be to be directly connected, and also can indirectly be connected by intermediary, for the ordinary skill in the art, can understand as the case may be the concrete meaning of above-mentioned term.

Describe with reference to the accompanying drawings according to the driver fatigue early warning detection system of embodiment of the present invention proposition, there is the automobile of this system and the driver fatigue early-warning detection method for automobile.

Fig. 1 is according to the block diagram of the driver fatigue early warning detection system of the embodiment of the present invention.As shown in Figure 1, this driver fatigue early warning detection system comprises video acquisition module 10, facial recognition modules 20, human eye detection module 30 and judge module 40.

Wherein, video acquisition module 10 is for gathering passenger's image information.In one embodiment of the invention, video acquisition module 10 can be shooting importation.

Facial recognition modules 20 processes to obtain driver's face characteristic parameter to described passenger's image information by gray level co-occurrence matrixes, color space transformation and skin color modeling, and according to the face positional information of driver described in described driver's face characteristic gain of parameter.

Wherein, according to one embodiment of present invention, facial recognition modules 20 processes to obtain described passenger's face texture characteristic information to described passenger's image information according to described gray level co-occurrence matrixes, and identify described driver according to described passenger's face texture characteristic information.

Particularly, in an embodiment of the present invention, in drive r's face recognition process, first by the difference of textural characteristics, can distinguish preferably the face under night.For example adopt gray level co-occurrence matrixes, and extract four relevant characteristic variables, according to the value of characteristic variable, thereby realized the division in people's face skin skin region and other regions, thereby tentatively judge face position, and effect was more obvious night.Wherein, the schematic diagram that textural characteristics face complexion area is cut apart as shown in Figure 2 A.

Wherein, to the mathematical definition of gray level co-occurrence matrixes be: the pixel (x that gray level co-occurrence matrixes is is i from gradation of image value, y) set out, statistics is d with its distance, the pixel (x+a that gray-scale value is j, y+b), the frequency P (i, j, the d that occur simultaneously, θ), its mathematical expression is:

P(i,j,d,θ)={[(x,y),(x+a,y+b)|f(x,y)=i;f(x+a,y+b)=j]}（1）

In formula (1), gray level co-occurrence matrixes be describe in θ direction, a pair of pixel of the d pixel distance of being separated by has respectively the probability of activity i and j.

According to gray level co-occurrence matrixes, can obtain 14 kinds of characteristic quantities, wherein energy, entropy, contrast, these four kinds of features of local stationary can be distinguished people's face skin skin region preferably.

Wherein, the mathematical expression of energy is:

$f_1=\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}{p}^2(i,j)--\left(2\right)$

Wherein, L is gray level.

The mathematical expression of entropy is:

$f_2=\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}\hat{p}(i,j)\log \hat{p}(i,j)--\left(3\right)$

The mathematical expression of contrast is:

$f_3=\underset{k=0}{\overset{L-1}{\mathrm{\Σ}}}{K}^2\left\{\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}\hat{p}(i,j)\right\}--\left(4\right)$

Wherein, k=(i-j) ².

The mathematical expression of local stationary is:

$f_4=\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}\frac{\hat{p}(i,j)}{|1+{(i-j)}^2|}--\left(5\right)$

In an example of the present invention, by textural characteristics face positioning result as shown in Figure 2 B.

In one embodiment of the invention, as shown in Figure 3, the flow process that tentatively realizes the division in people's face skin skin region and other regions according to textural characteristics is:

S301, initialization.

S302, image Compression.

S303, is written into compression and processes rear image parameter.

S304, generates gray level co-occurrence matrixes.

S305, calculates textural characteristics value (for example energy, entropy, contrast and four kinds of features of local stationary).

S306, judges whether face skin area.If so, execution step S307; If not, execution step S308.Wherein, energy, entropy, contrast and four kinds of eigenwerts of local stationary have a threshold value for face complexion, and in the time exceeding this threshold value, are judged to be face.

S307, grey scale pixel value is made as 255, enters step S309.

S308, grey scale pixel value is made as 0, enters step S309.

S309, generates newly figure, shows segmentation result.

S310, finishes.

Then, adopt color space transformation, complexion model is set up, thereby colour of skin similarity calculating face position coordinates hair position judgment accurately to judge the position of face.

Wherein, the conversion of color space, is mapped to YCbCr color space by RGB color space exactly, because mankind's colour of skin can embody preferably the Clustering features of the colour of skin on YCbCr color space.Conversion formula is:

$\left\{\begin{array}{c}Y=0.257R+0.504G+0.098B+16\\ \mathrm{Cb}=-0.1487R-0.291G+0.439B+128\\ \mathrm{Cr}=0.439R-0.368G-0.0071B+128\end{array}\right.--\left(6\right)$

After complexion model YCbCr space normalization chroma histogram, the colour of skin meets dimensional Gaussian model M=(m, C), and wherein m is average, and C is covariance.Can distinguish preferably face and non-face by this complexion model.

According to complexion model, can determine class face complexion area.The similar Y component according to YCbCr color space is determined class hair zones.Distribute according to the position relationship of area of skin color and hair zones, comprehensively judge the position of face.Wherein, face hair zones segmentation result as shown in Figure 4 A, adopts the accurate positioning result of color space skin color modeling method face as shown in Figure 4 B.

In one embodiment of the invention, as shown in Figure 5, the flow process of distinguishing area of skin color and hair zones and accurately locating face according to colour of skin space is:

S501, initialization.

S502, obtains image parameter.

S503, is mapped to YCbCr color space by image from rgb color space.

S504, sets up complexion model.

S505, reads current pixel value.

S506, judges whether skin or hair zones according to current pixel color component relation.If so, execution step S507; If not, execution step S508.

S507, judges whether to belong to skin area according to current pixel color component relation.If so, execution step S509; If not, execution step S510.

S508, is made as 2 by pixel value, enters step S511.

S509, is made as 0 by pixel value, enters step S511.

S510, is made as 1 by pixel value, enters step S511.

S511, obtains and cuts apart rear image result.

S512, according to face position, hair face position relationship location.

S513, finishes.

Therefore say, in an embodiment of the present invention, utilize gray level co-occurrence matrixes and colored colour of skin space orientation face process can extract driver's face characteristic value, thus identification of driver face, if driver can select anti-theft modes for fixing personnel user, play automobile burglar effect.That is to say, the driver fatigue early warning detection system of the embodiment of the present invention adopts intelligent facial recognition techniques, can automatically identify car owner, thereby can prevent that automobile is stolen.

In an embodiment of the present invention, human eye detection module 30 is for obtain described driver's ocular information according to described driver's face positional information, and according to the eye characteristic parameter of driver described in described driver's ocular acquisition of information to detect described driver's eye feature information.

Wherein, according to one embodiment of present invention, human eye detection module 30 carries out according to described driver's face positional information that histogram equalization is processed and subregion processes to lock described driver's ocular, and described driver's ocular is carried out to connected domain processing to obtain described driver's eye position, and extract described driver's eye characteristic parameter according to described driver's eye position.

Specifically, the fundamental purpose that characteristic area extracts is to extract eye feature region, and characteristic area analysis is obtained can be used as the octuple data of the feature of differentiating driver's state of mind.Wherein, whole characteristic area extracts and is divided into three parts: further locking, the extraction of eye areas, feature extraction three parts.The extraction of characteristic area be mainly take face facial zone divide and facial face position relationship, magnitude relationship as main modeling foundation, be aided with UNICOM's territory algorithm simultaneously and carry out, UNICOM's territory algorithm is protection algorithm.

First the further locking of characteristic area.For one, comparatively for the face of standard, face have fixed position separately, and for eyes, its position has been fixed on centre three/part of face.If obtained comparatively accurately after human face region, program just can be searched in a less scope.Concrete way is: first, according to the image collecting, carry out regional compartmentalization, as shown in Figure 6, in the middle of generally can choosing, 1/3 part is extracted and extracted as a further feature region.

For the image of input, first carry out histogram equalization, picture contrast is increased, be conducive to follow-up binaryzation.Algorithm, to image left and right piecemeal, carries out binaryzation subsequently.Carry out division in height according to the general location of human eye, the further locking in realization character region, looks for human eye by face subregion.Particularly, as shown in Figure 7, image denoising and face subregion (1/2 and 1/3) locking flow is:

S701, this part program starts.

S702, histogram equalization processing.

S703, carries out left and right subregion processing to whole face figure.

S704, binary conversion treatment is done respectively in twoth district, left and right.

S705, merges and two-value expansion process.

S706, analytic function connect1(or 2), for example, enter step S708.

S707, finish this part.

S708, the algorithm of analytic function connect2 in this part.

S709, judges that whether face region is for closely square.If so, execution step S710; If not, execution step S711.

S710, the part of getting face middle 1/2, enters step S712.

S711, the part of getting face middle 1/3, enters step S712.

S712, adds up every row Gray Projection value.

S713, the from bottom to top every row Gray Projection of sequential scanning value.

S714, judges whether that Gray Projection value arrives minimum.If so, execution step S715; If not, execution step S721.

S715, judges whether minimum Gray Projection value is border.If so, execution step S716; If not, execution step S717.

S716, its highest line of end mark is minimum row+5.

S717, judges whether to arrive doubtful peak.If so, execution step S718; If not, execution step S719.

S718, judges whether not meet continuation.If so, return to step S713; If not, execution step S720.

S719, returns to step S717 after cumulative line number.

S720, mark highest line.

S721, judges whether Gray Projection value is greater than 5.If so, execution step S722; If not, execution step S723.

S722, mark projection behavior lower limit, execution step S723.

S723, continues to return to step S714 after scanning.

Then, eye areas is extracted.In an embodiment of the present invention, precondition is that the human eye having obtained after further locking extracts region.Ensuing work is exactly the standby processing when verifying and extracting failure.The whatsoever people of appearance, for example, as long as (be seated on driver's seat) human eye spacing always in a fixing scope fixing in the situation that, the difference in height of two is a less side compared to the difference in height of eyebrow to two.It should be pointed out that especially due in the above-mentioned operation of further dwindling hunting zone, the operation of scanning screening is from bottom to top, therefore can be from avoiding to a great extent the interference from eyebrow.For the above-mentioned reasons, these parameters are suitable for judgement, extract human eye.And algorithm has been considered the difference on evening, daytime in compiling procedure, the impact of the situations such as spectacle-frame, eyebrow, torticollis, wherein: owing to being mainly reflective as main take face, scope out of Extract eyes on evening compared to daytime is confined near pupil, and therefore can to compare daytime less for size; The impact of spectacle-frame is to be mainly embodied in spectacle-frame can between two, " produce " connected domain, can cause identification chaotic than being easier to like this.These are all embodied in the concrete parameter of algorithm, mechanism one by one.Wherein, as shown in Figure 8, the algorithm flow of ocular locking is:

S801, algorithm starts.

S802, extracts the two-value data of input according to result previous stage (positional information).

S803, detects and whether also has new connected domain.If so, execution step S804; If not, execution step S807.

S804, the current connected domain all the elements of mark.

S805, judges whether to meet glasses condition.If so, execution step S806; If not, execution step S807.

S806, deletion record, returns to step S803.

S807, entry information gathers array.

S808, judges whether to travel through all information elements.If so, execution step S811; If not, execution step S809.

S809, judges whether eyes condition.If so, execution step S810; If not, return to step S808.

S810, the output first order is found successfully.

S811, finds unsuccessfully.

S812, expands hunting zone.

S813, extracts at the two-value data of input.

S814, detects and whether also has new connected domain.If so, execution step S815; If not, execution step S818.

S815, the current connected domain all the elements of mark.

S816, judges whether to meet glasses condition.If so, execution step S817; If not, execution step S818.

S817, deletion record, returns to step S814.

S818, entry information gathers array.

S819, judges whether to travel through all information elements.If so, execution step S822; If not, execution step S820.

S820, judges whether eyes condition.If so, execution step S821; If not, return to step S819.

S821, the output second level is found successfully.

S822, finds utter failure.

Wherein, connected domain analysis I is the further assurance to eye areas, carry out sequentially with this step of Gray Projection, the connected domain analysis here to as if carry out main binaryzation based on the definite eye areas of Gray Projection, the analysis of positional information, the information being mainly concerned with is whether the height of eyes is unified, and whether spacing is in set scope.

If correct judgment carries out mark by two eye position places immediately, and intercept picture and carry out signature analysis.If be out of one's reckoning, enter immediately connected domain analysis II.

As insurance measure, the mechanism of connected domain analysis II is on the basis of 1/3 human face region that obtains in recognition of face, carries out binaryzation, then finds and meet " whether the height of eyes is unified, spacing whether in set scope " this condition.If still can not find so algorithm thinks that face finds unsuccessfully, find and carry out subsequent treatment by connected domain analysis I.

Then, carry out feature extraction.About feature extraction aspect, consider that the different time in evening on daytime uses two kinds of features, i.e. the not size of bending moment and pupil of HUShi, wherein:

1, not bending moment of HUShi

Moment function has a wide range of applications in graphical analysis, as pattern-recognition, target classification, target identification and orientation estimation, Image Coding and reconstruct etc.A square collection calculating from a width digital picture, has described the global characteristics of this picture shape conventionally, and provides a large amount of about the dissimilar geometrical property information of this image, such as size, position, direction and shape etc.This characteristic description ability of image moment is widely used in the target identification in various image processing, computer vision and Robotics field is estimated with orientation.For example, first moment is relevant with shape, and second moment shows the degree of expansion of curve around straight line mean value, and third moment is the symmetric measurement about mean value.Can derive one group of totally seven bending moment not by second moment and third moment.Can simplify processing to image, retain the information that can reflect target property, then with the image calculation invariant moment features after simplifying, can reduce calculated amount.

For square, it is usually for differentiating the object that morphological differences is larger, so the opening to close can think to exist huge variation under this resolution of characteristic area of eyes, therefore algorithm has adopted this stack features.Below the not computing formula of bending moment of seven of HUShi:

$\begin{array}{c}{\mathrm{\φ}}_1={\mathrm{\η}}_{20}+{\mathrm{\η}}_{02}\\ {\mathrm{\φ}}_2={({\mathrm{\η}}_{20}-{\mathrm{\η}}_{02})}^2+4{\mathrm{\η}}_{11}^2\\ {\mathrm{\φ}}_{3={({\mathrm{\η}}_{30}-3{\mathrm{\η}}_{12})}^2+{({\mathrm{\η}}_{03}-3{\mathrm{\η}}_{21})}^2}\\ {\mathrm{\φ}}_4={({\mathrm{\η}}_{30}+{\mathrm{\η}}_{12})}^2+{({\mathrm{\η}}_{03}+{\mathrm{\η}}_{21})}^2\\ {\mathrm{\φ}}_5=({\mathrm{\η}}_{30}-3{\mathrm{\η}}_{12})({\mathrm{\η}}_{30}+{\mathrm{\η}}_{12})[{({\mathrm{\η}}_{30}+{\mathrm{\η}}_{12})}^2-3{({\mathrm{\η}}_{03}+{\mathrm{\η}}_{21})}^2]+({3\mathrm{\η}}_{21}-{\mathrm{\η}}_{03})({\mathrm{\η}}_{03}+{\mathrm{\η}}_{21})[3{({\mathrm{\η}}_{30}+{\mathrm{\η}}_{12})}^2-{({\mathrm{\η}}_{03}+{\mathrm{\η}}_{21})}^2]\\ {\mathrm{\φ}}_6=({\mathrm{\η}}_{20}-{\mathrm{\η}}_{02})[{({\mathrm{\η}}_{30}+{\mathrm{\η}}_{12})}^2-{({\mathrm{\η}}_{03}+{\mathrm{\η}}_{21})}^2]+{4\mathrm{\η}}_{11}({\mathrm{\η}}_{30}+{\mathrm{\η}}_{12})({\mathrm{\η}}_{03}+{\mathrm{\η}}_{21})\\ {\mathrm{\φ}}_7=({3\mathrm{\η}}_{21}-{\mathrm{\η}}_{03})({\mathrm{\η}}_{03}+{\mathrm{\η}}_{21})[{({\mathrm{\η}}_{30}+{\mathrm{\η}}_{12})}^2-3{({\mathrm{\η}}_{03}+{\mathrm{\η}}_{21})}^2]+({\mathrm{\η}}_{30}-{3\mathrm{\η}}_{12})({\mathrm{\η}}_{30}+{\mathrm{\η}}_{12})[3{({\mathrm{\η}}_{30}+{\mathrm{\η}}_{12})}^2-{({\mathrm{\η}}_{03}+{\mathrm{\η}}_{21})}^2]\end{array}$

${\mathrm{\η}}_{\mathrm{pq}}=\frac{{\mathrm{\μ}}_{\mathrm{pq}}}{{\mathrm{\μ}}_{00}^{\mathrm{\γ}}},\mathrm{\γ}=\frac{p+q}{2}+1---\left(7\right)$

Wherein, μ _pqweip+qJie center square, ${\mathrm{\η}}_{\mathrm{pq}}=\frac{{\mathrm{\μ}}_{\mathrm{pq}}}{{\mathrm{\μ}}_{1+\frac{p+q}{2}00}},p+q\≥2.$

2. the size of pupil

According to perclos standard, can determine fatigue state by the ratio of calculating eyes closed in a period of time.In conjunction with the restriction (figure) of actual imaging means, then image quality is used this feature and take this standard as criterion good night for pupil.

Wherein, the measuring method of perclos is: capture driver's face image with video camera, obtain eye image by image processing method, determine that through the means of graphical analysis and identification eyes are opened or closure.It is to open that definition eye pupil aperture is greater than 20%, and pupil aperture equals 20% or less of closed.The ratio of the time that eyes make and break is opened than the time and the eyes that are eyes closed in Measuring Time, the time is to go out by the Time Calculation of every two field picture processing.

Herein, owing to there not being the problem of calculating area,, add up within a certain period of time as independent variable with the state of mind classification value of neural network output, the shared ratio of opening, close one's eyes, as ratio is greater than threshold value, system provides feedback.

The parameter of specifically choosing in the time using perclos criterion is that the width of pupil region is as the 8th characteristic quantity, because it is constant that this parameter is not yardstick, thereby the width that therefore need to gather in advance the pupil region of a few minutes in the time of practical operation is realized follow-up Real-Time Monitoring in the hope of maximal value.

In a concrete example of the present invention, the contrast difference facial image situation in night by pupil image sharpness and daytime facial image situation.Wherein, night facial image situation as shown in Figure 9 A, daytime facial image situation as shown in Figure 9 B.

3, algorithm is realized

This part algorithm is mainly in principal function, and the result of finding according to human eye in principal function is, having, to process.Carry out in turn the locking of human eye area, binaryzation, determines pupil width, finally the constant moment function of image input HUShi after binaryzation is processed and is obtained remaining 7 degree of freedom yardstick invariant features.Wherein, as shown in figure 10, seven of HUShi not bending moment calculation process are:

S1001, algorithm starts.

S1002 extracts ocular from colour original.

S1003, dwindles region of search to original 1/4.

S1004, binary conversion treatment.

S1005, all row of traversal within the scope of from h/5 to 4h/5.

S1006, tries to achieve breadth extreme.

S1007, inputs HUShi moment function by binary image.

S1008, obtains result.

In an embodiment of the present invention, judge module 40 is for comparing to judge that according to described driver's eye feature information and default eye feature threshold value whether described driver is in fatigue state.

Wherein, according to one embodiment of present invention, judge module 40 is according to perclos standard and BP(Back Propagation, backpropagation) neural network algorithm judges that whether described driver is in described fatigue state.

Last judge module 40 carries out Classification and Identification according to perclose and BP neural network.Carry out examining of octuple data according to correct result in the result of BP judgement, utilize the pupil width maximal value that pupil width and first few minutes are measured to judge, peaked 0.7 if width is less than standard, continue to be judged to the situation of closing one's eyes.Wherein, BP neural network basic structure as shown in figure 11.

According to one embodiment of present invention, as shown in Figure 1, above-mentioned driver fatigue early warning detection system also comprises: warning module 50, described driver during in described fatigue state warning module 50 send alarm.Meanwhile, warning module 50 can with the engine controller ECU both-way communication of automobile, urgent in the situation that pro-active intervention ECU(for example brake, deceleration, bright light etc.).

Wherein, integrated part in the driver fatigue early warning detection system of the embodiment of the present invention can be divided into video acquisition and report to the police two parts of feedback, although they do not play a key effect in program, as realizing the part of trying to make a match in whole algorithm, need to describe.

As shown in figure 12, the integration flow process of integrated part comprises:

S1201, algorithm starts.

S1202, camera is affiliated on interface.

S1203, arrange shooting image parameter.

S1204, judges that whether camera is online.If so, execution step S1205; If not, return to step S1204, continue judgement.

S1205, arranges video recording parameter.

S1206, carries out call back function.

S1207, disassembles data.

S1208, YUV2 image format conversion becomes RGB picture format.

S1209, storage picture.

S1210, shows feedback interface in real time.

Wherein, algorithm has adopted the mode of zone bit array on carrying out, and carries out Comprehensive Assessment according to zone bits all in 60 frames, carries out the judgement of state according to 0/1/2 number in zone bit array: it is exactly nobody state onboard above that ' 0 ' accounts for 40 frames; ' 1 ' account for 30 frames be exactly above the state of current driver's be abnomal condition; ' 2 ' account for 30 frames be exactly above the state of current driver's be normal condition.Based on above result, driver's state is fed back.Must describe in addition, camera needs initialization, and therefore whole program is not processed image in the initialized process of camera.

After the driver fatigue early warning detection system study of the embodiment of the present invention finishes and is stable, user can select the special interface through CAN communication access ECU, in case of emergency pro-active intervention ECU(brake, deceleration, two sudden strains of a muscle etc.).

Therefore, the driver fatigue early warning detection system of the embodiment of the present invention is interactive software system, relies on DSP platform, has detecting driver face state, the function of the state of mind while driving to detect it.Wherein, as shown in Figure 1, the driver fatigue early warning detection system of the embodiment of the present invention can judge that four parts form by video acquisition, recognition of face, human eye detection, state.And the top level diagram of this driver fatigue early warning detection system internal module as shown in figure 13.

In sum, the driver fatigue early warning detection system of the embodiment of the present invention has adopted light conditions monitoring in autonomous car to revise, and can carry out the automatic correction of human face light in car; And the intelligent facial recognition techniques of sampling, can automatically identify car owner, thereby can there is anti-theft feature; And implanted new human eye detection mechanism, and learn different drivers' fatigue process and action and remember its fatigue characteristic by neural network, make it to have more adaptability, intellectuality; Realized and the communicating by letter of ECU, client can select the both-way communication with ECU, in case of emergency pro-active intervention ECU(brake, deceleration, two sudden strains of a muscle etc. simultaneously); In addition, can demarcate for different automobile types that to make it accuracy rate higher, be highly suitable for main engine plants and coordinate to demarcate and concentrate assembling.Therefore, the driver fatigue early warning detection system of the embodiment of the present invention makes capable of giving fatigue pre-warning more intelligent, has improved adaptability, and it is antitheft to carry out to identify car owner's identity, can participate in car networking, automobile is had more intelligent.

According to the driver fatigue early warning detection system of the embodiment of the present invention, gather passenger's image information by video acquisition module, and by facial recognition modules to passenger's image information analyzing and processing to identify driver's face characteristic parameter, then obtain driver's face positional information, human eye detection module is finally obtained driver's eye characteristic parameter to detect driver's eye feature information according to driver's face positional information, last judge module judges that according to driver's eye feature information whether driver is in fatigue state.Therefore, the driver fatigue early warning detection system of the embodiment of the present invention can be identified driver in car automatically, and can learn different drivers' fatigue process and action, and memory driver's fatigue characteristic, whether judge driver in fatigue state, thereby realize, the absorbed state of mind of driver is carried out to early warning, guarantee driving safety, meet user's needs, and reduce False Rate.

In addition, embodiments of the invention have also proposed a kind of automobile, and it comprises above-mentioned driver fatigue early warning detection system.

According to the automobile of the embodiment of the present invention, can automatically identify driver in car, and can learn different drivers' fatigue process and action, and memory driver's fatigue characteristic, whether judge driver in fatigue state, thereby realize, the absorbed state of mind of driver is carried out to early warning, guarantee driving safety, meet user's needs.

Figure 14 is according to the process flow diagram of the driver fatigue early-warning detection method for automobile of the embodiment of the present invention.As shown in figure 14, this driver fatigue early-warning detection method that is used for automobile comprises the following steps:

S1, collection passenger's image information.

S2, processes to obtain driver's face characteristic parameter to passenger's image information by gray level co-occurrence matrixes, color space transformation and skin color modeling, and according to driver's face characteristic gain of parameter driver's face positional information.

According to one embodiment of present invention, in step S2, according to gray level co-occurrence matrixes, passenger's image information is processed to obtain described passenger's face texture characteristic information, and identify driver according to passenger's face texture characteristic information.

Wherein, concrete face recognition, face location are described above, are just no longer described in detail here.

S3, obtains driver's ocular information according to driver's face positional information, and according to driver's ocular acquisition of information driver's eye characteristic parameter to detect driver's eye feature information.

According to one embodiment of present invention, in step S3, carry out according to driver's face positional information that histogram equalization is processed and subregion processes to lock driver's ocular, and driver's ocular is carried out to connected domain processing to obtain driver's eye position, and extract driver's eye characteristic parameter according to driver's eye position.

Wherein, concrete human eye detection process is described above, is just no longer described in detail here.

S4, compares to judge that according to driver's eye feature information and default eye feature threshold value whether driver is in fatigue state.

According to one embodiment of present invention, in step S4, judge that according to perclos standard and BP neural network algorithm whether driver is in fatigue state.

Similarly, the process of carrying out Classification and Identification according to perclose and BP neural network is described above, repeats no more here.

According to the driver fatigue early-warning detection method for automobile of the embodiment of the present invention, by gathering passenger's image information, and to passenger's image information analyzing and processing to identify driver's face characteristic parameter, then obtain driver's face positional information, finally obtain driver's eye characteristic parameter according to driver's face positional information to detect driver's eye feature information, finally judge that according to driver's eye feature information whether driver is in fatigue state.Therefore, the driver fatigue early-warning detection method for automobile of the embodiment of the present invention can be identified driver in car automatically, and can learn different drivers' fatigue process and action, and memory driver's fatigue characteristic, whether judge driver in fatigue state, thereby realize, the absorbed state of mind of driver is carried out to early warning, guarantee driving safety, and can reduce False Rate.

Any process of otherwise describing in process flow diagram or at this or method are described and can be understood to, represent to comprise that one or more is for realizing module, fragment or the part of code of executable instruction of step of specific logical function or process, and the scope of the preferred embodiment of the present invention comprises other realization, wherein can be not according to order shown or that discuss, comprise according to related function by the mode of basic while or by contrary order, carry out function, this should be understood by embodiments of the invention person of ordinary skill in the field.

The logic and/or the step that in process flow diagram, represent or otherwise describe at this, for example, can be considered to the sequencing list of the executable instruction for realizing logic function, may be embodied in any computer-readable medium, use for instruction execution system, device or equipment (as computer based system, comprise that the system of processor or other can and carry out the system of instruction from instruction execution system, device or equipment instruction fetch), or use in conjunction with these instruction execution systems, device or equipment.With regard to this instructions, " computer-readable medium " can be anyly can comprise, device that storage, communication, propagation or transmission procedure use for instruction execution system, device or equipment or in conjunction with these instruction execution systems, device or equipment.The example more specifically (non-exhaustive list) of computer-readable medium comprises following: the electrical connection section (electronic installation) with one or more wirings, portable computer diskette box (magnetic device), random-access memory (ram), ROM (read-only memory) (ROM), the erasable ROM (read-only memory) (EPROM or flash memory) of editing, fiber device, and portable optic disk ROM (read-only memory) (CDROM).In addition, computer-readable medium can be even paper or other the suitable medium that can print described program thereon, because can be for example by paper or other media be carried out to optical scanning, then edit, decipher or process in electronics mode and obtain described program with other suitable methods if desired, be then stored in computer memory.

Should be appreciated that each several part of the present invention can realize with hardware, software, firmware or their combination.In the above-described embodiment, multiple steps or method can realize with being stored in software or the firmware carried out in storer and by suitable instruction execution system.For example, if realized with hardware, the same in another embodiment, can realize by any one in following technology well known in the art or their combination: there is the discrete logic for data-signal being realized to the logic gates of logic function, there is the special IC of suitable combinational logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) etc.

Those skilled in the art are appreciated that realizing all or part of step that above-described embodiment method carries is can carry out the hardware that instruction is relevant by program to complete, described program can be stored in a kind of computer-readable recording medium, this program, in the time carrying out, comprises step of embodiment of the method one or a combination set of.

In addition, the each functional unit in each embodiment of the present invention can be integrated in a processing module, can be also that the independent physics of unit exists, and also can be integrated in a module two or more unit.Above-mentioned integrated module both can adopt the form of hardware to realize, and also can adopt the form of software function module to realize.If described integrated module realizes and during as production marketing independently or use, also can be stored in a computer read/write memory medium using the form of software function module.

The above-mentioned storage medium of mentioning can be ROM (read-only memory), disk or CD etc.

In the description of this instructions, the description of reference term " embodiment ", " some embodiment ", " example ", " concrete example " or " some examples " etc. means to be contained at least one embodiment of the present invention or example in conjunction with specific features, structure, material or the feature of this embodiment or example description.In this manual, the schematic statement of above-mentioned term is not necessarily referred to identical embodiment or example.And specific features, structure, material or the feature of description can be with suitable mode combination in any one or more embodiment or example.

Although illustrated and described embodiments of the invention, for the ordinary skill in the art, be appreciated that without departing from the principles and spirit of the present invention and can carry out multiple variation, modification, replacement and modification to these embodiment, scope of the present invention is by claims and be equal to and limit.

Claims (10)

1. a driver fatigue early warning detection system, is characterized in that, comprising:

Video acquisition module, for gathering passenger's image information;

Facial recognition modules, described facial recognition modules processes to obtain driver's face characteristic parameter to described passenger's image information by gray level co-occurrence matrixes, color space transformation and skin color modeling, and according to the face positional information of driver described in described driver's face characteristic gain of parameter;

Human eye detection module, for obtain described driver's ocular information according to described driver's face positional information, and according to the eye characteristic parameter of driver described in described driver's ocular acquisition of information to detect described driver's eye feature information;

Judge module, for comparing to judge that according to described driver's eye feature information and default eye feature threshold value whether described driver is in fatigue state.

2. driver fatigue early warning detection system as claimed in claim 1, it is characterized in that, described facial recognition modules processes to obtain described passenger's face texture characteristic information to described passenger's image information according to described gray level co-occurrence matrixes, and identify described driver according to described passenger's face texture characteristic information.

3. driver fatigue early warning detection system as claimed in claim 1, it is characterized in that, described human eye detection module carries out according to described driver's face positional information that histogram equalization is processed and subregion processes to lock described driver's ocular, and described driver's ocular is carried out to connected domain processing to obtain described driver's eye position, and extract described driver's eye characteristic parameter according to described driver's eye position.

4. driver fatigue early warning detection system as claimed in claim 1, is characterized in that, described judge module judges that according to perclos standard and BP neural network algorithm whether described driver is in described fatigue state.

5. the driver fatigue early warning detection system as described in claim 1-4 any one, is characterized in that, also comprises:

Warning module, described in described driver is during in described fatigue state, warning module sends alarm.

6. an automobile, is characterized in that, comprises the driver fatigue early warning detection system as described in claim 1-5 any one.

7. for a driver fatigue early-warning detection method for automobile, it is characterized in that, comprise the following steps:

S1, collection passenger's image information;

S2, by gray level co-occurrence matrixes, color space transformation and skin color modeling, described passenger's image information is processed to obtain driver's face characteristic parameter, and according to the face positional information of driver described in described driver's face characteristic gain of parameter;

S3, obtains described driver's ocular information according to described driver's face positional information, and according to the eye characteristic parameter of driver described in described driver's ocular acquisition of information to detect described driver's eye feature information;

S4, compares to judge that according to described driver's eye feature information and default eye feature threshold value whether described driver is in fatigue state.

8. the driver fatigue early-warning detection method for automobile as claimed in claim 7, it is characterized in that, in step S2, according to described gray level co-occurrence matrixes, described passenger's image information is processed to obtain described passenger's face texture characteristic information, and identify described driver according to described passenger's face texture characteristic information.

9. the driver fatigue early-warning detection method for automobile as claimed in claim 7, it is characterized in that, in step S3, carry out according to described driver's face positional information that histogram equalization is processed and subregion processes to lock described driver's ocular, and described driver's ocular is carried out to connected domain processing to obtain described driver's eye position, and extract described driver's eye characteristic parameter according to described driver's eye position.

10. the driver fatigue early-warning detection method for automobile as claimed in claim 7, is characterized in that, in step S4, judges that according to perclos standard and BP neural network algorithm whether described driver is in described fatigue state.

Images