CN101941425B - Intelligent recognition device and method for fatigue state of driver - Google Patents

Abstract

The invention relates to intelligent recognition device and method for the fatigue state of a driver in the technical field of control in automobile engineering. The device comprises a pinhole camera, a signal processor, an automobile speed control mechanism and a voice player, wherein the output interface of the pinhole camera is connected with the image input interface of the signal processor through a video cable, and the output interface of the signal processor is respectively connected with the digital input interfaces of the automobile speed control mechanism and the voice player. The method comprises the following steps of: 1. learning a training sample by a system; and 2. online recognizing the fatigue state of the driver, wherein the step 1 comprises the steps of driver fatigue-state face image acquisition and color space conversion, human face central position and eye area determination, training sample characteristic value and characteristic vector computation and the like; and the step 2 comprises the steps of driver face image real-time acquisition and color space conversion, automatic determination of driver face central positions and eye areas, tracking for the driver eye areas, testing sample characteristic vector computation, driver fatigue-state recognition, vehicle control decision-making and the like. The invention ensures that the driving state of the driver can be accurately recognized in the driving process of vehicles, and the fatigue driving of the driver is prevented, so that traffic accidents caused by the fatigue of the driver can be effectively avoided.

Description

Intelligent identification device and method to driver fatigue state

Technical field

What the present invention relates to is the apparatus and method in the control technology field in a kind of automotive engineering, specifically is a kind of intelligent identification device and method to driver fatigue state.

Background technology

As everyone knows, when chaufeur is driven over a long distance automobile, can be because of the physiological fatigue that is difficult to overcome occurrence of traffic accident often.Particularly the long distance haulage truck driver continues to drive more than tens hours from the selfless health of consideration meeting of economic interests.This type automobile carrier driver tends to occur under steam sleepy phenomenon on the way, therefore leads to painful car crash traffic accident.According to statistics, China in 2007 is because the dead number of traffic accident has reached people more than 80,000, and the traffic safety present situation allows of no optimist.U.S.'s road traffic accident statistics showed in 2007, had more than 1400 owing to the fatigue driving reason directly causes dead accident number of times.

World Health Organigation's research report points out that annual traffic accident seizes the life near 1,200,000, accounts for 2.2% of annual global death toll, causes occupying the 9th in the dead reason at all.If current traffic is not improved, expect the year two thousand thirty, the shared proportion of toll on traffic will reach 3.6%, will in all causes of death, rise to the 5th.And the trend of continuous rising arranged.

The research report of American National sleep foundation points out that U.S. every year is owing to the traffic accident number that the driver fatigue reason causes on average has 100,000 many cases.

Reason analysis to a large amount of traffic accidents shows that in driving fatigue, the perception of chaufeur is tired, judgement decision-making fatigue is to start the major cause of sending out traffic accident.Japan discovers more than 38000 accident reasons: perception fatigue accounts for 40.1%, judges that decision-making fatigue is 41.5%.

Driving fatigue influences the vigilance and the safe driving ability of chaufeur.The driving fatigue problem has caused common people's concern, and western developed country drops into the research that huge human and material resources are extensively carried out driving fatigue.

Present situation that China's driving fatigue monitoring method falls behind and severe traffic safety situation, also an urgent demand solves the difficult problem in the driving fatigue monitoring technology.

The driver fatigue Recognition Technology Research mainly concentrates on three aspects:

(1) based on the monitoring method of chaufeur individual character, the for example activity of eyelid, eyes closed, nodding action etc.;

(2) based on the monitoring method of chaufeur physiological parameter measurement, for example electroencephalogram, electromyogram, electromyogram, muscle activity etc.;

(3) based on the monitoring method of vehicle parameter, the for example speed of a motor vehicle, acceleration/accel, vehicle location etc.

Overcome the phenomenon of driver tired driving, except strengthening the production control to necessary traffic legal education of chaufeur and communications and transportation enterprise, relying on advanced science and technology to improve and having the vehicle security drive intelligent functions now then is vital technological means.Here said driver fatigue state mainly is meant the doze state that chaufeur demonstrates because of physiological fatigue.

Through the retrieval of prior art document is found, " a kind of recognition methods that is used for human eye location and human eye opening and closing " of Lu Yao etc. (one Chinese patent application number: 200510027371.X) introduced a kind of " human eye is located and the recognition methods of human eye opening and closing ".This method mainly solves the problem to the human eye identification of dynamic image.The steps include: with the camera Dynamic Extraction to a frame image utilize grey level histogram to carry out automatic gray balance, people's face is revealed to come from the background convexity, utilize adjustable half window thresholding that people's face is extracted from background again; Human eye pixel block size according to estimation; Remove non-human eye area, combine the definite people's of two-dimensional geometry relation of human eye eyes then, on original image, show with black surround; If do not detect eyes, the system sounds prompting; Utilize the size of eyes pixel again, judge opening of eyes with closed; If eyes open, will there be black surround to show that program is not sent prompt tone on the original image; If eyes are closed, will there be black surround to show that program is sent prompt tone on the original image.This method is intended and is applied to multiple checking system, like the fatigue driving warning.This method and technology defective is: " according to the human eye pixel block size of estimation; remove non-human eye area; combine the two-dimensional geometry relation of human eye to confirm people's eyes then ", the method accuracy rate of this identification human eye area is lower, and the time regular meeting judge by accident; Reason is: human eye is not of uniform size, hair density difference dressing colour contrast very big, the people is very big, is a kind of extremely insecure technology therefore; It is more to be suitable for limiting condition, as this method self " specification manual " said " human eye can not be blocked "; Background there is requirement etc.Therefore, this technical method is difficult to directly apply to the Intelligent Recognition process to driver fatigue state.

Find through retrieval that again " driver fatigue based on eye state identification is monitored in real time " (" automotive engineering " o. 11th in 2008) of Cheng Bo, Zhang Guangyuan, Feng Ruijia etc. proposed a kind of driver fatigue state of discerning based on eye state method of monitoring in real time to the prior art document.At first confirm face's scope, detect the region of search of confirming eyes through binaryzation and profile then through the barycenter that calculates the difference image that adds up background and present frame.Utilizing after heuristic rule screens the location, calculating the degree of opening that distance between eyes skeleton curve and the two canthus lines obtains eyes.Through calculate corresponding tired index such as PERCLOS, on average open eyes degree, slot mesh eyeball closing period inferred the driver fatigue state.Subjective scoring with the facial video of chaufeur is estimated method of inspection as estimating foundation, and the result shows that all there is significant difference in above-mentioned 3 indexs under different tired grades, can reach fatigue detecting accuracy rate preferably through the fusion to different indexs.The technological deficiency of this method mainly finds expression in: " confirming the region of search of eyes " will spend more computing time; With " tired index such as PERCLOS, on average open eyes degree, slot mesh eyeball closing period inferred the driver fatigue state ", it is not high that it detects accuracy rate.

Summary of the invention

The objective of the invention is to overcome deficiency of the prior art; A kind of intelligent identification device and method to driver fatigue state is provided; Guarantee that vehicle is at driving process; Accurately the state of driving of identification of driver prevents driver fatigue to drive, and therefore can avoid the traffic accident that causes because of driver fatigue effectively.

The present invention realizes through following technical scheme:

The present invention relates to a kind of intelligent identification device, comprising: pinhole cameras, signal processor, speed of a motor vehicle operating-controlling mechanism and speech player driver fatigue state.The output interface of pinhole cameras is connected with the image input interface of signal processor through vision cable, and the output interface of signal processor is connected with the input interface of speech player numeral is parallel with speed of a motor vehicle operating-controlling mechanism respectively.

Said pinhole cameras is arranged in the operator's compartment, faces the face of chaufeur, is used for gathering in real time the facial view of chaufeur.

Said signal processor comprises: image input interface, analog-to-digital conversion module, image processing module, GES input interface, output interface.Wherein: the input end of image input interface links to each other with the pinhole cameras output interface through vision cable; The mouth of image input interface is connected with the input end of analog-to-digital conversion module, and the output port of analog-to-digital conversion module is connected with the input port of image processing module; The input end of GES input interface is connected with the output interface of vehicle speed sensor, and the mouth of GES input interface is connected with the input port of image processing module.Described vehicle speed sensor utilizes the existing vehicle number word rate of vehicle sensor.

Said image processing module is the core technology module in the signal processor, bears the whole calculating process of processing, identification and the decision-making of graphicinformation.

Said speed of a motor vehicle operating-controlling mechanism comprises: input interface, first D and A converter, power amplifier, electromagnetic valve and electric brake push rod.Wherein, The input port of input interface is connected with the output interface of signal processor; The output port of input interface is connected with the input port of first D and A converter; The output port of first D and A converter is connected with the input port of power amplifier, the output port of power amplifier simultaneously with two end points of two end points of electromagnetic valve coil and electric brake push rod coil mutually and connect.

Described electromagnetic valve coil is enclosed within the outside of electromagnetic core; Produce magnetic force through electromagnetic core when magnet coil receives electricity valve is produced magnetic attraction, along with the size variation that is added in magnet coil two-end-point voltage signal, the magnetic attraction of electromagnetic core produces corresponding variation simultaneously; The magnetic attraction of electromagnetic core acts on valve, pulling valve to change the aperture of valve; Resistance spring is a kind of extension spring, and valve is in the pulling of electromagnetic core magnetic attraction, and resistance spring is also stretched simultaneously; Therefore produce an elastic force opposite with the electromagnetic core magnetic attraction; When electromagnetic core magnetic attraction and resistance spring elastic force reached balance, valve just was stopped pulling, and promptly valve opening is with to be added in magnet coil two-end-point voltage corresponding.

Said electric brake push rod; Comprise: electric brake coil and electromagnetism push rod; The electric brake mounting coil is at an end of electromagnetism push rod; The other end of electromagnetism push rod is connected with footbrake bar thick stick mechanism, and when the electric brake coil received electricity, the electromagnetic field that the electric brake coil is produced produced axial mechanical thrust to the push rod that is sleeved in the electric brake coil; This axial mechanical thrust acts on the electric pushrod of footbrake bar thick stick mechanism and does on the force, plays with the same effect of footbrake making the vehicle self-actuating brake through bar thick stick mechanism.

Said voice prompting device comprises: input interface, decoder, digital voice module, second D and A converter, power amplifier module, loud speaker; Wherein: the input port of input interface is connected with the output interface of signal processor; The mouth of input interface is connected with the input port of decoder; The output port of decoder is connected with the input port of digital voice module; The output port of digital voice module is connected with the input port of second D and A converter, and the output port of second D and A converter is connected with the input port of power amplifier module, and the output port of power amplifier module is connected with the input port of loud speaker.After the input interface of voice prompting device receives control command; Explanation through decoder; Related voice unit in the link digital voice module; Voice unit sequence after the link is transported to second D and A converter successively convert voice analog signal into, again voice analog signal is transported to power amplifier module and send the relevant voice caution through power gain rear drive loud speaker.

In the normal vehicle operation of the present invention, electromagnetic valve is in full-gear, and promptly aperture is 100%; When apparatus of the present invention were discerned the place ahead spacing less than safe distance between vehicles, electromagnetic valve coil was under the effect of input voltage signal, and electromagnet produces corresponding magnetic force pulling valve and reduces its original aperture, has therefore reduced flow fuel, forces car retardation; Simultaneously, electric brake push rod coil produces the transmission of torque of axial thrust through bar thick stick mechanism to push rod and drives foot brake and force vehicle to slow down gradually and finally stop also under the effect of this input voltage signal.

After signal processor image input interface of the present invention receives chaufeur face-image analog signal from the pinhole cameras output interface; Image analoging signal is sent into the input end of analog-to-digital conversion module, and the data image signal after analog-to-digital conversion module will be changed is again transported to image processing module respectively; The output signal of vehicle speed sensor inputs to image processing module through the GES input interface.After the chaufeur face image signal that signal processor collects pinhole cameras is handled and analyzed, confirm whether current driver's is in fatigue driving state; In case the affirmation driver tired driving, signal processor can generate control command by its output interface output with recognition result in real time; Under the effect of control command, make car retardation and even brake automatically through speed of a motor vehicle operating-controlling mechanism.Simultaneously, warn to chaufeur through speech player: " you have been in fatigue state, for you and everybody safety, the rest of please stopping! ".

The invention still further relates to a kind of intelligent identification Method, may further comprise the steps driver fatigue state:

Step 1, system are learnt training sample;

Said system is learnt training sample, and promptly apparatus of the present invention system is in learning state.When system was in learning state, system gathered and handles current driver fatigue state face image.Said driver fatigue state face image is promptly because of the tired chaufeur eyes image that causes drowsiness.Step 1 comprises following step by step:

(1) gathers driver fatigue state face image and carry out color space conversion;

After the driver fatigue state face image that pinhole cameras sampling is collected is strengthened, again it is expressed from RGB color space conversion to HSV color space; And continuous acquisition several, comprising: the different images constantly of a plurality of driver fatigue states.As: after gathering three chaufeurs and strengthening, express from RGB color space conversion to HSV color space again at fatigue state face image that different timetables reveal and with it.

(2) eye and other zone and background are made a distinction;

In the driver fatigue state face image that sampling collects; Scan the scenery tone that detects pixel from left to right, from top to bottom according to pixel coordinate; And will fall between the tone zone of HSV color space [2 °; 47 °] tone gather pairing pixel and draft and be human face region, therefore follow human face region to make a distinction exactly other zone of personage and image background.

This is that face complexion is distributed in different angular regions with clothing and background tone separately, and the tone of face complexion concentrates on certain angular regions in the HSV color space relatively stablely because face complexion in the character image and clothing and background exist visibly different tone difference; No matter through experiment confirm is natural lighting or artificial light source irradiation; No matter the also similarities and differences of camera system, the angle distribution of the tone H of the colour of skin all remain between 2～47 ° of HSV color space basically, therefore can distinguish face complexion and clothing, background and other scenery through the tone value of character image in the HSV space; In other words; Having only the tone when a certain scenery to be within the interval [2 °, 47 °], just might be face complexion; Otherwise be other scenery, like clothing or other article.Through testing further confirmation: the tone value of face complexion is that 11 ° probability is the highest, therefore claims that 11 ° tone value is the probability peak of face complexion.When order, face complexion tone were P (H) at the distribution probability of interval [2 °, 47 °], H=11 ° probability reached the highest, i.e. P (11 °)=P _Max, that is to say, when the tone of certain scenery is 11 °, assert that this scenery is that the confidence level of face complexion reaches the highest.

(3) the driver fatigue state face image that sampling is collected is confirmed people's face center and ocular;

Utilize the HSV color space of the driver fatigue state face image that sampling collects to express, in the set of face complexion tone, with near the pairing pixel coordinate of 11 ° tone value as people's face center-point; As: through the human face region search result, the tone set that obtains to fall into [2 °, 47 °] between the tone zone of HSV color space for ...; 9.7 °, 10.1 °, 9.5 °; ..., and should the set pairing pixel coordinate do ..., (i _K-1, j _K-1), (i _k, j _k), (i _K+1, j _K+1) ..., be 10.1 ° near 11 ° tone value wherein, pairing pixel coordinate is (i _k, j _k), therefore just can confirm (i _k, j _k) be people's face center position coordinates, the row coordinate of i remarked pixel, the row-coordinate of j remarked pixel, footnote are represented columns and line number, i _kK represent k row, j _kK represent that k is capable.And with people's face center-point be basic point upwards expand the capable pixel of u and to both sides each the expansion row pixel, as the ocular of u * v.

(4) import driver fatigue state face image ocular training sample;

With importing apparatus of the present invention system as driver fatigue state face image ocular training sample after the ocular intercepting of u * v.

The driver fatigue state face image ocular training sample of being gathered all has 256 gray levels, generally gets number of training n=k * l >=9; Wherein, K, l are respectively by the sampled sample number of driver fatigue state face image ocular of sampling chaufeur number and each chaufeur, and representing respectively by sampling chaufeur number like k=3, l=3 is that the sampled sample number of driver fatigue state face image ocular of 3, each chaufeur is 3.

(5) convert training sample image into one-dimensional vector;

The driver fatigue state face image ocular training sample image data conversion that each is two-dimentional is the vector of one dimension, and the definition " sleepy shape " be 1 type of eye feature, " non-sleepy shape " is-1 type of eye feature.Therefore can give expression to the one-dimensional vector x of i image _iFor

x _i=[x _I1x _I2... x _Im] ^T=[x _Ij] ^T(formula one)

In the formula, x _IjJ grey scale pixel value of 1 type i sample of expression; I=1,2 ..., n is 1 type of eye sample sequence number; J=1,2 ..., m is each sample image institute capture prime number, m=u * v, u and v are respectively the row and the row pixel count of sample image.

(6) calculation training sample characteristics and proper vector;

Calculate 1 type average

promptly

$\stackrel{\‾}{x}=\frac{1}{n\×m}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}\underset{j=1}{\overset{m}{\mathrm{\Σ}}}{x}_{\mathrm{Ij}}$ (formula two)

Claim that the average

of trying to achieve thus is 1 type of average eye.

Can be expressed as after above-mentioned training sample standardized

$v_i=x_i-\stackrel{\‾}{x};$ I=1,2 ..., n (formula three)

The 1 type of average eye normalization vector v that forms by training sample

V=[v ₁v ₂... v _n] ^T(formula three)

At this moment, 1 type of average eye covariance matrix does

Q=[v ₁v ₂... v _n] ^T[v ₁v ₂... v _n]; Q ∈ R ^{N * n}(formula four)

Utilize (formula four) to ask for the eigenvalue of Q _lAnd proper vector, and it is vectorial that it is arranged the back generating feature from big to small again

P=[λ ₁λ ₂λ ₃...] ^T(formula five)

Wherein, λ ₁>=λ ₂>=λ ₃>=...

(7) training sample is carried out projecting to feature space after the linear transformation;

Because bigger eigenwert characteristic of correspondence vector has comprised more people's face eye feature information, the vector space of s the pairing proper vector formation of bigger eigenwert just can be represented the main information of people's face eyes image approx before therefore choosing.The s value is confirmed by experiment.

For the n in the image library image x _i=[x _I1x _I2... x _Im] ^T(i=1,2 ..., n) can obtain projection vector Ω to this eigenspace projection _i=[ω _I1ω _I2... ω _Im] ^T

From v=[v ₁v ₂... v _n] ^TIn choose before the bigger pairing standardization value of eigenwert of s constitute new normalization vector

$\hat{v}={\left[\begin{array}{cccc}{v}_1& {\mathrm{<figure-callout\; id="2"\; label="v"\; filenames="HDA0000026510220000011.png"\; state="\{\{state\}\}">v</figure-callout>}}_2& ...& v_s\end{array}\right]}^T$ (formula six)

Therefore; Can directly represent 1 type of people's face eye feature, i.e. driver fatigue state face image ocular characteristic with

.

In other words, set up people's face eye feature normalization vector after, just can be successively as identification of driver whether because of the tired criterion that presents sleepy shape.

Described figure image intensifying is meant: adopt the Pulse Coupled Neural Network method simulation neuron synchronous behavior relevant with characteristic to show the link model of pulse granting phenomenon, the image that pinhole cameras in the operator's compartment is collected in real time strengthens.

Described Pulse Coupled Neural Network method (Pulse-Coupled Neural Networks is called for short PCNN) is that the link model of pulse granting phenomenon is showed in a kind of simulation neuron synchronous behavior relevant with characteristic.Therefore, it and the neural perception of visual sense have natural getting in touch.

Be applied in the PCNN structure mode of image processing each pixel f of pending image (i, j) corresponding each neuron N _Ij, pixel coordinate wherein, i=1,2,3 ..., j=1,2,3 ....With I _IjRemarked pixel point f (i, pixel intensity value j), each neuron N _IjRemove and receive from exterior stimulation I _IjAlso receive from other neuronic input F that present of internal network outward, _Ij(t) and connect input L _Ij(t), then pass through neuron Joint strenght β with product coupled mode F _Ij(t) [1+ β L _Ij(t)] constitute neuron N _IjInternal act U _Ij(t), again through dynamic threshold θ _Ij(t) and U _Ij(t) comparison and encourage or suppress neuronic impulse singla output Y _Ij(t) (be called igniting again), t represents the time.

Because always the pixel brightness intensity difference than spatial neighbor in the zone is big relatively for the pixel brightness intensity difference on both sides, edge in the normal image; Therefore, if adopt PCNN to handle in two dimensional image, each neuron is corresponding one by one with image pixel; Its luminance intensity value is as neuronic outside stimulus; Then inner at PCNN, the similar pixel cluster of spatial neighbor, intensity can simultaneous ignition, otherwise asynchronous igniting.This shows as the cooresponding image pixel of simultaneous ignition and presents identical luminance intensity value in the figure image intensifying, thus level and smooth image-region; The cooresponding image pixel of asynchronous igniting presents the different brightness strength rating, thereby has strengthened the gradient of intensity of brightness between image-region, and then has given prominence to edge of image more, makes that the brightness of image intensity distributions after strengthening has more level.

In the PCNN of standard model, because the effect of hard limit function, its output is a binary map picture frame.The PCNN that sets up in order to make output mapping function can more effectively carry out the processing that the integral image contrast ratio strengthens, and based on above-mentioned human eye vision apperceive characteristic, adopts a type logarithm mapping function, and the intensity of brightness of image is mapped to a suitable visual range.

The great advantage of this method is that the perception of it and visual system has natural getting in touch; Make this model smoothed image zone, outstanding image border preferably, and can improve significantly coloured image visual effect, strengthen the true effect of image color.

Described color space conversion is meant: the image after will strengthening carries out color space conversion, and to the HSV color space, the tone H after the conversion, degree of saturation specific humidity S and brightness V are expressed as respectively the digital image after being about to strengthen from the RGB color space conversion

V≤max (R, G, B) (formula seven)

(formula eight)

(formula nine)

And calculating the H process; If H＜0 is then got

and is the actual value of H.

Said RGB and HSV are respectively the describing mode in image color space.The former space vector [R G B] ^TNot only represent the color of red R, green G and blue B three primary colours, also represent the brightness of three primary colours simultaneously, exist very big correlativity between RGB three looks.In other words, through [R G B] ^TThe different values of element can form the various colors effect.The latter is a kind of three-dimensional colour spatial model that comprises tone H, degree of saturation specific humidity S and brightness V of creating according to the characteristic directly perceived of color, also claims the hexagonal pyramid model.In this color space model; Tone H measures with angle, and span is 0～360 °, begins to calculate by anticlockwise direction from redness; Redness is that 0 °, green are that 120 °, blueness are 240 °, their complementary color: yellow is that 60 °, cyan are that 180 °, magenta are 300 °; Degree of saturation specific humidity S span is 0.0～1.0; Brightness V span is: 0.0 (black)～1.0 (white).As: pure red is [H S V] ^T=[0 1 1] ^T, and S=0 representes achromaticity, in this case, tone is undefined.

Step 2, ONLINE RECOGNITION driver fatigue state;

Said ONLINE RECOGNITION driver fatigue state, promptly apparatus of the present invention system gets into (being in) state that works online, and vehicle driver's fatigue state is implemented monitoring in real time.Step 2 comprises following step by step:

(1) gathers chaufeur face image and carry out color space conversion;

After the chaufeur face image that pinhole cameras is collected is in real time strengthened, again it is expressed from RGB color space conversion to HSV color space.

(2) eye and other zone and background are made a distinction;

In the chaufeur face image that collects; Scan the scenery tone that detects pixel from left to right, from top to bottom according to pixel coordinate; And will fall between the tone zone of HSV color space [2 °; 47 °] tone gather pairing pixel and draft and be human face region, therefore follow human face region to make a distinction exactly other zone of personage and image background.

(3) the chaufeur face image that collects is confirmed people's face center and ocular;

Utilize the HSV color space of people's face to express; With near the pairing pixel coordinate of 11 ° tone value as people's face center-point; And be that basic point is upwards expanded the capable pixel of u and to each expansion

row pixel of both sides, can be obtained the eye tracing area of u * v with people's face center-point.

(4) to the tracking of chaufeur ocular;

The method that adopts the single order prediction algorithm to follow the tracks of as the chaufeur ocular.

If current driver's facial movement speed is V (t _k)=[V _i(t _k) V _j(t _k)] ^T, and

$\left[\begin{array}{c}{V}_i\left(t_k\right)\\ V_j\left(t_k\right)\end{array}\right]=\left[\begin{array}{c}\frac{i_k-i_{k-1}}{t_k-t_{k-1}}\\ \frac{j_k-j_{k-1}}{t_k-t_{k-1}}\end{array}\right]$ (formula ten)

That is, adopt under the pitch time Δ t, Δ t=t is asked in twice computing in front and back of people's face center _k-t _K-1

Its single order prediction estimated valve should be

$\left[\begin{array}{c}{\tilde{V}}_i\left(t_k\right)\\ {\tilde{V}}_j\left(t_k\right)\end{array}\right]=\left[\begin{array}{c}\frac{i_{k-1}-i_{k-2}}{t_k-t_{k-1}}\\ \frac{j_{k-1}-j_{k-2}}{t_k-t_{k-1}}\end{array}\right]$ (formula 11)

The pixel coordinate of prediction chaufeur face target does

$\left[\begin{array}{c}{\hat{i}}_{k+1}\\ {\hat{j}}_{k+1}\end{array}\right]=\left[\begin{array}{c}{\tilde{V}}_i\left(t_k\right)(t_k-t_{k-1})\\ {\tilde{V}}_j\left(t_k\right)(t_k-t_{k-1})\end{array}\right]+\left[\begin{array}{c}{i}_k\\ j_k\end{array}\right]$ (formula 12)

In the formula, V _i(t _k) and V _j(t _k) be respectively k speed V (t constantly _k) component in pixel coordinate system on i and two coordinate axlees of j; With

Be respectively k speed V (t constantly _k) component estimated valve on i and two coordinate axlees of j; i _k, i _K-1With i _K-2Be respectively k, k-1 and k-2 i coordinate figure constantly; j _k, j _K-1With j _K-2Be respectively k, k-1 and k-2 v coordinate figure constantly;

With

Be respectively the coordinate estimated valve of k+1 moment i and j.

Therefore, as k moment t _kPeople's face center be (i _k, j _k) time, can dope chaufeur face at k+1 moment t through the single order prediction algorithm _K+1People's face center do

(5) import test sample book;

The people's face centre coordinate (i that arrives according to system keeps track _k, j _k), upwards expand successively the capable pixel of u and to both sides each the expansion

Row pixel, intercepting u * v ocular image be as the test sample book of chaufeur ocular, and it is imported apparatus of the present invention system.

(6) proper vector of calculating test sample book;

The computing of recycling (formula two) to (formula six) is accomplished the image feature value of test sample book and the calculating of proper vector thereof, obtains from v=[v ₁v ₂... v _n] ^TIn choose before the bigger pairing standardization value of eigenwert of s constitute new normalization vector

$\tilde{v}={\left[\begin{array}{cccc}{v}_1& {\mathrm{<figure-callout\; id="2"\; label="v"\; filenames="HDA0000026510220000011.png"\; state="\{\{state\}\}">v</figure-callout>}}_2& ...& v_s\end{array}\right]}^T$ (formula 13)

Therefore, can directly represent the current face image ocular of chaufeur characteristic with

.

(7) to the identification of driver fatigue state;

U * v ocular the characteristic

that projects in the feature space is compared through the distance classification function with training sample characteristic one by one; Confirm the affiliated classification of sample to be identified, promptly

$G(\tilde{v},\hat{v})=\left|\right|\tilde{v}-\hat{v}\left|\right|\&le;\mathrm{\&epsiv;}$ The time, (formula 14)

Explain: current " judgement driver fatigue state " belongs to 1 type, judges that promptly chaufeur is in fatigue state; Otherwise current " judgement driver fatigue state " belongs to-1 type, judges that promptly chaufeur is in the abnormal driving state formula.

In the formula (formula 14),

represents the eye feature normalization vector of test sample book and training sample feature space respectively.

(8) control decision

According to recognition result current driving condition is confirmed control command output; When promptly in a single day judging " judging that chaufeur is in fatigue state "; System exports control command in real time; When the prompting chaufeur should stop rest, adopt Optimal Control Strategy that automobile is progressively slowed down and final stagnation of movement according to parameters of going such as the speed of a motor vehicle.

The cycle repeats step 2 is carried out online in real time identification to driver fatigue state from (1) to (8) step by step, realizes the complete monitoring to driver fatigue state.

The fatigue state of the present invention when identification of driver is driven automatically determines whether to propose caution or self-actuating brake stagnation of movement to chaufeur.Therefore can effectively avoid vehicle to cause the generation of traffic accident because of driver tired driving.

Description of drawings

Fig. 1 is a system architecture scheme drawing of the present invention;

Fig. 2 is provided with position view for pinhole cameras among the present invention on automobile;

Fig. 3 is a signal processor structure scheme drawing of the present invention;

Fig. 4 is a speed of a motor vehicle operating-controlling mechanism scheme drawing of the present invention;

Fig. 5 is a voice prompting device structural representation of the present invention;

Fig. 6 gathers the training sample scheme drawing for the present invention.

The specific embodiment

Below in conjunction with accompanying drawing embodiments of the invention are elaborated: present embodiment provided detailed embodiment and concrete operating process, but protection scope of the present invention is not limited to following embodiment being to implement under the prerequisite with technical scheme of the present invention.

Like Fig. 1, shown in 2, present embodiment relates to a kind of intelligent identification device to driver fatigue state, comprising: pinhole cameras 1, signal processor 2, speed of a motor vehicle operating-controlling mechanism 3, voice prompting device 4.Wherein: pinhole cameras 1 is arranged in the operator's compartment, faces the face of chaufeur, is used for gathering in real time the facial view of chaufeur.The output interface of pinhole cameras 1 links to each other with the image input interface of signal processor 2 respectively, and the input interface of the output interface of signal processor 2 and speed of a motor vehicle operating-controlling mechanism 3 and the input interface of voice prompting device 4 are parallel to be connected.

As shown in Figure 3, said signal processor 2 comprises: image input interface 21, analog-to-digital conversion module 22, image processing module 23, GES input interface 24, output interface 25; Wherein: the input end of image input interface 21 links to each other through the output interface of vision cable with pinhole cameras 1; The mouth of image input interface 21 is connected with the input end of analog-to-digital conversion module 22 respectively, and the output port of analog-to-digital conversion module 22 is connected with the first input end mouth of image processing module 23; The input end of GES input interface 24 is connected with the output interface of vehicle speed sensor, and the mouth of GES input interface 24 is connected with second input port of image processing module 23; The output port of image processing module 23 is the output interface 25 of signal processor 2.

As shown in Figure 4, said speed of a motor vehicle operating-controlling mechanism 3 comprises: input interface 31, first D and A converter 32, power amplifier 33, electromagnetic valve 34 and electric brake push rod 35.Wherein, The input port of input interface 31 is connected with the output interface of signal processor 2; The output port of input interface 31 is connected with the input port of D and A converter 32; The output port of D and A converter 32 is connected with the input port of power amplifier 33, the output port of power amplifier 33 simultaneously with two end points of two end points of electromagnetic valve 34 coils and electric brake push rod 35 coils mutually and connect; Said electromagnetic valve 34 is made up of magnet coil 36, electromagnetic core 37, valve 38, resistance spring 39 and 40 5 parts of valve body; Magnet coil 36 is enclosed within the outside of electromagnetic core 37; Produce magnetic force through electromagnetic core 37 when magnet coil 36 receives electricity valve 38 is produced magnetic attraction, along with the size variation that is added in magnet coil 36 two-end-point voltages, the magnetic attraction of electromagnetic core 37 produces corresponding variation simultaneously; The magnetic attraction of electromagnetic core 37 acts on valve 38, pulling valve 38 to change the aperture of valve 38; Resistance spring 39 is a kind of extension springs, and valve 38 is in the pulling of electromagnetic core 37 magnetic attraction, and resistance spring 39 is also stretched simultaneously; Therefore produce one with the opposite elastic force of electromagnetic core 37 magnetic attraction; When electromagnetic core 37 magnetic attraction and resistance spring 39 elastic force reached balance, valve 38 just was stopped pulling, promptly stops at and be added in corresponding valve 38 apertures of magnet coil 36 two-end-point voltages; Said electric brake push rod 35; Comprise: electric brake coil and electromagnetism push rod; The electric brake mounting coil is at an end of electromagnetism push rod; The other end of electromagnetism push rod is connected with footbrake bar thick stick mechanism, and when the electric brake coil received electricity, the electromagnetic field that the electric brake coil is produced produced axial mechanical thrust to the push rod that is sleeved in the electric brake coil; This axial mechanical thrust acts on the doing on the force of electric pushrod of footbrake bar thick stick mechanism, plays with the same effect of footbrake making the vehicle self-actuating brake through bar thick stick mechanism.

As shown in Figure 5, said voice prompting device 4 comprises: input interface 41, decoder 42, digital voice module 43, second D and A converter 44, power amplifier module 45, loud speaker 46; Wherein: the input port of input interface 41 is connected with the output interface 25 of signal processor 2, in order to receive the control command from image processing module 23; The mouth of input interface 41 is connected with the input port of decoder 42; The output port of decoder 42 is connected with the input port of digital voice module 43; The output port of digital voice module 43 is connected with the input port of second D and A converter 44; The output port of second D and A converter 44 is connected with the input port of power amplifier module 45, and the output port of power amplifier module 45 is connected with the input port of loud speaker 46.After the input interface 41 of voice prompting device 4 receives control command; Explanation through decoder 42; Related voice unit in the link digital voice module 43; Voice unit sequence after the link is transported to second D and A converter 44 successively convert voice analog signal into, again voice analog signal is transported to power amplifier module 45, send the relevant voice prompting through power gain rear drive loud speaker 46.

Present embodiment also relates to a kind of intelligent identification Method to driver fatigue state, may further comprise the steps:

Step 1, system are learnt training sample

(1) gathers driver fatigue state face image and carry out color space conversion;

After strengthening at fatigue state face image that different timetables reveal and with it through three chaufeurs of pinhole cameras continuous acquisition, express from RGB color space conversion to HSV color space, wherein every width of cloth figure image all has 256 gray levels again.

(2) eye and other zone and background are made a distinction;

In the driver fatigue state face image that sampling collects; Scan the scenery tone that detects pixel from left to right, from top to bottom according to pixel coordinate; And will fall between the tone zone of HSV color space [2 °; 47 °] tone gather pairing pixel and draft and be human face region, therefore follow human face region to make a distinction exactly other zone of personage and image background.

(3) the driver fatigue state face image that sampling is collected is confirmed people's face center and ocular;

Express the HSV color space of the driver fatigue state face image that utilizing samples collects; In the set of face complexion tone; With near the pairing pixel coordinate of 11 ° tone value as people's face center-point; And with people's face center-point be basic point upwards expand the capable pixel of u and to both sides each the expansion row pixel, as the ocular of u * v.

(4) import driver fatigue state face image ocular training sample;

With importing apparatus of the present invention system as driver fatigue state face image ocular training sample after the ocular intercepting of u * v.

The driver fatigue state face image ocular training sample of being gathered all has 256 gray levels, and gets number of training n=k * l=9; Wherein, k=3, l=3 are that the sampled sample number of driver fatigue state face image ocular of 3, each chaufeur is 3 by sampling chaufeur number promptly.

(5) convert training sample image into one-dimensional vector;

The driver fatigue state face image ocular training sample image data conversion that each is two-dimentional is the vector of one dimension, and the definition " sleepy shape " be 1 type of eye feature, " non-sleepy shape " is-1 type of eye feature.Therefore can give expression to the one-dimensional vector x of i image _i=[x _I1x _I2... x _Im] ^T=[x _Ij] ^TIn the formula, x _IjJ grey scale pixel value of 1 type i sample of expression; I=1,2 ..., n is 1 type of eye sample sequence number; J=1,2 ..., m is each sample image institute capture prime number, m=u * v, u and v are respectively the row and the row pixel count of sample image.

Cause is got n=9, thus i=1,2 ..., 9; When the u=246 of every width of cloth sample image, v=112, m=27552 then, formula this moment (formula one) can be expressed as x _i=[x _I1x _I2... x _I27552] ^T=[x _Ij] ^T

(6) calculation training sample characteristics and proper vector;

By the (Formula 2) calculating an average class obtained mean

is a Class 1 eye average figure;

Press and be expressed as

i=1 after (formula three) standardizes to above-mentioned training sample; 2; ..., 9;

Press 1 type of eye normalization vector v=[v that (formula three) is made up of training sample ₁v ₂... v ₉] ^TAt this moment, 1 type of eye covariance matrix does

Q＝[v ₁?v ₂...v ₉] ^T[v ₁?v ₂...v ₉]；Q∈R ^9×9

Ask for the eigenvalue of Q successively _lAnd proper vector, and it is arranged back generating feature vector p=[λ from big to small again ₁λ ₂λ ₃...] ^T, wherein, λ ₁>=λ ₂>=λ ₃>=...

(7) training sample is carried out projecting to feature space after the linear transformation;

Because bigger eigenwert characteristic of correspondence vector has comprised more people's face eye feature information, the vector space of s the pairing proper vector formation of bigger eigenwert just can be represented the main information of people's face eyes image approx before therefore can choosing.Get s=6.

For 9 image x in the image library _i=[x _I1x _I2... x _Im] ^T(i=1,2 ..., 9) can obtain projection vector Ω to this eigenspace projection _i=[ω _I1ω _I2... ω _Im] ^T

From v=[v ₁v ₂... v ₉] ^TIn choose before the bigger pairing standardization value of eigenwert of s=6 constitute new normalization vector Therefore, can directly use

Represent 1 type of people's face eye feature, i.e. driver fatigue state face image ocular characteristic, and successively as identification of driver whether because of the tired criterion that presents sleepy shape.

Step 2, ONLINE RECOGNITION driver fatigue state;

The ONLINE RECOGNITION driver fatigue state comprises specifically following step by step:

(1) gathers chaufeur face image and carry out color space conversion;

The chaufeur face image that pinhole cameras is collected in real time adopts Pulse Coupled Neural Network method (Pulse-CoupledNeural Networks is called for short PCNN) to strengthen.Image is expressed it after strengthening again from RGB color space conversion to HSV color space.

(2) eye and other zone and background are made a distinction;

In the chaufeur face image that collects; Scan the scenery tone that detects pixel from left to right, from top to bottom according to pixel coordinate; And will fall between the tone zone of HSV color space [2 °; 47 °] tone gather pairing pixel and draft and be human face region, therefore follow human face region to make a distinction exactly other zone of personage and image background.

(3) the chaufeur face image that collects is confirmed people's face center and ocular;

Utilize the HSV color space of people's face to express, be according to the tone set of [2 °, 47 °] between the tone zone that falls into the HSV color space:

{...，9.7°，10.1°，9.5°，...}，

And the pairing pixel coordinate of this set is:

{...，(i _k-1，j _k-1)，(i _k，j _k)，(i _k+1，j _k+1)，...}，

With near 10.1 ° of pairing pixel coordinates of tone value of 11 ° as people's face center-point, promptly confirm (i _k, j _k) be people's face center position coordinates, the row coordinate of i remarked pixel, the row-coordinate of j remarked pixel, footnote are represented columns and line number, i _kK represent k row, j _kK represent that k is capable.

With people's face center-point is that basic point is upwards expanded the capable pixel of u and to each expansion

row pixel of both sides, can be obtained the eye tracing area of u * v.

(4) to the tracking of chaufeur ocular;

The method that adopts the single order prediction algorithm to follow the tracks of as the chaufeur ocular.

If current driver's ocular movement speed is V (t _k)=[V _i(t _k) V _j(t _k)] ^T, and

Its single order prediction estimated valve should be

The pixel coordinate of prediction chaufeur ocular target is

Therefore, as k moment t _kPeople's face center be (i _k, j _k) time, can dope the chaufeur ocular at k+1 moment t through the single order prediction algorithm _K+1People's face center do

(5) import test sample book;

The people's face centre coordinate (i that arrives according to system keeps track _k, j _k), upwards expand successively the capable pixel of u and to both sides each the expansion

Row pixel, intercepting u * v ocular image be as the test sample book of chaufeur ocular, and it is imported apparatus of the present invention system.

(6) proper vector of calculating test sample book;

The computing of recycling (formula two) to (formula six) is accomplished the image feature value of test sample book and the calculating of proper vector thereof, obtains from v=[v ₁v ₂... v _n] ^TIn choose before the bigger pairing standardization value of eigenwert of s constitute new normalization vector Therefore, can directly use

Represent the current face image ocular of chaufeur characteristic.

(7) to the identification of driver fatigue state;

U * v ocular the characteristic and the training sample characteristic that project in the feature space are compared one by one, confirm the affiliated classification of sample to be identified.Adopt the distance classification function to discern; During as ; Explain: current " judgement driver fatigue state " belongs to 1 type, judges that promptly chaufeur is in fatigue state; Otherwise current " judgement driver fatigue state " belongs to-1 type, judges that promptly chaufeur is in the abnormal driving state formula.

(8) control decision

According to recognition result current driving condition is confirmed control command output; When promptly in a single day judging " judging that chaufeur is in fatigue state "; System exports control command in real time; When the prompting chaufeur should stop rest, adopt Optimal Control Strategy that automobile is progressively slowed down and final stagnation of movement according to parameters of going such as the speed of a motor vehicle.

The cycle repeats step 2 is carried out online in real time identification to driver fatigue state from (1) to (8) step by step, realizes the complete monitoring to driver fatigue state.

The fatigue state of present embodiment when identification of driver is driven automatically determines whether to propose caution or self-actuating brake stagnation of movement to chaufeur.Present embodiment reaches more than 96% the recognition accuracy of driver fatigue state, therefore can effectively avoid vehicle to cause the generation of traffic accident because of driver tired driving.

Claims (3)

1. intelligent identification device to driver fatigue state; It is characterized in that; Comprise: pinhole cameras, signal processor, speed of a motor vehicle operating-controlling mechanism and speech player; The output interface of pinhole cameras is connected with the image input interface of signal processor through vision cable, and the output interface of signal processor is connected with the input interface of speed of a motor vehicle operating-controlling mechanism with the speech player numeral respectively;

Said signal processor; Comprise: image input interface, analog-to-digital conversion module, image processing module, GES input interface, output interface; Wherein: the input end of image input interface links to each other with the pinhole cameras output interface through vision cable; The mouth of image input interface is connected with the input end of analog-to-digital conversion module; The output port of analog-to-digital conversion module is connected with the input port of image processing module, and the input end of GES input interface is connected with the output interface of vehicle speed sensor, and the mouth of GES input interface is connected with the input port of image processing module;

Said speed of a motor vehicle operating-controlling mechanism; Comprise: input interface, first D and A converter, power amplifier, electromagnetic valve and electric brake push rod; Wherein, Input interface is connected with the input port of first D and A converter, and the output port of first D and A converter is connected with the input port of power amplifier, the output port of power amplifier simultaneously with two end points of two end points of magnet coil and electric brake push rod coil mutually and connect;

Said electromagnetic valve is made up of magnet coil, electromagnetic core, valve, resistance spring and valve body; Magnet coil is enclosed within the outside of electromagnetic core; Produce magnetic force through electromagnetic core when magnet coil receives electricity valve is produced magnetic attraction, along with the size variation that is added in magnet coil two-end-point voltage, the magnetic attraction of electromagnetic core produces corresponding variation simultaneously; The magnetic attraction of electromagnetic core acts on valve, pulling valve to change the aperture of valve; Resistance spring is a kind of extension spring, and valve is in the pulling of electromagnetic core magnetic attraction, and resistance spring is also stretched simultaneously; Therefore produce an elastic force opposite with the electromagnetic core magnetic attraction; When electromagnetic core magnetic attraction and resistance spring elastic force reached balance, valve just was stopped pulling, promptly stops at and be added in the corresponding valve opening of magnet coil two-end-point voltage;

Said electric brake push rod; Comprise: electric brake coil and electromagnetism push rod; The electric brake mounting coil is at an end of electromagnetism push rod; The other end of electromagnetism push rod is connected with footbrake bar thick stick mechanism, and when the electric brake coil received electricity, the electromagnetic field that the electric brake coil is produced produced axial mechanical thrust to the electromagnetism push rod that is sleeved in the electric brake coil; This axial mechanical thrust acts on the electric pushrod of footbrake bar thick stick mechanism and does on the force, plays with the same effect of footbrake making the vehicle self-actuating brake through bar thick stick mechanism.

2. the intelligent identification device to driver fatigue state according to claim 1 is characterized in that, said pinhole cameras is arranged in the operator's compartment, faces the face of chaufeur, is used for gathering in real time the facial view of chaufeur.

3. the intelligent identification Method to driver fatigue state is characterized in that, may further comprise the steps: step 1, system are learnt training sample; Step 2, ONLINE RECOGNITION driver fatigue state;

System is learnt training sample described in the step 1, and promptly system is in learning state, and when system was in learning state, system gathered and handles current driver fatigue state face image; Step 1 comprises following step by step:

(1) gathers driver fatigue state face image and carry out color space conversion;

With the pinhole cameras continuous acquisition several, comprising: after the different images constantly of a plurality of driver fatigue states are strengthened, again it is expressed from RGB color space conversion to HSV color space;

(2) people's face and other zone and background are made a distinction;

In the driver fatigue state face image that sampling collects, scan the scenery tone that detects pixel from left to right, from top to bottom according to pixel coordinate, and will fall between the tone zone of HSV color space [2 ⁰, 47 ⁰] tone gather pairing pixel and draft and be human face region, therefore follow human face region to make a distinction exactly other zone of personage and image background;

(3) the driver fatigue state face image that sampling is collected is confirmed people's face center and ocular;

Utilize the HSV color space of the driver fatigue state face image that sampling collects to express, in the set of face complexion tone, with near 11 ⁰The pairing pixel coordinate of tone value as people's face center-point, and with people's face center-point be basic point upwards expand the capable pixel of u and to both sides each the expansion The row pixel is as the ocular of u * v;

(4) import driver fatigue state face image ocular training sample;

With importing the said device of claim 1 as driver fatigue state face image ocular training sample after the ocular intercepting of u * v; The driver fatigue state face image ocular training sample of being gathered all has 256 gray levels;

(5) convert training sample image into one-dimensional vector;

The driver fatigue state face image ocular training sample image data conversion that each is two-dimentional is the vector of one dimension, and the definition " sleepy shape " be 1 type of eye feature, " non-sleepy shape " is-1 type of eye feature; Therefore can give expression to the one-dimensional vector X of i image _iBe x _i=[x _I1x _I2... x _Im] ^T=[x _Ij] ^T, wherein, X _IjJ grey scale pixel value of 1 type i sample of expression; I=1,2 ..., n is 1 type of eye sample sequence number; J=1,2 ..., m is each sample image institute capture prime number, m=u * v, u and V are respectively the row and the row pixel count of sample image;

(6) calculation training sample characteristics and proper vector;

Calculate and can be expressed as after 1 type average eye

is standardized to training sample

$v_i=x_i-\stackrel{\&OverBar;}{x};$ i＝1，2，...，n

The 1 type of average eye normalization vector v=[v that forms by training sample ₁v ₂... v _n] ^T

At this moment, 1 type of anti-variance matrix of average eye is Q=[v ₁v ₂... v _n] ^T[v ₁v ₂... v _n]; Q ∈ R ^{N * n}Ask for the eigenvalue of Q ₁And proper vector, and it is arranged back generating feature vector p=[λ from big to small again ₁λ ₂λ ₃...] ^TAnd λ ₁>=λ ₂>=λ ₃>=...

(7) training sample is carried out projecting to feature space after the linear transformation;

The vector space of s the pairing proper vector formation of bigger eigenwert just can be represented the main information of people's face eyes image approx before choosing, and the s value is confirmed by experiment, for the image of the n in the image library

x _i=[x _I1x _I2... x _Im] ^T(i=1,2 ..., n) can obtain projection vector Ω to this eigenspace projection _i=[ω _I1ω _I2... ω _Im] ^T,

From v=[v ₁v ₂... v _n] ^TIn choose before the bigger pairing standardization value of eigenwert of s constitute new normalization vector

$\hat{v}={\left[\begin{array}{cccc}{v}_1& v_2& ...& v_s\end{array}\right]}^T,$

Therefore; Directly represent 1 type of people's face eye feature, i.e. driver fatigue state face image ocular characteristic with

;

After having set up people's face eye feature normalization vector, with regard to successively as identification of driver whether because of the tired criterion that presents sleepy shape;

ONLINE RECOGNITION driver fatigue state described in the step 2, promptly the said device of claim 1 is in the state of working online, and vehicle driver's fatigue state is implemented monitoring in real time; Step 2 comprises specifically following step by step:

(1) gathers chaufeur face image and carry out color space conversion;

After the chaufeur face image that pinhole cameras is collected is in real time strengthened, again it is expressed from RGB color space conversion to HSV color space;

(2) people's face and other zone and background are made a distinction;

In the chaufeur face image that collects, scan the scenery tone that detects pixel from left to right, from top to bottom according to pixel coordinate, and will fall between the tone zone of HSV color space [2 ⁰, 47 ⁰] tone gather pairing pixel and draft and be human face region, therefore follow human face region to make a distinction exactly other zone of personage and image background;

(3) the chaufeur face image that collects is confirmed people's face center and ocular;

Utilize the HSV color space of people's face to express, with near 11 ⁰The pairing pixel coordinate of tone value as people's face center-point, and with people's face center-point be basic point upwards expand the capable pixel of u and to both sides each the expansion

The row pixel can be obtained the eye tracing area of u * v;

(4) to the tracking of chaufeur ocular;

The method that adopts the single order prediction algorithm to follow the tracks of as the chaufeur ocular;

If current driver's facial movement speed is V (t _k)=[V _i(t _k) V _j(t _k)] ^T, and

$\left[\begin{array}{c}{V}_i\left(t_k\right)\\ V_j\left(t_k\right)\end{array}\right]=\left[\begin{array}{c}\frac{i_k-i_{k-1}}{t_k-t_{k-1}}\\ \frac{j_k-j_{k-1}}{t_k-t_{k-1}}\end{array}\right]$

That is, adopt under the pitch time Δ t, Δ t=t is asked in twice computing in front and back of people's face center _k-t _K-1

Its single order prediction estimated valve should be

$\left[\begin{array}{c}{\tilde{V}}_i\left(t_k\right)\\ {\tilde{V}}_j\left(t_k\right)\end{array}\right]=\left[\begin{array}{c}\frac{i_{k-1}-i_{k-2}}{t_k-t_{k-1}}\\ \frac{j_{k-1}-j_{k-2}}{t_k-t_{k-1}}\end{array}\right]$

The pixel coordinate of prediction chaufeur face target does

$\left[\begin{array}{c}{\hat{i}}_{k+1}\\ {\hat{j}}_{k+1}\end{array}\right]=\left[\begin{array}{c}{\tilde{V}}_i\left(t_k\right)(t_k-t_{k-1})\\ {\tilde{V}}_j\left(t_k\right)(t_k-t_{k-1})\end{array}\right]+\left[\begin{array}{c}{i}_k\\ j_k\end{array}\right]$

In the formula, v _i(t _k) and v _j(t _k) be respectively k speed V (t constantly _k) component in pixel coordinate system on i and two coordinate axlees of j;

With

Be respectively k speed V (t constantly _k) component estimated valve on i and two coordinate axlees of j;

i _k, i _K-1With i _K-2Be respectively k, k-1 and k-2 i coordinate figure constantly; j _k, j _K-1With j _K-2Be respectively k, k-1 and k-2 v coordinate figure constantly; With Be respectively the coordinate estimated valve of k+1 moment i and j;

Therefore, as k moment t _kPeople's face center be (i _k, j _k) time, can dope chaufeur face at k+1 moment t through the single order prediction algorithm _K-1People's face center do

(5) import test sample book;

The people's face centre coordinate (i that arrives according to system keeps track _k, j _k), upwards to expand the capable pixel of u successively and respectively expand j row pixel to both sides, intercepting u * v ocular image is as the test sample book of chaufeur ocular, and it is imported the said device of claim 1;

(6) proper vector of calculating test sample book;

Utilize the same quadrat method of training sample computed image eigenwert and proper vector thereof, accomplish, obtain from v=[v to the image feature value of test sample book and the calculating of proper vector thereof ₁v ₂... v _n] ^TIn choose before the bigger pairing standardization value of eigenwert of s constitute new normalization vector $\tilde{v}={\left[\begin{array}{cccc}{v}_1& v_2& ...& v_s\end{array}\right]}^T,$ Therefore, can directly use

Represent the current face image ocular of chaufeur characteristic;

(7) to the identification of driver fatigue state;

U * v ocular the characteristic and the training sample characteristic that project in the feature space are compared one by one, confirm the affiliated classification of sample to be identified; Adopt the distance classification function to discern; During as

; Explain: current " judgement driver fatigue state " belongs to 1 type, judges that promptly chaufeur is in fatigue state; Otherwise current " judgement driver fatigue state " belongs to-1 type, judges that promptly chaufeur is in the abnormal driving state formula;

(8) control decision

Based on recognition result current driving condition is confirmed control instruction output; When promptly in a single day judging " judging that the driver is in fatigue state "; System exports control instruction in real time; When the prompting driver should be stopped rest, adopt Optimal Control Strategy that automobile is progressively slowed down and final stagnation of movement based on parameters of travelling such as the speed of a motor vehicle;

The cycle repeats step 2 is carried out online in real time identification to driver fatigue state from (1) to (8) step by step, realizes the complete monitoring to driver fatigue state.

Images