CN103714659A - Fatigue driving identification system based on double-spectrum fusion - Google Patents
Fatigue driving identification system based on double-spectrum fusion Download PDFInfo
- Publication number
- CN103714659A CN103714659A CN201310731288.5A CN201310731288A CN103714659A CN 103714659 A CN103714659 A CN 103714659A CN 201310731288 A CN201310731288 A CN 201310731288A CN 103714659 A CN103714659 A CN 103714659A
- Authority
- CN
- China
- Prior art keywords
- image
- driver
- fatigue
- black white
- image sensor
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a fatigue driving identification system based on double-spectrum fusion. The fatigue driving identification system based on double-spectrum fusion comprises a double-spectrum image collection device, an image fusion device and a fatigue identification device, and is characterized in that the double-spectrum image collection device comprises a light source, a semi-reflecting semi-permeable mirror, a black-and-white image sensor module and a color image sensor module, wherein a color image imaging system is formed by the light source, the semi-reflecting semi-permeable mirror and the color image sensor module, and a black-and-white image imaging system is formed by the light source, the semi-reflecting semi-permeable mirror and the black-and-white image sensor module; the image fusion device is connected with the black-and-white image sensor module and the color image sensor module, and is used for obtaining black-and-white images collected by the black-and-white image sensor module and color images collected by the color image sensor module, and fusing the black-and-white images and the color images to form driver images for identification; the fatigue identification device is used for carrying out image analysis on the fused driver images, determining the eye region of a driver through characteristic point locating, carrying out fatigue state judgment, and carrying out prompting or early warning according to the judgment result. According to the fatigue driving identification system based on double-spectrum fusion, due to the fact that locating is carried out on the eyes in high-quality face images, effectiveness and accuracy of fatigue driving judgment are greatly improved.
Description
Technical field
The invention belongs to intelligent transport technology field, be specifically related to a kind of fatigue driving recognition system merging based on two spectrum.
Background technology
In causing driver's human factor of road traffic accident, fatigue driving is one of the main reasons, the economic loss that the traffic hazard only causing because of fatigue driving in China brings is every year up to hundreds billion of units, it is annual because the accident that fatigue driving causes causes the death of people more than 90,000 or severely injured, hundreds thousand of family's wellbeings are destroyed, and fatigue driving endangers with harmonious stabilized zone greatly to society.
In order to reduce the generation of similar traffic hazard, since the nineties, started to study energetically driver fatigue method for early warning both at home and abroad.From the technology of current employing, these methods are mainly divided into three major types.
(1) detection method based on physical signs, employing be contact type measurement mode, by test drives people's physiological signal, infer driver's fatigue state.In this method, need measured to wear corresponding device, then by eeg analysis, ecg analysis and pulses measure and steering wheel grip Measurement and analysis etc., carry out fatigue state identification.This method can cause great interference to driving behavior, is not suitable for the application under actual environment.
(2) fatigue detection method based on driver's behavioral trait, by analyzing driver's steering wheel, pedal operation characteristic or vehicle driving trace feature supposition driver's fatigue state.The Chinese patent that on September 16th, 2009, disclosed application number was ZL200820177083.1 discloses a kind of alarm system for preventing fatigue driving, comprising: monitoring unit, controlling alarm unit and control unit for vehicle; Described monitoring unit detects the information of driver's operating and controlling vehicle, and output detections information is to controlling alarm unit; Described controlling alarm unit is connected with monitoring unit, receives described detection information, according to the information of detection output alarm steering order, arrives control unit for vehicle; Described control unit for vehicle is connected with controlling alarm unit, receives the controlling alarm instruction that controlling alarm unit sends, and controls vehicle carry out alarm operation according to controlling alarm instruction.In fact this patent is passed through according to the number of times of beating bearing circle in driver's schedule time, the number of times of stepping on the gas, and these driving habitses of travel speed judge that whether driver is in fatigue driving.Although the fatigue detection method based on driver's operating characteristic can reach certain accuracy of identification, and measuring process can not brought interference to driver.But driver's operation, except outside the Pass having with fatigue state, is also subject to the impact of road environment, travel speed, personal habits, operative skill etc., and its accuracy has much room for improvement.
(3) fatigue detection method based on facial expression, take machine vision as means, utilizes imageing sensor collection driver face-image, by the analysis of driver's facial expression feature is judged fatigue state.Driver's eye feature and mouth motion feature all can be directly used in and detect fatigue, and wherein the information relevant to eye state is most widely used at present.
The quick and precisely location of driver people's face and eyes is prerequisites of the fatigue detection method based on facial expression.By day in well-lighted situation, image obtain the general demand that can meet fatigue detecting system, but under the condition of the low-light (level)s such as night or sleet, normally by installation infrared line power valve in harvester, carry out light filling, to assist camera work, publication number is all to have adopted infrared light filling mode in the Chinese patent literature of CN201765668 and CN102436715.But infrared light supply is only responsive to black-white CCD, and the image collecting is black white image, and picture quality can decrease, and driver's fatigue of driving night more easily to produce, so this has just reduced the validity that adopts the fatigue detection method based on facial expression.The present invention therefore.
Summary of the invention
The invention provides a kind of fatigue driving recognition system merging based on two spectrum, this system has solved picture quality in the fatigue detection method of prior art based on facial expression and has caused not fatigue detection method precision and accuracy to have certain problems such as limitation.
In order to solve these problems of the prior art, technical scheme provided by the invention is:
A kind of fatigue driving recognition system merging based on two spectrum, comprise two spectrum picture harvesters, image co-registration device and tired device for identifying, it is characterized in that described pair of spectrum picture harvester comprises light source, half-reflecting half mirror, black white image sensor assembly and color image sensor module, wherein light source, half-reflecting half mirror and color image sensor module form coloured image imaging system, and light source, half-reflecting half mirror and black white image sensor assembly form black white image imaging system; Described image co-registration device is connected with color image sensor module with black white image sensor assembly respectively, obtain the black white image of black white image sensor assembly collection and the coloured image that color image sensor module gathers, coloured image and black white image are fused into the driver's image that carries out identification; Described tired device for identifying carries out graphical analysis for the driver's image to after merging, and by positioning feature point, determines driver's eye areas, carries out fatigue state judgement, and points out or early warning according to the result of judgement.
Preferred technical scheme is: described light source is infrared emission tube, for generation of infrared light supply.
Preferred technical scheme is: between described black white image sensor assembly and half-reflecting half mirror, 940nm optical filter is set; Described light emitting source, half-reflecting half mirror, optical filter and black white image sensor form black white image imaging system.
Preferred technical scheme is: described tired device for identifying comprises image processing module, people's face detection module, positioning feature point module, tired recognition module, capable of giving fatigue pre-warning module, wherein:
Image processing module, for the driver's facial image after merging is carried out to territory conversion, obtains amplitude Characteristics and the phase characteristic of image, then carries out dimension-reduction treatment;
People's face detection module, for judging whether image has people's face, if having, exports position, size, the posture information of people's face;
Positioning feature point module, at image, face characteristic being positioned, and finds out the key point that feature is described, and determines driver's eyes region;
Tired recognition module, for according to the driver's eyes region of location, carries out fatigue according to PERCLOS index and judges, judges that whether driver is in fatigue state;
Capable of giving fatigue pre-warning module, for according to tired result of determination, reminds Huo Xiang related management department to send early warning information by voice mode to driver.
Preferred technical scheme is: described image co-registration device, the black white image that image collecting device can be obtained and coloured image are by wavelet transformation, multi-resolution Fusion, Image Reconstruction obtains higher-quality facial image, for the tired identification in later stage ready, thereby realize tired recognition system in round-the-clock situation, can normally work, make system works no longer be subject to weather condition, light change and limit daytime.
Described image co-registration device comprises wavelet transformation, multi-resolution Fusion and Image Reconstruction, wherein:
Wavelet transformation, by multiple dimensioned wavelet transformation, obtains the high-frequency information of low frequency direction under image different scale, horizontal direction, vertical direction, miter angle direction.
Multi-resolution Fusion, is weighted fusion to the characteristic pattern of two width image different levels, different characteristic layer, obtains the Wavelet Multiresolution Decomposition structure of fused images.
Image Reconstruction, according to the wavelet sequence of fused images, carries out wavelet inverse transformation, reconstruct fused images.
Another object of the present invention is to provide a kind of fatigue driving recognition methods of merging based on two spectrum, it is characterized in that said method comprising the steps of:
(1) gather black white image and the coloured image of driver's face;
(2) coloured image and black white image are carried out to image co-registration, form the driver's image that carries out identification;
(3) the driver's image after merging is carried out to graphical analysis, by positioning feature point, determine driver's eye areas, carry out fatigue state judgement, and point out or early warning according to the result of judgement.
Preferred technical scheme is: in described method step (2), coloured image and black white image are carried out to image co-registration and carry out in accordance with the following steps:
1) coloured image and black white image are carried out respectively to wavelet transform process, by the wavelet transformation of different scale, obtain the image information of low frequency direction under image different scale, horizontal direction, vertical direction, 450 angular direction;
2) by coloured image and black white image multi-resolution Fusion: the characteristic pattern to the coloured image of synchronization and black white image different levels, different characteristic layer is weighted fusion, obtains the Wavelet Multiresolution Decomposition structure of fused images;
3) Image Reconstruction: according to the wavelet sequence of fused images, carry out wavelet inverse transformation, reconstruct fused images.
For above-mentioned situation, object of the present invention is exactly the in the situation that of night or low-light (level), to obtain high-quality driver's facial image, and locate fast and accurately driver people's face and eyes, thus guarantee that fatigue driving recognition system works in round-the-clock situation, and obtain better performance.
For solving the deficiencies in the prior art, the object of the present invention is to provide a kind of fatigue driving recognition system merging based on two spectrum, by the two width images under the white light obtaining and infrared light, merge, obtain high-quality driver's facial image, thereby relieving fatigue driving recognition system is obtained the problem of high-quality facial image in night or low-light (level) situation.By the processing to high-quality facial image, carry out the location of the detection of people's face and face feature point, the driver's eyes state obtaining according to analysis, carries out the judgement of driver fatigue by PERCLOS index.
For completing above-mentioned purpose, the fatigue driving recognition system merging based on two spectrum in the present invention comprises two spectrum picture harvesters, image co-registration device and tired device for identifying, two spectrum picture harvesters wherein, can obtain respectively by monolithic half-reflecting half mirror the image of white light and infrared light filling situation, wherein light source also comprises light emitting source except white light under normal circumstances.Described light emitting source is infrared emission tube, is arranged in the image collecting device of fatigue driving recognition system.
The reflection function of half-reflecting half mirror can obtain white light source under normal circumstances, and the transmission function of half-reflecting half mirror obtains the light source beyond white light.In well-lighted situation, white light source can obtain the extraordinary coloured image of quality by colored CCD by day.The infrared light supply of transmission, by the optical filter of 940nm, obtains the infrared light supply that wavelength is 940nm, then, by black-white CCD, obtains black white image.
Image co-registration device is for the sub-picture that finally permeates after two width images process image processing techniquess.Coloured image by acquiring and black white image are carried out to fusion treatment, and according to the picture quality obtaining under different illumination conditions, fusion can obtain quality image comparatively clearly.
Tired device for identifying, is mainly that the driver's facial image to gathering carries out graphical analysis, by positioning feature point, determines driver's eye areas, carries out tired mode decision.
The described fatigue driving identification system merging based on two spectrum, the infrared light supply wavelength adopting in described light-source system is 940nm.
Described tired device for identifying comprises image processing module, people's face detection module, positioning feature point module, tired recognition module, capable of giving fatigue pre-warning module, wherein:
Image processing module, for the image after merging is carried out to territory conversion, obtains respectively amplitude Characteristics and the phase characteristic of image, carries out respectively dimension-reduction treatment, for follow-up people's face detects ready.
People's face detection module, carries out modeling to the image after processing, and judges in image whether have people's face, if having, exports the information such as position, size, pose of people's face.
Positioning feature point module positions face characteristic in image, and finds out the key point that feature is described.The face characteristic using in this patent comprises eyebrow, eyes, nose and face.Final definite driver's eyes region.
Tired recognition module, according to the eye areas of location, carries out fatigue according to PERCLOS index and judges, judges that whether driver is in fatigue state.
Capable of giving fatigue pre-warning module, according to tired result of determination, reminds driver by voice mode, and early warning information also can be sent to related management department.
With respect to scheme of the prior art, advantage of the present invention is:
The two spectroscopic systems that use in the present invention can obtain two width images of Same Scene under white light and two kinds of light sources of infrared light simultaneously, thereby can in round-the-clock situation, carry out scene image shooting.Image co-registration module in the present invention can be the higher image of a width quality by the black and white of Same Scene and colored two width image co-registration according to picture characteristics.In the present invention, by the location to eyes in high-quality facial image, improve greatly validity and the accuracy of fatigue driving judgement.
Accompanying drawing explanation
Below in conjunction with drawings and Examples, the invention will be further described:
Fig. 1 is the fatigue driving recognition system block diagram merging based on two spectrum according to of the present invention;
Fig. 2 is according to the structured flowchart of two spectrum picture harvesters of the fatigue driving recognition system merging based on two spectrum of the present invention;
Fig. 3 is according to the structured flowchart of the image co-registration device of the fatigue driving recognition system merging based on two spectrum of the present invention.
Fig. 4 is according to the structured flowchart of the tired device for identifying of the fatigue driving recognition system merging based on two spectrum of the present invention.
Embodiment
Below in conjunction with specific embodiment, such scheme is described further.Should be understood that these embodiment are not limited to limit the scope of the invention for the present invention is described.The implementation condition adopting in embodiment can be done further adjustment according to the condition of concrete producer, and not marked implementation condition is generally the condition in normal experiment.
Embodiment
Fig. 1 is the fatigue driving recognition system block diagram merging based on two spectrum according to of the present invention, comprises two spectrum picture harvesters 101, image co-registration device 102, tired device for identifying 103.
Wherein two spectrum picture harvesters 101 comprise light source, half-reflecting half mirror, optical filter, black white image sensor (CCD) module, color image sensor (CCD) module, form two-way light path system.Be light source, half-reflecting half mirror, optical filter, black white image sensor (CCD) module composition the 1st road light path system (black white image imaging system), can carry out black white image imaging, for gathering driver's black white image.Light source, half-reflecting half mirror, color image sensor (CCD) module form the 2nd road light path system (coloured image imaging system), can carry out coloured image imaging, for gathering driver's coloured image.
Particularly, light source is the light emitting source being arranged on automobile, and described light emitting source is infrared emission tube, and infrared wavelength is 940nm.Light source can provide white light and two kinds of light sources of infrared light for two spectrum picture harvesters 101.Generally white light obtains light such as (daytime) sunlight by normal daylighting.
Two spectrum picture harvesters 101, can one side reflected white-light, a transmitted infrared light, thus on the one hand white light enters colored CCD Same Scene is carried out to colour imaging; On the other hand, infrared light enters black-white CCD Same Scene is carried out to black and white imaging, thereby two spectrum picture harvester 101 has been realized the colour of Same Scene and black and white imaging.Colour imaging and black and white imaging can be simultaneously, can be also asynchronous.
Fig. 2 is according to the structured flowchart of two spectrum picture harvesters 101 of the fatigue driving recognition system merging based on two spectrum of invention, comprises half-reflecting half mirror 201, colored CCD 202,940nm optical filter 203, black-white CCD 204, image co-registration module 102.
Half-reflecting half mirror 201, both can reflect white light, can carry out transmission to infrared light again, realized two-way light source separated.Colored CCD 202, the white light source reflecting by half-reflecting half mirror 201 carries out colour imaging to scene.940nm optical filter 203, filters to seeing through the light of half-reflecting half mirror 201, obtains the infrared light that wavelength is 940nm.Black-white CCD 204, utilizes the infrared light that the wavelength by optical filter is 940nm to carry out black and white imaging to scene.
Fig. 3 is according to the structured flowchart of the image co-registration device 102 of the fatigue driving recognition system merging based on two spectrum of invention, fusing device 102 is according to the quality of the two width images that obtain under different exposure conditions, merge and the good new images of quality, for tired device for identifying 103, to carry out tired identification ready.Image collecting device 101, to Same Scene, colored CCD 202 has obtained coloured image, and black-white CCD 204 has obtained black white image.By day in well-lighted situation, color image quality is relatively good, but in night and low-light (level) in the situation that, color image quality sharply declines, or do not collect effective information, but the black white image that in infrared ray light filling situation, black-white CCD 204 obtains can meet the demands.Therefore need to merge two width images, thereby in round-the-clock situation, can obtain high-quality driver's facial image.
Tired device for identifying 103, processes by the image that two spectrum picture fusing devices 102 are sent here, just can carry out the detection of people's face, and positioning feature point, thereby carries out driver fatigue judgement.If driver tired driving, sends capable of giving fatigue pre-warning signal.
Fig. 4 is according to the structured flowchart of the tired device for identifying of the fatigue driving recognition system merging based on two spectrum of invention, comprise image processing module 401, people's face detection module 402, positioning feature point module 403, tired recognition module 404, capable of giving fatigue pre-warning module 405.
People's face detection module 402, carries out modeling for the image that image processing module 401 is obtained, and judges in image whether have people's face, if having, exports the information such as position, size, pose of people's face.
Positioning feature point module 403, at image, face characteristic being positioned, and finds out the eyebrow of describing people's face key feature, eyes, the unique point of nose and face.
Capable of giving fatigue pre-warning module 405, for according to tired identification result, if driver, in fatigue state, reminds driver by alarm mode, relevant early warning information also can be uploaded to administrative authority.
Accordingly, the fatigue driving recognition system that the present invention is based on two spectrum fusions is identified according to following workflow:
(1) gather black white image and the coloured image of driver's face
Light source all weather operations, it is the infrared light supply of 940nm that white light and wavelength are provided.101 of two spectrum picture harvesters obtain driver's facial image for same pilothouse scene respectively under two kinds of light sources.In daytime well-lighted situation, colored CCD 202 can obtain the good coloured image of quality; The situation of the low-light (level) such as night or sleet, color image quality declines, and is even difficult to identification, and the black white image quality that black-white CCD 204 obtains is better comparatively speaking.
(2) coloured image and black white image are carried out to image co-registration, form the driver's image that carries out identification.Image co-registration device 102 passes through the fusion of two width images and processing, thereby obtains the good facial image of quality.
Wherein coloured image and black white image being carried out to image co-registration carries out in accordance with the following steps:
1) coloured image and black white image are carried out respectively to wavelet transform process, by the wavelet transformation of different scale, obtain the image information of low frequency direction under image different scale, horizontal direction, vertical direction, 450 angular direction;
2) by coloured image and black white image multi-resolution Fusion: the characteristic pattern to the coloured image of synchronization and black white image different levels, different characteristic layer is weighted fusion, obtains the Wavelet Multiresolution Decomposition structure of fused images;
3) Image Reconstruction: according to the wavelet sequence of fused images, carry out wavelet inverse transformation, reconstruct fused images.
(3) the driver's image after merging is carried out to graphical analysis, by positioning feature point, determine driver's eye areas, carry out fatigue state judgement, and point out or early warning according to the result of judgement.
Tired device for identifying 103 positions the people's face in image by the method for machine vision, confirms driver's eye areas by the description of corresponding unique point, according to PERCLOS index, carries out fatigue judgement.If find that driver is just in fatigue driving, 405 of capable of giving fatigue pre-warning modules give a warning driver are reminded.Wherein, can use ASM(Active Shape Model, active shape model) facial image is carried out to the location of eye, nose and mouth, the present embodiment mainly carries out eye location.Active shape model comprises training and two parts of search:
Wherein, the training of ASM is comprised of following steps:
(1) collect n and open the samples pictures that contains people's face facial zone;
(2) for each samples pictures, manually demarcate k key feature points in each training sample, so just formed a shape vector a
i, thus, n training sample picture just formed n shape vector, wherein, and a
ibe expressed as follows:
Wherein,
the coordinate that represents j unique point on i training sample;
(3) in order to eliminate people's face in picture, because the non-shape that the extraneous factors such as different angles, distance distance, posture changing cause is disturbed, make a distributed model more effective, adopt Procrustes method to be normalized or alignment operation;
(4) shape vector after alignment is carried out to PCA processing:
Calculating average shape vector:
Calculate covariance matrix Φ:
Then ask covariance matrix Φ eigenwert and by its by from big to small successively sequence;
(5) calculate n local grain g of i unique point on j training image
i1, g
i2..., g
in, ask its average
and variance S
i, just obtain this unique point and build local feature:
In each iterative process, the similarity measurement between the new feature g of a unique point and the local feature that it trains represents with mahalanobis distance:
Sample set is trained and obtained can carrying out ASM search after ASM model, average shape Yi Qi center is rotated counterclockwise to θ convergent-divergent s, and then translation X
cobtain initial model X=M (s, θ) [a
i]+X
c, by radiation, convert and parameter adjustment, with this initial model, in target shape shown in new images, calculate the reposition of each unique point, the unique point in the net shape that makes to search and corresponding real unique point are the most approaching.
After the eye position of location, find out the edge characteristic point positions such as eyes, according to the size of its pixel position and facial image, determine applicable height and width, the image at intercepting eye areas place, carries out fatigue judgement according to PERCLOS index.
Obtain behind the eyes position of facial image, eyes are analyzed and set up to eye feature and open and close condition discrimination model, by Hough, convert and determine that whether eyes are closed.
Hough conversion is to utilize image overall characteristic and edge pixel is coupled together to a kind of method of compositing area closed boundary.Utilize the target that Hough conversion can some known form of direct-detection, its major advantage is that the impact that be interrupted by noise and curve is less, and concrete steps are as follows:
1) the eyeball region of eyes is circles of a relative standard, utilize the Hough of circle to convert center and the radius that can very effectively detect eyeball: first to carry out rim detection, recycling edge detection results obtains the totalizer array of quick Hough conversion, then totalizer array is added up.Traveled through the maximal value of obtaining totalizer array after all frontier points, its coordinate is eyeball center and radius, is designated as (a, b, r);
2) upper eyelid is towards obvious with the difference of position under eyes different conditions; so use parabolical quick Hough conversion, obtain upper eyelid parameter (a0, x, y) (for avoiding confusion; use a0 replace in the parameter of upper eyelid a), the auxiliary eye state opening degree that characterizes;
3) parameter in the eyeball based on obtaining above and upper eyelid, provides following eye state opening degree evaluation criterion:
A) during a0<0 in closed-eye state;
B) eyes, for opening state, and are got the difference of the summit in upper eyelid and the ordinate of eyeball center as the evaluation criterion of eyes opening degree during a0>0, and now closure can be weighed with following formula:
For the image in certain hour section and data, according to eyes opening degree, can describe eyes and open closed process completely.
PERCLOS is the abbreviation of Percent Eye Closure, shared time scale while referring within regular hour eyes closed.
Suppose that t1 is that eyes open for closed 20% time completely; T2 is that eyes open for closed 80% time completely; T3 is that eyes are opened the time of next time opening 20% completely; T4 is that eyes are opened the time of next time opening 80% completely.By measuring t1, to the value of t4, just can calculate the value f:f=(t3-t2) of PERCLOS/(t4-t1);
Wherein, f is the percent of shared a certain special time of eyes closed time.When PERCLOS value f>0.15, can think that driver is in fatigue state.
Above-mentioned example is only explanation technical conceive of the present invention and feature, and its object is to allow person skilled in the art can understand content of the present invention and implement according to this, can not limit the scope of the invention with this.All equivalent transformations that Spirit Essence is done according to the present invention or modification, within all should being encompassed in protection scope of the present invention.
Claims (7)
1. the fatigue driving recognition system merging based on two spectrum, comprise two spectrum picture harvesters, image co-registration device and tired device for identifying, it is characterized in that described pair of spectrum picture harvester comprises light source, half-reflecting half mirror, black white image sensor assembly and color image sensor module, wherein light source, half-reflecting half mirror and color image sensor module form coloured image imaging system, and light source, half-reflecting half mirror and black white image sensor assembly form black white image imaging system; Described image co-registration device is connected with color image sensor module with black white image sensor assembly respectively, obtain the black white image of black white image sensor assembly collection and the coloured image that color image sensor module gathers, coloured image and black white image are fused into the driver's image that carries out identification; Described tired device for identifying carries out graphical analysis for the driver's image to after merging, and by positioning feature point, determines driver's eye areas, carries out fatigue state judgement, and points out or early warning according to the result of judgement.
2. the fatigue driving recognition system merging based on two spectrum according to claim 1, is characterized in that described light source is infrared emission tube, for generation of infrared light supply.
3. the fatigue driving recognition system merging based on two spectrum according to claim 2, is characterized in that, between described black white image sensor assembly and half-reflecting half mirror, 940nm optical filter is set; Described light emitting source, half-reflecting half mirror, optical filter and black white image sensor form black white image imaging system.
4. the fatigue driving recognition system merging based on two spectrum according to claim 1, it is characterized in that described tired device for identifying comprises image processing module, people's face detection module, positioning feature point module, tired recognition module, capable of giving fatigue pre-warning module, wherein:
Image processing module, for the driver's facial image after merging is carried out to territory conversion, obtains amplitude Characteristics and the phase characteristic of image, then carries out dimension-reduction treatment;
People's face detection module, for judging whether image has people's face, if having, exports position, size, the posture information of people's face;
Positioning feature point module, at image, face characteristic being positioned, and finds out the key point that feature is described, and determines driver's eyes region;
Tired recognition module, for according to the driver's eyes region of location, carries out fatigue according to PERCLOS index and judges, judges that whether driver is in fatigue state;
Capable of giving fatigue pre-warning module, for according to tired result of determination, reminds Huo Xiang related management department to send early warning information by voice mode to driver.
5. the fatigue driving recognition system merging based on two spectrum according to claim 1, it is characterized in that described image co-registration device, the black white image that image collecting device can be obtained and coloured image by wavelet transformation, multi-resolution Fusion, be reconstructed into reconstruction facial image.
6. a fatigue driving recognition methods of merging based on two spectrum, is characterized in that said method comprising the steps of:
(1) gather black white image and the coloured image of driver's face;
(2) coloured image and black white image are carried out to image co-registration, form the driver's image that carries out identification;
(3) the driver's image after merging is carried out to graphical analysis, by positioning feature point, determine driver's eye areas, carry out fatigue state judgement, and point out or early warning according to the result of judgement.
7. fatigue driving recognition methods according to claim 6, is characterized in that in described method step (2) that coloured image and black white image are carried out to image co-registration to carry out in accordance with the following steps:
1) coloured image and black white image are carried out respectively to wavelet transform process, by the wavelet transformation of different scale, obtain the image information of low frequency direction under image different scale, horizontal direction, vertical direction, miter angle direction;
2) by coloured image and black white image multi-resolution Fusion: the characteristic pattern to the coloured image of synchronization and black white image different levels, different characteristic layer is weighted fusion, obtains the Wavelet Multiresolution Decomposition structure of fused images;
3) Image Reconstruction: according to the wavelet sequence of fused images, carry out wavelet inverse transformation, reconstruct fused images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310731288.5A CN103714659B (en) | 2013-12-26 | 2013-12-26 | Fatigue driving identification system based on double-spectrum fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310731288.5A CN103714659B (en) | 2013-12-26 | 2013-12-26 | Fatigue driving identification system based on double-spectrum fusion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103714659A true CN103714659A (en) | 2014-04-09 |
| CN103714659B CN103714659B (en) | 2017-02-01 |
Family
ID=50407588
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310731288.5A Active CN103714659B (en) | 2013-12-26 | 2013-12-26 | Fatigue driving identification system based on double-spectrum fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103714659B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104183091A (en) * | 2014-08-14 | 2014-12-03 | 苏州清研微视电子科技有限公司 | System for adjusting sensitivity of fatigue driving early warning system in self-adaptive mode |
| CN106161925A (en) * | 2015-05-14 | 2016-11-23 | 聚晶半导体股份有限公司 | Image acquiring device and the image processing method of combined type thereof |
| CN106919898A (en) * | 2017-01-16 | 2017-07-04 | 北京龙杯信息技术有限公司 | Feature modeling method in recognition of face |
| WO2018040751A1 (en) * | 2016-08-29 | 2018-03-08 | 努比亚技术有限公司 | Image generation apparatus and method therefor, and image processing device and storage medium |
| WO2018082165A1 (en) * | 2016-11-03 | 2018-05-11 | 华为技术有限公司 | Optical imaging method and apparatus |
| CN109243144A (en) * | 2018-10-16 | 2019-01-18 | 南京伊斯特机械设备有限公司 | A kind of recognition of face warning system and its method for fatigue driving |
| CN110435665A (en) * | 2019-08-14 | 2019-11-12 | 东风汽车有限公司 | Driver detection device and car |
| CN110516435A (en) * | 2019-09-02 | 2019-11-29 | 国网电子商务有限公司 | Private key management method and device based on biological characteristics |
| CN110781793A (en) * | 2019-10-21 | 2020-02-11 | 合肥成方信息技术有限公司 | Artificial intelligence real-time image recognition method based on quadtree algorithm |
| CN111783563A (en) * | 2020-06-15 | 2020-10-16 | 深圳英飞拓科技股份有限公司 | Double-spectrum-based face snapshot and monitoring method, system and equipment |
| CN112866531A (en) * | 2021-01-13 | 2021-05-28 | 辽宁省视讯技术研究有限公司 | Multi-camera image synthesis micro-gloss image photographing system and implementation method |
| CN114697630A (en) * | 2020-12-28 | 2022-07-01 | 比亚迪半导体股份有限公司 | Driver state monitoring method and device, electronic equipment and readable storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007295263A (en) * | 2006-04-25 | 2007-11-08 | Fujitsu Ltd | Driving control circuit and infrared imaging device |
| CN101510007A (en) * | 2009-03-20 | 2009-08-19 | 北京科技大学 | Real time shooting and self-adapting fusing device for infrared light image and visible light image |
| CN101814137A (en) * | 2010-03-25 | 2010-08-25 | 浙江工业大学 | Driver fatigue monitor system based on infrared eye state identification |
| CN102306381A (en) * | 2011-06-02 | 2012-01-04 | 西安电子科技大学 | Method for fusing images based on beamlet and wavelet transform |
| CN103273882A (en) * | 2013-06-08 | 2013-09-04 | 无锡北斗星通信息科技有限公司 | Predetermining system for fatigue state of automobile driver |
-
2013
- 2013-12-26 CN CN201310731288.5A patent/CN103714659B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007295263A (en) * | 2006-04-25 | 2007-11-08 | Fujitsu Ltd | Driving control circuit and infrared imaging device |
| CN101510007A (en) * | 2009-03-20 | 2009-08-19 | 北京科技大学 | Real time shooting and self-adapting fusing device for infrared light image and visible light image |
| CN101814137A (en) * | 2010-03-25 | 2010-08-25 | 浙江工业大学 | Driver fatigue monitor system based on infrared eye state identification |
| CN102306381A (en) * | 2011-06-02 | 2012-01-04 | 西安电子科技大学 | Method for fusing images based on beamlet and wavelet transform |
| CN103273882A (en) * | 2013-06-08 | 2013-09-04 | 无锡北斗星通信息科技有限公司 | Predetermining system for fatigue state of automobile driver |
Non-Patent Citations (2)
| Title |
|---|
| 王国胤等: "《人脸感知 从二维到三维》", 30 September 2011 * |
| 胡世锋: "基于眼部特征的疲劳驾驶实时检测算法研究", 《万方数据》 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104183091A (en) * | 2014-08-14 | 2014-12-03 | 苏州清研微视电子科技有限公司 | System for adjusting sensitivity of fatigue driving early warning system in self-adaptive mode |
| CN106161925A (en) * | 2015-05-14 | 2016-11-23 | 聚晶半导体股份有限公司 | Image acquiring device and the image processing method of combined type thereof |
| CN106161925B (en) * | 2015-05-14 | 2019-03-01 | 聚晶半导体股份有限公司 | The image processing method of image acquiring device and its combined type |
| WO2018040751A1 (en) * | 2016-08-29 | 2018-03-08 | 努比亚技术有限公司 | Image generation apparatus and method therefor, and image processing device and storage medium |
| CN108605097B (en) * | 2016-11-03 | 2020-09-08 | 华为技术有限公司 | Optical imaging method and device |
| WO2018082165A1 (en) * | 2016-11-03 | 2018-05-11 | 华为技术有限公司 | Optical imaging method and apparatus |
| CN108605097A (en) * | 2016-11-03 | 2018-09-28 | 华为技术有限公司 | Optical imaging method and its device |
| US10810720B2 (en) | 2016-11-03 | 2020-10-20 | Huawei Technologies Co., Ltd. | Optical imaging method and apparatus |
| CN106919898A (en) * | 2017-01-16 | 2017-07-04 | 北京龙杯信息技术有限公司 | Feature modeling method in recognition of face |
| CN109243144A (en) * | 2018-10-16 | 2019-01-18 | 南京伊斯特机械设备有限公司 | A kind of recognition of face warning system and its method for fatigue driving |
| CN110435665A (en) * | 2019-08-14 | 2019-11-12 | 东风汽车有限公司 | Driver detection device and car |
| CN110516435A (en) * | 2019-09-02 | 2019-11-29 | 国网电子商务有限公司 | Private key management method and device based on biological characteristics |
| CN110781793A (en) * | 2019-10-21 | 2020-02-11 | 合肥成方信息技术有限公司 | Artificial intelligence real-time image recognition method based on quadtree algorithm |
| CN111783563A (en) * | 2020-06-15 | 2020-10-16 | 深圳英飞拓科技股份有限公司 | Double-spectrum-based face snapshot and monitoring method, system and equipment |
| CN114697630A (en) * | 2020-12-28 | 2022-07-01 | 比亚迪半导体股份有限公司 | Driver state monitoring method and device, electronic equipment and readable storage medium |
| CN112866531A (en) * | 2021-01-13 | 2021-05-28 | 辽宁省视讯技术研究有限公司 | Multi-camera image synthesis micro-gloss image photographing system and implementation method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN103714659B (en) | 2017-02-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103714659A (en) | Fatigue driving identification system based on double-spectrum fusion | |
| CN103824420B (en) | Fatigue driving identification system based on heart rate variability non-contact measurement | |
| CN105769120B (en) | Method for detecting fatigue driving and device | |
| CN101375796B (en) | Real-time detection system of fatigue driving | |
| CN101540090B (en) | Driver fatigue monitoring method based on multivariate information fusion | |
| CN104013414B (en) | A kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone | |
| CN104751600B (en) | Anti-fatigue-driving safety means and its application method based on iris recognition | |
| CN101639894B (en) | Method for detecting train driver behavior and fatigue state on line and detection system thereof | |
| CN104573646A (en) | Detection method and system, based on laser radar and binocular camera, for pedestrian in front of vehicle | |
| CN104224204A (en) | Driver fatigue detection system on basis of infrared detection technology | |
| US20100098295A1 (en) | Clear path detection through road modeling | |
| CN106485233A (en) | Drivable region detection method, device and electronic equipment | |
| CN105844257A (en) | Early warning system based on machine vision driving-in-fog road denoter missing and early warning method | |
| CN105512720A (en) | Public transport vehicle passenger flow statistical method and system | |
| CN105654753A (en) | Intelligent vehicle-mounted safe driving assistance method and system | |
| CN103902976A (en) | Pedestrian detection method based on infrared image | |
| CN103714660A (en) | System for achieving fatigue driving judgment on basis of image processing and fusion between heart rate characteristic and expression characteristic | |
| CN101950355A (en) | Method for detecting fatigue state of driver based on digital video | |
| CN105354988A (en) | Driver fatigue driving detection system based on machine vision and detection method | |
| CN104318237A (en) | Fatigue driving warning method based on face identification | |
| CN110826369A (en) | Driver attention detection method and system during driving | |
| CN106682603A (en) | Real time driver fatigue warning system based on multi-source information fusion | |
| CN102085099A (en) | Method and device for detecting fatigue driving | |
| CN110490139A (en) | Night fatigue driving judgment method based on recognition of face | |
| CN110188606A (en) | Lane recognition method, device and electronic equipment based on high light spectrum image-forming |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |