CN104013414A - Driver fatigue detecting system based on smart mobile phone - Google Patents
Driver fatigue detecting system based on smart mobile phone Download PDFInfo
- Publication number
- CN104013414A CN104013414A CN201410181705.8A CN201410181705A CN104013414A CN 104013414 A CN104013414 A CN 104013414A CN 201410181705 A CN201410181705 A CN 201410181705A CN 104013414 A CN104013414 A CN 104013414A
- Authority
- CN
- China
- Prior art keywords
- face
- image
- module
- pixel
- driver
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Image Analysis (AREA)
- Emergency Alarm Devices (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a driver fatigue detecting system based on a smart mobile phone. A video colleting module (1) collects images of the face and the eyes of a driver. An image preprocessing module (2) removes image noise through the gray scale adjustment of the image pixel points. A face detecting and locating module (3) achieves face detection and locating based on image pixel nonlinear color transformation. A face following module (4) improves the speed of detection of the face area of the driver through the correlation between every two adjacent images. An eye detecting module (5) conducts binarization on the images based on the improved horizontal Sobel edge detection method. An eye feature parameter extracting module (6) extracts the pupil opening degree feature parameters. A fatigue judging module (7) judges whether the driver is fatigue when driving a vehicle based on the PERCLOS method. According to the driver fatigue detecting system based on the smart mobile phone, the face detection is rapid, efficient and accurate, the requirement for the resolution is low, a camera of an ordinary mobile phone can achieve image collection and detection, popularization and application are convenient, cost is low, and the accuracy is high.
Description
Technical field
The present invention relates to a kind of automotive safety aid, be specially a kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone.
Background technology
In today of intellectualized technology fast development, car steering, towards future development intelligent, safe, is also the auxiliary key technology of driving of automobile intelligent to the detection of driver fatigue state.This patent is mainly realized monitoring, tracking and the tired judgement of driver face state, for safe driving vehicle provides safeguard.
Fatigue driving is the one of the main reasons that causes vehicle accident.According to American National expressway traffic safety portion (NHSA) report, U.S. domestic in 2000 only because driver is absent minded, tired, lack the death that the reasons such as sleep cause and approximately have 4700 people.The data demonstration providing according to China Ministry of Public Security, there is altogether once 27 of the great road traffic accidents below dead 30 people, more than 10 people in whole nation road transportation industry in January, 2002 to November.In these 27 major traffic accidents, having the direct or indirect risk factor of 19 is fatigue driving.
Due to ultra-long time driving, nighttime driving or lack the reasons such as sleep, driver there will be tired or sleepy state when driving.Driver fatigue is that the confounding factor of physiological fatigue and psychological fatigue causes, generally comprise following characteristics: absent minded, sleepiness, yawn, react slow, eyes are ached or tired, be sick of sense, have the sensation that will get angry, the number of times of rotation steering wheel reduces and angle becomes large, cannot see road sign, in track, drives and have any problem, and microsleep dormancy etc.Medical expert points out, tired response speed, judgement and the vision that not only can affect driver also can affect his Vigilance and the disposal ability to problem.Particularly " microsleep dormancy " phase of about 2/3rds seconds tired and that produce increases, and is the major incentive that vehicle accident occurs.
Existing fatigue detecting scheme one is to rely on to detect physiological signal, as variations such as electric wave, frequency of wink, heart rate, pulse frequency and skin voltages.This scheme is accurately better, but cannot collect these information by equipment easy to use in reality.
First scheme is the driving behavior that detects driver.As dynamics and the speed of steering dish, accelerator, brake pedal, gear etc.This scheme be because will read driving behavior data from automotive system, so will carry out to a certain degree integrated with automotive system, cannot independently on equipment, realize.
For example patent CN103465857 discloses a kind of method for early warning of the automobile active safety based on mobile phone, main by monocular cam integrated on mobile phone to road ahead, vehicle, pedestrian's information is carried out image acquisition, and the graphical analysis of the photographic head by mobile phone, the intelligent processing units such as algorithm for pattern recognition to camera collection to information is carried out real-time analysis calculating, thereby the position in estimation track, place, distance with front vehicles, whether the place ahead has the results such as pedestrian, and result is presented on mobile phone screen, in adventurous situation, remind in time driver, prevent that driver is tired or vehicle accident occurs absent-minded in the situation that, can effectively reduce the incidence rate of vehicle accident.But, this application is the detection based on external environment, driver's fatigue detecting itself is not related to, and driver's fatigue driving is to cause to drive unsafe key factor, when driver is during in fatigue state, even if the extraneous early warning detecting may not cause driver's attention.
For example patent CN2461804 discloses a kind of doze-proof warner for automobile driver, according to driver, the variation of steering wheel grip force is judged to driver is whether tired, detects inaccurately, and wrong report is many.
Patent CN101763711 discloses a kind of anti-doze device for driving, according to the motion frequency of hand of driver, judges whether in doze state, when automobile, keeps straight on, and hand of driver motion frequency is low, and misinformation probability is large.
Patent CN202855027 discloses the anti-fatigue doze alarming device of a kind of driver, detects driver's state by the reaction of driver's head, requires driver's head always in identical state, practical inconvenient.
In prior art, face recognition technology is a preface problem, people's face has complicated three-dimensional surface structure, the motion of facial muscle simultaneously makes people's face become a kind of non-rigid object, identification difficulty, human face expression is abundant, simultaneously, people's image that face becomes is subject to the impact of illumination, imaging angle and image-forming range, and people's face and eye recognition system are very complicated.
In recent years, it is movable that automotive safety aid field starts to explore by detecting eye, a kind of Study in Driver Fatigue State Surveillance System is provided, for example the research and implementation > > of the driving fatigue detection system of paper < < based on DSP discloses a kind of eye detection of passing through, follow the tracks of and tired recognition methods, human-eye positioning method based on active infrared light, improved the accuracy of human eye detection, can guarantee the real-time detecting, but this paper directly detects eyes, image is processed, high to image collecting device pixel request, and need to carry out parity frame difference processing to video sequence, process complicated, if the image pixel gathering is lower, Gaussian smoothing filtering likely filtering critical data, cause detecting unsuccessfully.
At present, intelligent movable mobile phone is widely used, if image acquisition and processing system that can be based on intelligent movable mobile phone, provide a kind of cost low, detection speed is fast, and check accurately, driver is fettered to few Study in Driver Fatigue State Surveillance System, can greatly solve the problems of the prior art.
Summary of the invention
Goal of the invention: for addressing the above problem, the invention provides a kind of check accurately fast, implement a kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone easily.The present invention can realize based on mobile device, the detection of photographic head based on mobile phone to people's face and eyes, realizing driver fatigue detects, it is convenient to implement, do not increase extra cost, with the abundant combination of image collecting function of smart mobile phone, provide a kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone easy to use.
Technical scheme of the present invention is:
A Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone, comprises video acquisition module, image pre-processing module, the detection of people's face and locating module, face tracking module, eye detection module, eye feature parameter extraction module, tired judge module and alarm module; Video acquisition module, image pre-processing module, the detection of people's face and locating module, face tracking module, eye detection module, eye feature parameter extraction module, tired judge module and alarm module are linked in sequence.
Video acquisition module comprises the photographic head of intelligent movable mobile phone, for the image acquisition to driver people's face and eyes.
Image pre-processing module is by the gray scale adjustment of image slices vegetarian refreshments is removed to picture noise, for illumination compensation.
People's face detects and locating module is realized the detection of people's face and location, the boundary rectangle of the facial image behaviour face face finally drawing based on image pixel nonlinear color transformation.
Face tracking module, according to there being the feature of larger dependency between the successive frame of video image, is utilized the dependency of adjacent two two field pictures, obtains face's boundary rectangle, adopts Kalman filter tracking method to improve the speed that driver's face area detects; In order to improve detection speed, in face tracking process, tracking be face's boundary rectangle.
The method binary image of eye detection module based on improved horizontal Sobel rim detection, detect eye areas, by the image approximate of eye areas, it is eyes rectangular area, in the application, the detection of people's face and eyes is not processed for all image pixels of eyes, only the pixel in respective rectangular region and profile are detected, detection efficiency is high.
Eye feature parameter extraction module is extracted the pupil aperture characteristic parameter of eyes.
Tired judge module, according to the percentage ratio of pupil aperture and eyes size, based on PERCLOS method, judges whether driver is fatigue driving.
Alarm module comprises audio alert and display alarm, and alarm module, when receiving the fatigue of tired judge module and judgment result is that fatigue, starts audio alert and display alarm.
Image pre-processing module illumination compensation specifically comprises the following steps:
The brightness of all pixels in the entire image of video acquisition module collection is arranged from high to low,
If maximum brightness threshold value is I, preset reference brightness is x*I, and wherein x represents the percentage ratio of maximum brightness threshold value, and the span of x is 0～100%; Generally, x value is 95%;
If the ratio that in image, the brightness value of pixel is greater than the pixel number of preset reference brightness and the total pixel number of image arrives preset ratio limit value, general preset ratio limit value is 2%, by preset reference brightness adjustment, be reference white, the gray value of reference white is 255, and the gray value of other pixels of entire image is converted by the yardstick of the brightness adjustment of preset reference brightness and reference white.Image pre-processing module illumination compensation can strengthen the marginal information of image, improves people's face and detects tracking efficiency, and people's face and eye detection speed are fast.
People's face detects and locating module executor face detects and location specifically comprises the following steps,
301, use nonlinear color transformation, the color value of each pixel is projected to Cb ' Cr ' two-dimensional sub-spaces from three-dimensional YCbCr color space, in Cb ' Cr ' two-dimensional sub-spaces, the pixel that represents the colour of skin flocks together, thereby is easy to separated with non-skin pixel point.
302, after detecting skin pixel point, the algorithm that adopts method based on border and the method based on region to combine, carries out cutting apart of human face region; First the algorithm based on border obtains a series of borders by brightness step, and border, for the generation of initial rectangular sequence in early stage, can obtain the boundary rectangle that comprises human face region fast; Then use the method based on region, to belonging to each pixel on border, the brightness value of the pixel of the restriction quantity of inspection pixel periphery, if brightness value surpasses boundary threshold, pixel is also comprised into border, border closed after region merges is facial contour, obtains final output face boundary rectangle.The application identifies facial contour fast, the three-dimensional surface structure to people's face, and Fast transforms, to two-dimensional space, is removed unnecessary face image, quick obtaining facial contour.
Face tracking module specifically comprises the following steps for face tracking,
401, during according to driver drives vehicle, between facial movement successive frame, there is larger dependency, in two continuous frames image, people's face does not have significant change in location, the motion that is people's face is linear, therefore adopts and from predicted current frame, goes out the position of face next frame based on Kalman (Kalman) filter tracking method;
402, Kalman filter tracking method is estimated the position of moving target in next frame image and the uncertainty of position prediction with one group of recursive algorithm, determines adaptively search window position and size in next frame, and Kalman filter tracking specifically comprises:
Face is described by position and the speed of frame in the motion of each frame, with (m
tn
t) position constantly of t that represents a pixel in face's rectangular image, (u
t, v
t) represent that described pixel is at the t speed on m and vertical direction n in the horizontal direction constantly, x
tfor the state vector of moment t is expressed as formula (1),
x
t＝[m
tn
tu
tv
t]
T (1)
Rectangle [m wherein
tn
tu
tv
t]
tfor rectangle [m
tn
tu
tv
t] transposed matrix;
Face tracking model representation is formula (2),
x
t+1＝Ax
t+w
t (2)
Wherein, A is state transition matrix, w
tfor described pixel t state error amount constantly, described w
tnormal Distribution, is expressed as w
t～N (0, Q), N (0, Q) represent normal distribution, Q is state covariance matrix;
In driver's video, think that the motion of face is linear, so state transition matrix A is formula (3):
After correctly detecting face location in initial two frames continuously, start Kalman filter tracking, establish the state vector x of described initial two frames
0and x
1,
Dependency during due to driver drives vehicle between facial movement successive frame,,
m
0＝m
1，n
0＝n
1，u
0＝m
1-n
0，v
0＝n
1-n
0
403, state covariance matrix Q calculates,
According to the observation to driver's facial movement, the noise of supposing system is as follows: the standard deviation of position system error is all 6 pixels in the horizontal and vertical directions, the standard deviation of supposing velocity error is 0.5 pixel/frame, and therefore, state covariance matrix Q is formula (4):
404, bring formula 4 into normal distribution N (0, Q), obtain t state error amount wt functional relation constantly, formula (2) is obtained the state vector of a pixel moment t in face's boundary rectangle image, wherein (mtnt) represents the position in the t moment of a pixel in face's boundary rectangle image, pass through the method, get all pixel t position constantly in face's boundary rectangle image, obtained the position of face's boundary rectangle image, by the Kalman filter tracking method of step 402, recurrence goes out the t+1 position of face's boundary rectangle image constantly, realizes face tracking.Based on Kalman (Kalman) filtering, with one group of recursive algorithm, estimate the position of moving target in next frame image and the uncertainty of position prediction, determine adaptively search window position and size in next frame, check accurately, corresponding speed is fast, and kalman filter method can better be followed the tracks of target.
Eye detection module specifically comprises the following steps eye detection process,
501, the pretreated driver's face image of image pre-processing module 2 is adopted to the method binary image of improved horizontal Sobel rim detection, by the method for connected component labeling, detect candidate's eye areas, obtain eyes boundary rectangle;
502, for described detected eyes boundary rectangle, remove and disturb rectangle, disturb rectangle to comprise the rectangular area that is highly greater than length, be less than or equal to 2mmx2mm, be greater than the rectangle of face area 5%;
503, remove and disturb after rectangle, obtain the center of gravity of each eyes circumscribed rectangular region, center of gravity for each region, take this center of gravity as summit, what the face's boundary rectangle of take was high 20%～30% searches the center of gravity in other region in high isosceles right triangle, if successfully search the center of gravity in other region, these two regions can be considered a pair of eyebrow-eye areas;
504, after completing the whole eyebrow-eye areas of searching, according to the relation of the two pairs of eyebrow-eyes in left and right, finally determine eye position;
505, after step 504, if detect eyes failure, point out user to adjust photographic head angle, photographic head reenters 501 steps after adjusting.
Eye feature parameter extraction module accounts for the percentage ratio of eyes size for the tired eye feature parameter of judging as pupil aperture, image in Preset Time is arranged from more to less by the percentage ratio of the shared number of pixels of eye areas, get front 5%～10% image of percentage ratio, the size of eyes when the meansigma methods of eye areas pixel count is considered as opening, the pupil aperture of eye areas is the pixel value that current eye detection module detects eye areas.
The percentage ratio that tired judge module definition eye pupil aperture accounts for eyes size is greater than 20% and opens for eyes, be equal to or less than 20% for eyes closed, fatigue detecting adopts PERCLOS (Percentage of EyeIid CIosure over the PupiI, over Time, be called for short PERCLOS, tolerance tired/sleepy physical quantity) method, according to PERCLOS method, when the time of eyes closed detected within continuous 2～5 second time, be judged to be fatigue while surpassing 1.6～2 seconds.PERCLOS method is not the invention improvement of this patent, the research and implementation > > of the implementation step of PERCLOS method driving fatigue detection system based on DSP referring to paper < <.
Compared with prior art, beneficial effect of the present invention comprises:
The present invention is detected and is followed the tracks of by people's face, what follow the tracks of is face and eye image, the image pretreatment of passing through, people's face detects and location, after face tracking, and then eye detection is realized to driver tired driving and detect, not directly to following the tracks of eyes, face detection quickly, efficiently and accurately, not high to pixel request, common mobile phone camera can be realized collection and the detection of image, conveniently apply, need to be by other external devices, cost is low, accuracy is high, the driver fatigue of realization based on intelligent movable mobile phone detects, the application's image pre-processing module illumination compensation, people's face detects and location, based on Kalman (Kalman) filter tracking method, carry out the steps such as face tracking and jointly solved design complexity in prior art, accuracy is low, to equipment performance, require high, the problem that monitoring velocity is slow, be conducive to apply on intelligent movable mobile phone.
Further, the application passes through nonlinear color transformation, the color value of each pixel is projected to Cb ' Cr ' two-dimensional sub-spaces from three-dimensional YCbCr color space, in Cb ' Cr ' two-dimensional sub-spaces, quick obtaining facial contour, simplicity of design, fast response time, and it is general by video sequence difference processing in prior art, human eye in positioning video sequence, design is complicated, committed memory is large, the application identifies facial contour fast, three-dimensional surface structure to people's face, Fast transforms is to two-dimensional space, remove unnecessary face image, quick obtaining facial contour.
Further, the application passes through illumination compensation denoising, first denoising, then identify people's face, in denoising process, do not affect the quality of image, in general prior art, by difference processing, obtain after human face structure feature, adopt gaussian filtering smoothing processing, image pixel reduces, and impact detects quality.
Further, the applicant's face tracking module is based on Kalman (Kalman) filtering, with one group of recursive algorithm, estimate the position of moving target in next frame image and the uncertainty of position prediction, determine adaptively search window position and size in next frame, accurately, corresponding speed is fast in check.
Accompanying drawing explanation
Fig. 1 is modular structure schematic diagram of the present invention.
The specific embodiment
Below in conjunction with accompanying drawing and specific embodiment, technical solution of the present invention is described in further detail, so that those skilled in the art can better understand the present invention also, can be implemented, but illustrated embodiment is not as a limitation of the invention.
As shown in Figure 1, a Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone, comprises that video acquisition module 1, image pre-processing module 2, people's face detect and locating module 3, face tracking module 4, eye detection module 5, eye feature parameter extraction module 6, tired judge module 7 and alarm module 8; Video acquisition module 1, image pre-processing module 2, people's face detect and locating module 3, face tracking module 4, eye detection module 5, eye feature parameter extraction module 6, tired judge module 7 and alarm module 8 are linked in sequence.
Video acquisition module 1 comprises the photographic head of intelligent movable mobile phone, for the image acquisition to driver people's face and eyes; Image pre-processing module 2 is by the gray scale adjustment of image slices vegetarian refreshments is removed to picture noise, for illumination compensation; People's face detects and locating module 3 is realized the detection of people's face and location based on image pixel nonlinear color transformation; Face tracking module 4, according to there being the feature of larger dependency between the successive frame of video image, is utilized the dependency of adjacent two two field pictures, adopts the method for following the tracks of to improve the speed that driver's face area detects; The method binary image of eye detection module 5 based on improved horizontal Sobel rim detection, detects eye areas; Eye feature parameter extraction module 6 is extracted the pupil aperture characteristic parameter of eyes; Tired judge module 7, according to the percentage ratio of pupil aperture and eyes size, based on PERCLOS method, judges whether driver is fatigue driving; Alarm module 8 comprises audio alert and display alarm, and alarm module 8, when receiving the fatigue of tired judge module 7 and judgment result is that fatigue, starts audio alert and display alarm.
Image pre-processing module 2 illumination compensations specifically comprise the following steps,
The brightness of all pixels in entire image is arranged from high to low,
If maximum brightness threshold value is I, preset reference brightness is x*I, and wherein x represents the percentage ratio of maximum brightness threshold value, and the span of x is 0～100%; X value is 95%;
If the ratio that in image, the brightness value of pixel is greater than the pixel number of 95%I and the total pixel number of image arrives preset ratio limit value 2%, by preset reference brightness adjustment, be reference white, the gray value of reference white is 255, and the gray value of other pixels of entire image is converted by the yardstick of the brightness adjustment of preset reference brightness and reference white.
People's face detects and locating module 3 executor's faces detect and location specifically comprises the following steps,
301, use nonlinear color transformation, the color value of each pixel is projected to Cb ' Cr ' two-dimensional sub-spaces from three-dimensional YCbCr color space, in Cb ' Cr ' two-dimensional sub-spaces, the pixel that represents the colour of skin flocks together, thereby is easy to separated with non-skin pixel point.
302, after detecting skin pixel point, the algorithm that adopts method based on border and the method based on region to combine, carries out cutting apart of human face region; First the algorithm based on border obtains a series of borders by brightness step, and border, for the generation of initial rectangular sequence in early stage, can obtain the boundary rectangle that comprises human face region fast; Then use the method based on region, to belonging to each pixel on border, the brightness value of the pixel of the restriction quantity (the present embodiment is used 8 pixels) of inspection pixel periphery, if brightness value surpasses boundary threshold, pixel is also comprised into border, border closed after region merges is facial contour, obtains final output face boundary rectangle.
Face tracking module 4 specifically comprises the following steps for face tracking,
Face is described by position and the speed of frame in the motion of each frame, with (m
tn
t) position constantly of t that represents a pixel in face's rectangular image, (u
t, v
t) represent that described pixel is at the t speed on m and vertical direction n in the horizontal direction constantly, x
tfor the state vector of moment t is expressed as formula (1),
x
t＝[m
tn
tu
tv
t]
T (1)，
Rectangle [m wherein
tn
tu
tv
t]
tfor rectangle [m
tn
tu
tv
t] transposed matrix;
Face tracking model representation is formula (2),
x
t+1＝Ax
t+w
t (2)
Wherein, A is state transition matrix, w
tfor described pixel t state error amount constantly, described w
tnormal Distribution, is expressed as w
t～N (0, Q), N (0, Q) represent normal distribution, Q is state covariance matrix;
In driver's video, think that the motion of face is linear, so state transition matrix A is formula (3):
After correctly detecting face location in initial two frames continuously, start Kalman filter tracking, establish the state vector x of described initial two frames
0and x
1, the dependency during due to driver drives vehicle between facial movement successive frame,,
m
0＝m
1，n
0＝n
1，u
0＝m
1-n
0，v
0＝n
1-n
0
403, state covariance matrix Q calculates,
According to the observation to driver's facial movement, the noise of supposing system is as follows: the standard deviation of position system error is all 6 pixels in the horizontal and vertical directions, the standard deviation of supposing velocity error is 0.5 pixel/frame, and therefore, state covariance matrix Q is formula (4):
404, by formula (4) bring into normal distribution N (0, Q), obtain t state error amount w constantly
tfunctional relation, formula (2) is obtained the state vector of a pixel moment t in face's boundary rectangle image, wherein (m
tn
t) position constantly of t that represents a pixel in face's boundary rectangle image, pass through the method, get all pixel t position constantly in face's boundary rectangle image, obtained the position of face's boundary rectangle image, by the Kalman filter tracking method of step 402, recurrence goes out the t+1 position of face's boundary rectangle image constantly, realizes face tracking.
5 pairs of eye detection processes of eye detection module specifically comprise the following steps,
501, the pretreated driver's face image of image pre-processing module 2 is adopted to the method binary image of improved horizontal Sobel rim detection, by the method for connected component labeling, detect candidate's eye areas, obtain eyes boundary rectangle;
502, for described detected eyes boundary rectangle, remove and disturb rectangle; Disturb rectangle to comprise the rectangular area that is highly greater than length, be less than or equal to 2mmx2mm, be greater than the rectangle of face area 5%;
503, remove and disturb after rectangle, obtain the center of gravity of each eyes circumscribed rectangular region, center of gravity for each region, take this center of gravity as summit, what the face's boundary rectangle of take was high 20%～30% searches the center of gravity in other region in high isosceles right triangle, if successfully search the center of gravity in other region, these two regions can be considered a pair of eyebrow-eye areas;
504, find after whole eyebrow-eye areas completing, according to the relation of the two pairs of eyebrow-eyes in left and right, two center of gravity lines are less than 30 ° with the angle of image X-axis, finally definite eye position;
505, after step 504, if detect eyes failure, point out user to adjust photographic head angle, photographic head reenters 501 steps after adjusting.
The eye feature parameter that eye feature parameter extraction module 6 is judged for fatigue accounts for the percentage ratio of eyes size as pupil aperture, image in Preset Time is arranged from more to less by the percentage ratio of the shared number of pixels of eye areas, get front 5%～10% image of percentage ratio, the size of eyes when the meansigma methods of eye areas pixel count is considered as opening, the pupil aperture of eye areas is the pixel value of 5 pairs of current eye areas of eye detection module.
The percentage ratio that tired judge module 7 definition eye pupil apertures account for eyes size is greater than 20% and opens for eyes, be equal to or less than 20% for eyes closed, fatigue detecting adopts PERCLOS method, according to PERCLOS method, when the time of eyes closed detected within continuous 2～5 second time, be judged to be fatigue while surpassing 1.6～2 seconds.PERCLOS method is not the invention improvement of this patent, the research and implementation > > of the implementation step of PERCLOS method driving fatigue detection system based on DSP referring to paper < <.
Below be only the preferred embodiment of the present invention; be noted that for those skilled in the art; under the premise without departing from the principles of the invention, can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.
Claims (10)
1. the Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone, it is characterized in that, comprise that video acquisition module (1), image pre-processing module (2), people's face detect and locating module (3), face tracking module (4), eye detection module (5), eye feature parameter extraction module (6), tired judge module (7) and alarm module (8);
Described video acquisition module (1), image pre-processing module (2), people's face detect and locating module (3), face tracking module (4), eye detection module (5), eye feature parameter extraction module (6), tired judge module (7) and alarm module (8) are linked in sequence;
Described video acquisition module (1) comprises the photographic head of intelligent movable mobile phone, for the image acquisition to driver people's face and eyes;
Described image pre-processing module (2) is by the gray scale adjustment of image slices vegetarian refreshments is removed to picture noise, for illumination compensation;
Described people's face detects and locating module (3) is realized the detection of people's face and location based on image pixel nonlinear color transformation;
Described face tracking module (4), according to the feature of the continuous frame-to-frame correlation of video image, is utilized the dependency of adjacent two two field pictures, obtains face's boundary rectangle, adopts Kalman filter tracking method to improve the speed that driver's face area detects;
The method binary image of described eye detection module (5) based on improved horizontal Sobel rim detection, detects eye areas;
Described eye feature parameter extraction module (6) is extracted the pupil aperture characteristic parameter of eyes;
Described tired judge module (7), according to the percentage ratio of pupil aperture and eyes size, based on PERCLOS method, judges whether driver is fatigue driving;
Described alarm module (8) comprises audio alert and display alarm.
2. a kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone according to claim 1, is characterized in that, described image pre-processing module (2) illumination compensation specifically comprises the following steps,
The brightness of all pixels in entire image is arranged from high to low,
If maximum brightness threshold value is I, preset reference brightness is x*I, and wherein x represents the percentage ratio of maximum brightness threshold value, and the span of x is 0～100%;
If the ratio that in image, the brightness value of pixel is greater than the pixel number of preset reference brightness and the total pixel number of image arrives preset ratio limit value, by described preset reference brightness adjustment, be reference white, the gray value of described reference white is 255, and the gray value of other pixels of entire image is converted by the yardstick of the brightness adjustment of preset reference brightness and described reference white.
3. a kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone according to claim 1, is characterized in that, described people's face detects and locating module (3) executor's face detects and location specifically comprises the following steps,
301, use nonlinear color transformation, the color value of each pixel is projected to Cb ' Cr ' two-dimensional sub-spaces from three-dimensional YCbCr color space, in Cb ' Cr ' two-dimensional sub-spaces, the pixel that represents the colour of skin flocks together, thereby is easy to separated with non-skin pixel point;
302, after detecting skin pixel point, the algorithm that adopts method based on border and the method based on region to combine, carries out cutting apart of human face region; First the algorithm based on border obtains a series of borders by brightness step, and described border, for the generation of initial rectangular sequence in early stage, can obtain the boundary rectangle that comprises human face region fast; Then use the method based on region, to belonging to each pixel on described border, the brightness value of the pixel of the restriction quantity of inspection pixel periphery, if described brightness value surpasses boundary threshold, described pixel is also comprised into border, border closed after region merges is facial contour, obtains final output face boundary rectangle.
4. a kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone according to claim 1, is characterized in that, described face tracking module (4) specifically comprises the following steps for face tracking,
401, dependency during according to driver drives vehicle between facial movement successive frame, in two continuous frames image, people's face does not have significant change in location, i.e. the motion of people's face is linear, therefore adopts and from predicted current frame, goes out the position of face next frame based on Kalman filter tracking method;
402, Kalman filter tracking method is estimated the position of moving target in next frame image and the uncertainty of position prediction with one group of recursive algorithm, determines adaptively the search box size in next frame, and Kalman filter tracking specifically comprises:
Face is described by position and the speed of frame in the motion of each frame, with (m
tn
t) position constantly of t that represents a pixel in face's rectangular image, (u
t, v
t) represent that described pixel is at the t speed on m and vertical direction n in the horizontal direction constantly, x
tfor the state vector of moment t, be expressed as formula (1),
x
t＝[m
tn
tu
tv
t]
T (1)，
Rectangle [m wherein
tn
tu
tv
t]
tfor rectangle [m
tn
tu
tv
t] transposed matrix;
Face tracking model representation is formula (2), x
t+1for the state vector of moment t+1,
x
t+1＝Ax
t+w
t (2)
Wherein, A is state transition matrix, w
tfor described pixel t state error amount constantly, described wt Normal Distribution, is expressed as w
t～N (0, Q), N (0, Q) represent normal distribution, Q is state covariance matrix;
In driver's video, think that the motion of face is linear, so state transition matrix A is formula (3):
After correctly detecting face location in initial two frames continuously, start Kalman filter tracking, establish the state vector x of described initial two frames
0and x
1, the dependency during due to driver drives vehicle between facial movement successive frame,,
m
0＝m
1，n
0＝n
1，u
0＝m
1-n
0，v
0＝n
1-n
0
403, state covariance matrix Q calculates,
According to the observation to driver's facial movement, the noise of supposing system is as follows: the standard deviation of position system error is all 6 pixels in the horizontal and vertical directions, the standard deviation of supposing velocity error is 0.5 pixel/frame, and therefore, state covariance matrix Q is formula (4):
404, by formula (4) bring into normal distribution N (0, Q), obtain t state error amount w constantly
tfunctional relation, formula (2) is obtained the state vector of a pixel moment t in face's boundary rectangle image, wherein (m
tn
t) position constantly of t that represents a pixel in face's boundary rectangle image, pass through the method, get all pixel t position constantly in face's boundary rectangle image, obtained the position of face's boundary rectangle image, by the Kalman filter tracking method of step 402, recurrence goes out the t+1 position of face's boundary rectangle image constantly, realizes face tracking.
5. a kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone according to claim 1, is characterized in that, described eye detection module (5) specifically comprises the following steps eye detection process,
501, the pretreated driver's face image of image pre-processing module (2) is adopted to the method binary image of improved horizontal Sobel rim detection, by the method for connected component labeling, detect candidate's eye areas, obtain eyes boundary rectangle;
502, for described detected eyes boundary rectangle, remove and disturb rectangle;
503, remove and disturb after rectangle, obtain the center of gravity of each eyes circumscribed rectangular region, center of gravity for each region, take this center of gravity as summit, what the face's boundary rectangle of take was high 20%～30% searches the center of gravity in other region in high isosceles right triangle, if successfully search the center of gravity in other region, these two regions can be considered a pair of eyebrow-eye areas;
504, after completing the whole eyebrow-eye areas of searching, according to the relation of the two pairs of eyebrow-eyes in left and right, finally determine eye position;
505, after step 504, if detect eyes failure, point out user to adjust photographic head angle, photographic head reenters 501 steps after adjusting.
6. a kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone according to claim 1, it is characterized in that, the eye feature parameter that described eye feature parameter extraction module (6) is judged for fatigue accounts for the percentage ratio of eyes size as pupil aperture, image in Preset Time is arranged from more to less by the percentage ratio of the shared number of pixels of eye areas, get front 5%～10% image of described percentage ratio, the size of eyes when the meansigma methods of described eye areas pixel count is considered as opening, the pupil aperture of described eye areas is the pixel value of eye detection module (5) to the detection of current eye areas.
7. a kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone according to claim 1, it is characterized in that, the percentage ratio that described tired judge module (7) definition eye pupil aperture accounts for eyes size is greater than 20% and opens for eyes, be equal to or less than 20% for eyes closed, fatigue detecting adopts PERCLOS method, according to PERCLOS method, when the time of eyes closed detected within continuous 2～5 second time, be judged to be fatigue while surpassing 1.6～2 seconds.
8. a kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone according to claim 5, it is characterized in that, described step 502 disturbs rectangle to comprise the rectangular area that is highly greater than length, is less than or equal to the rectangular area of 2mmx2mm, is greater than the rectangular area of face area 5%.
9. a kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone according to claim 5, is characterized in that, the pass of described step 504 eyebrow-eyes is that the angle of two center of gravity lines and image X-axis is less than 30 °.
10. a kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone according to claim 2, it is characterized in that, in described image pre-processing module (2) illumination compensation, the percentage ratio x value of maximum brightness threshold value is 95%, be that described preset reference brightness is 95%I, described preset ratio limit value is 2%.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410181705.8A CN104013414B (en) | 2014-04-30 | 2014-04-30 | A kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410181705.8A CN104013414B (en) | 2014-04-30 | 2014-04-30 | A kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104013414A true CN104013414A (en) | 2014-09-03 |
| CN104013414B CN104013414B (en) | 2015-12-30 |
Family
ID=51430625
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410181705.8A Active CN104013414B (en) | 2014-04-30 | 2014-04-30 | A kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104013414B (en) |
Cited By (40)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104269028A (en) * | 2014-10-23 | 2015-01-07 | 深圳大学 | Fatigue driving detection method and system |
| CN104398235A (en) * | 2014-12-04 | 2015-03-11 | 张晓� | Eye fatigue detector |
| CN104484803A (en) * | 2014-11-24 | 2015-04-01 | 苏州福丰科技有限公司 | Mobile phone payment method employing three-dimensional human face recognition based on neural network |
| CN104574819A (en) * | 2015-01-09 | 2015-04-29 | 安徽清新互联信息科技有限公司 | Fatigued drive detection method based on mouth features |
| CN105034804A (en) * | 2015-05-11 | 2015-11-11 | 南京理工大学 | Vehicle active safety control method and vehicle active safety control system |
| CN105286802A (en) * | 2015-11-30 | 2016-02-03 | 华南理工大学 | Driver fatigue detection method based on video information |
| CN105373767A (en) * | 2015-07-23 | 2016-03-02 | 中山大学深圳研究院 | Eye fatigue detection method for smart phones |
| CN105389948A (en) * | 2015-11-11 | 2016-03-09 | 上海斐讯数据通信技术有限公司 | System and method for preventing fatigue driving of driver |
| CN105512613A (en) * | 2015-11-26 | 2016-04-20 | 中山大学 | Smartphone-based eye fatigue detection method |
| CN106023168A (en) * | 2016-05-12 | 2016-10-12 | 广东京奥信息科技有限公司 | Method and device for edge detection in video surveillance |
| CN106108922A (en) * | 2015-05-07 | 2016-11-16 | 铃木株式会社 | Sleepy detection device |
| CN106166074A (en) * | 2016-08-30 | 2016-11-30 | 西南交通大学 | The method of testing of driver's emotion control ability and system |
| CN106361270A (en) * | 2015-07-22 | 2017-02-01 | 松下电器（美国）知识产权公司 | Method for predicting arousal level and arousal level prediction apparatus |
| CN106491073A (en) * | 2016-10-20 | 2017-03-15 | 鹄誉医疗科技（上海）有限公司 | A kind of human eye pupil detection method of quick high robust |
| CN106740581A (en) * | 2017-01-03 | 2017-05-31 | 青岛海信移动通信技术股份有限公司 | A kind of control method of mobile unit, AR devices and AR systems |
| CN106781280A (en) * | 2016-11-24 | 2017-05-31 | 上海海事大学 | A kind of vehicle safety travel real-time monitoring device |
| CN106904169A (en) * | 2015-12-17 | 2017-06-30 | 北京奇虎科技有限公司 | Traffic safety method for early warning and device |
| CN107103294A (en) * | 2017-04-20 | 2017-08-29 | 上海耐相智能科技有限公司 | A kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone |
| WO2018058275A1 (en) * | 2016-09-27 | 2018-04-05 | 深圳智乐信息科技有限公司 | Smart driving method and system employing mobile terminal |
| CN107944434A (en) * | 2015-06-11 | 2018-04-20 | 广东欧珀移动通信有限公司 | A kind of alarm method and terminal based on rotating camera |
| US10089543B2 (en) | 2016-07-29 | 2018-10-02 | Honda Motor Co., Ltd. | System and method for detecting distraction and a downward vertical head pose in a vehicle |
| CN108639056A (en) * | 2018-04-16 | 2018-10-12 | Oppo广东移动通信有限公司 | Driving condition detection method, device and mobile terminal |
| CN109308445A (en) * | 2018-07-25 | 2019-02-05 | 南京莱斯电子设备有限公司 | A kind of fixation post personnel fatigue detection method based on information fusion |
| CN109859438A (en) * | 2019-01-30 | 2019-06-07 | 北京津发科技股份有限公司 | Safe early warning method, system, vehicle and terminal device |
| CN110147738A (en) * | 2019-04-29 | 2019-08-20 | 中国人民解放军海军特色医学中心 | A kind of driver fatigue monitoring and pre-alarming method and system |
| CN110200745A (en) * | 2018-02-28 | 2019-09-06 | 深圳市掌网科技股份有限公司 | Eyeshade and sleep promotion method |
| WO2020034541A1 (en) * | 2018-08-14 | 2020-02-20 | 深圳壹账通智能科技有限公司 | Driver drowsiness detection method, computer readable storage medium, terminal device, and apparatus |
| CN111127464A (en) * | 2019-12-30 | 2020-05-08 | 执鼎医疗科技（杭州）有限公司 | Human eye congestion detection device |
| CN111158493A (en) * | 2020-01-02 | 2020-05-15 | 维沃移动通信有限公司 | Night mode switching method and electronic equipment |
| CN111291590A (en) * | 2018-12-06 | 2020-06-16 | 广州汽车集团股份有限公司 | Driver fatigue detection method, driver fatigue detection device, computer equipment and storage medium |
| CN111402317A (en) * | 2020-03-26 | 2020-07-10 | 北京新氧科技有限公司 | Eye feature measuring method, device and terminal |
| CN111797654A (en) * | 2019-04-09 | 2020-10-20 | Oppo广东移动通信有限公司 | Driver fatigue state detection method and device, storage medium and mobile terminal |
| CN111814516A (en) * | 2019-04-11 | 2020-10-23 | 上海集森电器有限公司 | Driver fatigue detection method |
| CN112528906A (en) * | 2020-12-18 | 2021-03-19 | 武汉理工大学 | Driver state detection equipment |
| CN112617772A (en) * | 2021-01-05 | 2021-04-09 | 上海工程技术大学 | Driving fatigue identification method and system based on pulse wave signals |
| CN112653844A (en) * | 2020-12-28 | 2021-04-13 | 珠海亿智电子科技有限公司 | Camera holder steering self-adaptive tracking adjustment method |
| CN113052064A (en) * | 2021-03-23 | 2021-06-29 | 北京思图场景数据科技服务有限公司 | Attention detection method based on face orientation, facial expression and pupil tracking |
| CN113212126A (en) * | 2021-06-16 | 2021-08-06 | 深圳市小马立行科技有限公司 | Cabin self-adaptive real-time intelligent regulation and control method and system applied to automobile |
| CN113343820A (en) * | 2021-05-31 | 2021-09-03 | 湖北微特传感物联研究院有限公司 | Pedestrian detection method and device, computer equipment and storage medium |
| CN114915772A (en) * | 2022-07-13 | 2022-08-16 | 沃飞长空科技(成都)有限公司 | Method and system for enhancing visual field of aircraft, aircraft and storage medium |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106539581B (en) * | 2016-12-07 | 2019-08-20 | 中国民用航空总局第二研究所 | Controller's fatigue detection method and system based on probabilistic method |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002008020A (en) * | 2000-06-26 | 2002-01-11 | Nissan Motor Co Ltd | Ocular state detector |
| US20060132319A1 (en) * | 2004-12-08 | 2006-06-22 | Denso Corporation | Driver fatigue assessment device and method |
| CN101032405A (en) * | 2007-03-21 | 2007-09-12 | 汤一平 | Safe driving auxiliary device based on omnidirectional computer vision |
| JP2009219555A (en) * | 2008-03-13 | 2009-10-01 | Toyota Motor Corp | Drowsiness detector, driving support apparatus, drowsiness detecting method |
| CN101732055A (en) * | 2009-02-11 | 2010-06-16 | 北京智安邦科技有限公司 | Method and system for testing fatigue of driver |
| CN101803928A (en) * | 2010-03-05 | 2010-08-18 | 北京智安邦科技有限公司 | Video-based driver fatigue detection device |
| CN102752458A (en) * | 2012-07-19 | 2012-10-24 | 北京理工大学 | Driver fatigue detection mobile phone and unit |
| CN103273882A (en) * | 2013-06-08 | 2013-09-04 | 无锡北斗星通信息科技有限公司 | Predetermining system for fatigue state of automobile driver |
| CN103425970A (en) * | 2013-08-29 | 2013-12-04 | 大连理工大学 | Human-computer interaction method based on head postures |
-
2014
- 2014-04-30 CN CN201410181705.8A patent/CN104013414B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2002008020A (en) * | 2000-06-26 | 2002-01-11 | Nissan Motor Co Ltd | Ocular state detector |
| US20060132319A1 (en) * | 2004-12-08 | 2006-06-22 | Denso Corporation | Driver fatigue assessment device and method |
| CN101032405A (en) * | 2007-03-21 | 2007-09-12 | 汤一平 | Safe driving auxiliary device based on omnidirectional computer vision |
| JP2009219555A (en) * | 2008-03-13 | 2009-10-01 | Toyota Motor Corp | Drowsiness detector, driving support apparatus, drowsiness detecting method |
| CN101732055A (en) * | 2009-02-11 | 2010-06-16 | 北京智安邦科技有限公司 | Method and system for testing fatigue of driver |
| CN101803928A (en) * | 2010-03-05 | 2010-08-18 | 北京智安邦科技有限公司 | Video-based driver fatigue detection device |
| CN102752458A (en) * | 2012-07-19 | 2012-10-24 | 北京理工大学 | Driver fatigue detection mobile phone and unit |
| CN103273882A (en) * | 2013-06-08 | 2013-09-04 | 无锡北斗星通信息科技有限公司 | Predetermining system for fatigue state of automobile driver |
| CN103425970A (en) * | 2013-08-29 | 2013-12-04 | 大连理工大学 | Human-computer interaction method based on head postures |
Non-Patent Citations (1)
| Title |
|---|
| 刘小磊: "基于图像信息融合的嵌入式驾驶疲劳检测的研究", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (49)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104269028A (en) * | 2014-10-23 | 2015-01-07 | 深圳大学 | Fatigue driving detection method and system |
| CN104269028B (en) * | 2014-10-23 | 2017-02-01 | 深圳大学 | Fatigue driving detection method and system |
| CN104484803A (en) * | 2014-11-24 | 2015-04-01 | 苏州福丰科技有限公司 | Mobile phone payment method employing three-dimensional human face recognition based on neural network |
| CN104398235A (en) * | 2014-12-04 | 2015-03-11 | 张晓� | Eye fatigue detector |
| CN104574819A (en) * | 2015-01-09 | 2015-04-29 | 安徽清新互联信息科技有限公司 | Fatigued drive detection method based on mouth features |
| CN106108922A (en) * | 2015-05-07 | 2016-11-16 | 铃木株式会社 | Sleepy detection device |
| CN106108922B (en) * | 2015-05-07 | 2019-04-05 | 铃木株式会社 | Drowsiness detection device |
| CN105034804A (en) * | 2015-05-11 | 2015-11-11 | 南京理工大学 | Vehicle active safety control method and vehicle active safety control system |
| CN107944434A (en) * | 2015-06-11 | 2018-04-20 | 广东欧珀移动通信有限公司 | A kind of alarm method and terminal based on rotating camera |
| CN106361270B (en) * | 2015-07-22 | 2021-05-07 | 松下电器（美国）知识产权公司 | Arousal level prediction method and arousal level prediction device |
| CN106361270A (en) * | 2015-07-22 | 2017-02-01 | 松下电器（美国）知识产权公司 | Method for predicting arousal level and arousal level prediction apparatus |
| CN105373767A (en) * | 2015-07-23 | 2016-03-02 | 中山大学深圳研究院 | Eye fatigue detection method for smart phones |
| CN105389948A (en) * | 2015-11-11 | 2016-03-09 | 上海斐讯数据通信技术有限公司 | System and method for preventing fatigue driving of driver |
| CN105512613A (en) * | 2015-11-26 | 2016-04-20 | 中山大学 | Smartphone-based eye fatigue detection method |
| CN105286802B (en) * | 2015-11-30 | 2019-05-14 | 华南理工大学 | Driver Fatigue Detection based on video information |
| CN105286802A (en) * | 2015-11-30 | 2016-02-03 | 华南理工大学 | Driver fatigue detection method based on video information |
| CN106904169A (en) * | 2015-12-17 | 2017-06-30 | 北京奇虎科技有限公司 | Traffic safety method for early warning and device |
| CN106023168A (en) * | 2016-05-12 | 2016-10-12 | 广东京奥信息科技有限公司 | Method and device for edge detection in video surveillance |
| US10089543B2 (en) | 2016-07-29 | 2018-10-02 | Honda Motor Co., Ltd. | System and method for detecting distraction and a downward vertical head pose in a vehicle |
| CN106166074B (en) * | 2016-08-30 | 2019-02-05 | 西南交通大学 | The test method and system of driver's emotion control ability |
| CN106166074A (en) * | 2016-08-30 | 2016-11-30 | 西南交通大学 | The method of testing of driver's emotion control ability and system |
| WO2018058275A1 (en) * | 2016-09-27 | 2018-04-05 | 深圳智乐信息科技有限公司 | Smart driving method and system employing mobile terminal |
| CN106491073A (en) * | 2016-10-20 | 2017-03-15 | 鹄誉医疗科技（上海）有限公司 | A kind of human eye pupil detection method of quick high robust |
| CN106781280A (en) * | 2016-11-24 | 2017-05-31 | 上海海事大学 | A kind of vehicle safety travel real-time monitoring device |
| CN106740581A (en) * | 2017-01-03 | 2017-05-31 | 青岛海信移动通信技术股份有限公司 | A kind of control method of mobile unit, AR devices and AR systems |
| CN107103294A (en) * | 2017-04-20 | 2017-08-29 | 上海耐相智能科技有限公司 | A kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone |
| CN110200745A (en) * | 2018-02-28 | 2019-09-06 | 深圳市掌网科技股份有限公司 | Eyeshade and sleep promotion method |
| CN108639056A (en) * | 2018-04-16 | 2018-10-12 | Oppo广东移动通信有限公司 | Driving condition detection method, device and mobile terminal |
| CN109308445A (en) * | 2018-07-25 | 2019-02-05 | 南京莱斯电子设备有限公司 | A kind of fixation post personnel fatigue detection method based on information fusion |
| CN109308445B (en) * | 2018-07-25 | 2019-06-25 | 南京莱斯电子设备有限公司 | A kind of fixation post personnel fatigue detection method based on information fusion |
| WO2020034541A1 (en) * | 2018-08-14 | 2020-02-20 | 深圳壹账通智能科技有限公司 | Driver drowsiness detection method, computer readable storage medium, terminal device, and apparatus |
| CN111291590A (en) * | 2018-12-06 | 2020-06-16 | 广州汽车集团股份有限公司 | Driver fatigue detection method, driver fatigue detection device, computer equipment and storage medium |
| CN109859438A (en) * | 2019-01-30 | 2019-06-07 | 北京津发科技股份有限公司 | Safe early warning method, system, vehicle and terminal device |
| CN111797654A (en) * | 2019-04-09 | 2020-10-20 | Oppo广东移动通信有限公司 | Driver fatigue state detection method and device, storage medium and mobile terminal |
| CN111814516A (en) * | 2019-04-11 | 2020-10-23 | 上海集森电器有限公司 | Driver fatigue detection method |
| CN110147738A (en) * | 2019-04-29 | 2019-08-20 | 中国人民解放军海军特色医学中心 | A kind of driver fatigue monitoring and pre-alarming method and system |
| CN111127464A (en) * | 2019-12-30 | 2020-05-08 | 执鼎医疗科技（杭州）有限公司 | Human eye congestion detection device |
| CN111158493A (en) * | 2020-01-02 | 2020-05-15 | 维沃移动通信有限公司 | Night mode switching method and electronic equipment |
| CN111158493B (en) * | 2020-01-02 | 2022-11-15 | 维沃移动通信有限公司 | Night mode switching method and electronic equipment |
| CN111402317A (en) * | 2020-03-26 | 2020-07-10 | 北京新氧科技有限公司 | Eye feature measuring method, device and terminal |
| CN112528906A (en) * | 2020-12-18 | 2021-03-19 | 武汉理工大学 | Driver state detection equipment |
| CN112653844A (en) * | 2020-12-28 | 2021-04-13 | 珠海亿智电子科技有限公司 | Camera holder steering self-adaptive tracking adjustment method |
| CN112617772A (en) * | 2021-01-05 | 2021-04-09 | 上海工程技术大学 | Driving fatigue identification method and system based on pulse wave signals |
| CN112617772B (en) * | 2021-01-05 | 2022-12-27 | 上海工程技术大学 | Driving fatigue identification method and system based on pulse wave signals |
| CN113052064A (en) * | 2021-03-23 | 2021-06-29 | 北京思图场景数据科技服务有限公司 | Attention detection method based on face orientation, facial expression and pupil tracking |
| CN113052064B (en) * | 2021-03-23 | 2024-04-02 | 北京思图场景数据科技服务有限公司 | Attention detection method based on face orientation, facial expression and pupil tracking |
| CN113343820A (en) * | 2021-05-31 | 2021-09-03 | 湖北微特传感物联研究院有限公司 | Pedestrian detection method and device, computer equipment and storage medium |
| CN113212126A (en) * | 2021-06-16 | 2021-08-06 | 深圳市小马立行科技有限公司 | Cabin self-adaptive real-time intelligent regulation and control method and system applied to automobile |
| CN114915772A (en) * | 2022-07-13 | 2022-08-16 | 沃飞长空科技(成都)有限公司 | Method and system for enhancing visual field of aircraft, aircraft and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104013414B (en) | 2015-12-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104013414B (en) | A kind of Study in Driver Fatigue State Surveillance System based on intelligent movable mobile phone | |
| Zhang et al. | A new real-time eye tracking based on nonlinear unscented Kalman filter for monitoring driver fatigue | |
| CN103824420B (en) | Fatigue driving identification system based on heart rate variability non-contact measurement | |
| CN105769120B (en) | Method for detecting fatigue driving and device | |
| CN101639894B (en) | Method for detecting train driver behavior and fatigue state on line and detection system thereof | |
| CN101593425B (en) | Machine vision based fatigue driving monitoring method and system | |
| CN102054163B (en) | Method for testing driver fatigue based on monocular vision | |
| CN110119676A (en) | A kind of Driver Fatigue Detection neural network based | |
| CN105654753A (en) | Intelligent vehicle-mounted safe driving assistance method and system | |
| Flores et al. | Driver drowsiness detection system under infrared illumination for an intelligent vehicle | |
| CN104224204A (en) | Driver fatigue detection system on basis of infrared detection technology | |
| Al-Madani et al. | Real-time driver drowsiness detection based on eye movement and yawning using facial landmark | |
| CN105286802A (en) | Driver fatigue detection method based on video information | |
| CN102930693A (en) | Early warning system and method for safe driving | |
| CN109740477A (en) | Study in Driver Fatigue State Surveillance System and its fatigue detection method | |
| Tang et al. | Real-time image-based driver fatigue detection and monitoring system for monitoring driver vigilance | |
| CN104881956A (en) | Fatigue driving early warning system | |
| CN104123549A (en) | Eye positioning method for real-time monitoring of fatigue driving | |
| CN107563346A (en) | One kind realizes that driver fatigue sentences method for distinguishing based on eye image processing | |
| CN109977930A (en) | Method for detecting fatigue driving and device | |
| Kiran et al. | Driver drowsiness detection | |
| Luo et al. | The driver fatigue monitoring system based on face recognition technology | |
| Hou et al. | A lightweight framework for abnormal driving behavior detection | |
| Bergasa et al. | Visual monitoring of driver inattention | |
| Wathiq et al. | Optimized driver safety through driver fatigue detection methods |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C41 | Transfer of patent application or patent right or utility model | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20151117 Address after: 518000 Guangdong city of Shenzhen province Nanshan District south road six Taiho Technology Building 410 Applicant after: SHENZHEN MINIEYE INNOVATION TECHNOLOGY Co.,Ltd. Address before: No. 3 Gu Tan Road in Gaochun Economic Development Zone of Nanjing city in Jiangsu province 211300 Applicant before: NANJING CHERUI INFORMATION TECHNOLOGY Co.,Ltd. |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: Floor 25, Block A, Zhongzhou Binhai Commercial Center Phase II, No. 9285, Binhe Boulevard, Shangsha Community, Shatou Street, Futian District, Shenzhen, Guangdong 518000 Patentee after: Shenzhen Youjia Innovation Technology Co.,Ltd. Address before: 518000, 410, Taibang Technology Building, Gaoxin South Sixth Road, Nanshan District, Shenzhen, Guangdong Province Patentee before: SHENZHEN MINIEYE INNOVATION TECHNOLOGY Co.,Ltd. |
|
| CP03 | Change of name, title or address |