CN110147738A - A kind of driver fatigue monitoring and pre-alarming method and system - Google Patents
A kind of driver fatigue monitoring and pre-alarming method and system Download PDFInfo
- Publication number
- CN110147738A CN110147738A CN201910352155.4A CN201910352155A CN110147738A CN 110147738 A CN110147738 A CN 110147738A CN 201910352155 A CN201910352155 A CN 201910352155A CN 110147738 A CN110147738 A CN 110147738A
- Authority
- CN
- China
- Prior art keywords
- head
- image
- human
- hand
- fatigue monitoring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
Abstract
The present invention provides a kind of driver fatigue monitoring and pre-alarming method and system, and the method includes acquisition images and output giving fatigue pre-warning information, further includes: carries out human testing to the image of acquisition；Carry out human skeleton model detection；Carry out head and/or hand positioning；The position of head and/or hand is judged；Tired judgement is carried out according to human skeleton model and/or head and/or hand position.The characteristics of driver fatigue monitoring and pre-alarming method of the invention and system drive for the long boat equipment confined space in deep-sea, effectively driver fatigue can be monitored and early warning, overcome the shortcomings that fatigue monitoring technology simply based on face-image processing can not adapt to deep-sea Changhai personnel's fatigue monitoring.The neural network in the present invention has passed through the repetition training of a large amount of abundant samples simultaneously, and higher to the detection accuracy and speed of human body associated components, robustness is more preferable.
Description
Technical field
The present invention relates to safe driving fields, and in particular to a kind of driver fatigue monitoring and pre-alarming method and system.
Background technique
Driver keeps driving condition easily to generate fatigue for a long time, and major accident easily occurs for fatigue driving, effectively examines
Whether survey driver is in a state of fatigue and the generation reminded and can effectively prevented accident is given in driver tired driving.When
Before have the technologies of various detection driver tired drivings, fatigue-driving detection technology such as based on human facial expression recognition, based on company
Fatigue-driving detection technology, the fatigue-driving detection technology based on vehicle data etc. of continuous driving time monitoring.
Application No. is 2016100569844 patents of invention to disclose a kind of method for detecting fatigue driving and device, described
Method includes the direct picture for receiving the driver of acquisition；Face datection is carried out in the image of acquisition；In the face detected
In further position human eye；Based on deep neural network model, the human eye detected is positioned, identifies the state of personnel；
And the variation of the state of human eye in multiple image is tracked, judge whether driver is tired.Boat equipment confined air long for deep-sea
Between driver because equipment may be implemented semi-automatic or even full-automatic drive and focus on constantly without driver
And steering wheel is grasped, therefore allow that driver drives longer time or driver is of short duration leaves driver's compartment, in such cases
When fatigue occurs for driver, driver is easier the variation for body posture occur, and such as hand branch head or lie prone crouches in bridge.Only
The driving of the long boat equipment confined space in deep-sea is not suitable for the device and method that driver's face carries out image variants
Member's fatigue detecting.
Summary of the invention
In order to solve the above technical problems, the present invention provides a kind of driver fatigue monitoring and pre-alarming method and system, needles
The characteristics of boat equipment confined space long to deep-sea drives, can effectively be monitored the driving condition of driver and fatigue driving
Early warning.
A kind of driver fatigue monitoring and pre-alarming method, including acquisition image and output giving fatigue pre-warning information, it is characterised in that:
Further include:
Human testing is carried out to the image of acquisition；
Carry out human skeleton model detection；
Carry out head and/or hand positioning；
The position of head and/or hand is judged；
Tired judgement is carried out according to human skeleton model and/or head and/or hand position.
Preferably, human testing is carried out to the image of acquisition and is based on deep learning technology, using lightweight convolutional Neural
Network.
Any of the above-described scheme is preferably, and the lightweight convolutional neural networks are by end-to-end single phase training.
Any of the above-described scheme is preferably, and the lightweight convolutional neural networks have 2 different 3x3 convolution kernels to spy
Sign figure carries out convolution.
Any of the above-described scheme is preferably, and described 2 different 3x3 convolution kernel one classification for exporting human body is general
Rate, one for exporting the location information of human body frame.
Any of the above-described scheme is preferably, and the result for being exported 3x3 convolution kernel using non-maxima suppression method is comprehensive, shape
Adult body testing result.
Any of the above-described scheme is preferably, and when detecting human body in the image of acquisition, carries out human skeleton model to image
Detection.
Any of the above-described scheme is preferably, and is carried out human skeleton model detection and is included:
Input picture is handled using depth convolutional neural networks, it is locating in picture to export each position of human body upper body
Position and corresponding confidence level；
The interconnection vector field between each position of human body upper body is predicted to indicate the connection relationship between each position；
The interconnection vector field between each site location and position is made inferences to obtain human skeleton model using greedy algorithm.
Any of the above-described scheme is preferably, and each position of output human body upper body location in picture includes:
Extract at least one of human eye, ear, nose, neck, shoulder joint, elbow joint, wrist joint and hip joint
The center at position is as key point；
Export the position for the key point extracted.
Any of the above-described scheme is preferably, and by the attitudes vibration of skeleton pattern, judges whether driver is in tired shape
State.
Any of the above-described scheme is preferably, when detecting human body in the image of acquisition, to the image of acquisition carry out head and/
Or hand positioning.
Any of the above-described scheme is preferably, and using vision detection technology, passes through lightweight target detection network implementations head
And/or hand detection and localization.
Any of the above-described scheme is preferably, and the detection and localization of the head and/or hand includes the (side bounding box
Frame) it returns and classification confidence recurrence.
Any of the above-described scheme is preferably, when carrying out head and/or hand detection and localization to the image of acquisition, image conduct
The input of target detection network is divided into multiple grids, is the multiple frames of each grid forecasting and corresponding classification confidence.
Any of the above-described scheme is preferably, shown grid frame include four parameters, be expressed as bounding box (x, y,
W, h), wherein (x, y) represents the center of frame relevant to grid, (w, h) is the width and height of frame relevant to full figure information.
Any of the above-described scheme is preferably, the classification confidence be each classification probability, be object probability, overlapping
Spend the product of (IOU).
Any of the above-described scheme is preferably, and the lightweight target detection network is by training, the sample being trained to it
This collection is established
It chooses enough images containing target and the target in image is manually marked；
Picture comprising target and containing different complex backgrounds is trained and check and evaluation；
According to trained and check and evaluation as a result, carrying out the processing of the update including increasing, revising to the image of selection；
Updated picture is trained again, check and evaluation and update processing, until selecting out figure the most suitable
As forming sample set.
Any of the above-described scheme is preferably, and using the sample set and lightweight target detection network, trains detection mould
Type.
Any of the above-described scheme is preferably, and is carried out repeatedly training to the detection model trained and is detected, chooses detection and comment
Estimate confidence level height and the correct detection model of result.
Any of the above-described scheme is preferably, and according to trained and testing result, is carried out to original training pattern and training parameter
Fine tuning, obtains most suitable detection model, reaches best detection effect.
Any of the above-described scheme is preferably, judge the position of head and/or hand comprising steps of
Calculate the center that head part is under normal circumstances；
Detect whether that hand information occurs in the picture；
Calculate the center for appearing in the hand in image.
Any of the above-described scheme is preferably, and the location information by detecting head in every frame image counts under normal circumstances
The center that head part is in.
Any of the above-described scheme is preferably, and head and hand center distance continuous a period of time are less than the distance set
Threshold value determines that driver is in a state of fatigue.
Any of the above-described scheme is preferably, and the scope of activities of hand center continuous a period of time is less than the threshold of setting
Value determines that driver is in a state of fatigue.
Any of the above-described scheme is preferably, and the scope of activities of head center position continuous a period of time is less than the threshold of setting
Value determines that driver is in a state of fatigue.
Any of the above-described scheme is preferably, and judges to export warning information when driver is in a state of fatigue.
The present invention also provides a kind of driver fatigue monitoring and warning systems, comprising: image collecting device, processing unit and defeated
Device out, for the system for implementing the driver fatigue monitoring and pre-alarming method, the processing unit executes the method
Step:
Human testing is carried out to the image of acquisition；
Carry out human skeleton model detection；
Carry out head and/or hand positioning；
The position of head and/or hand is judged；
Tired judgement is carried out according to human skeleton model and/or head and/or hand position.
Preferably, human testing is carried out to the image of acquisition and is based on deep learning technology, using lightweight convolutional Neural
Network implementations.
Any of the above-described scheme is preferably, and when detecting human body in the image of acquisition, carries out human skeleton model to image
Detection.
Any of the above-described scheme is preferably, and is carried out human skeleton model detection and is included:
Input picture is handled using depth convolutional neural networks, it is locating in picture to export each position of human body upper body
Position and corresponding confidence level；
The interconnection vector field between each position of human body upper body is predicted to indicate the connection relationship between each position；
The interconnection vector field between each site location and position is made inferences to obtain human skeleton model using greedy algorithm.
Any of the above-described scheme is preferably, and by the attitudes vibration of skeleton pattern, judges whether driver is in tired shape
State.
Any of the above-described scheme is preferably, when detecting human body in the image of acquisition, to the image of acquisition carry out head and/
Or hand positioning.
Any of the above-described scheme is preferably, and using vision detection technology, passes through lightweight target detection network implementations head
And/or hand detection and localization.
Any of the above-described scheme is preferably, judge the position of head and/or hand comprising steps of
Calculate the center that head part is under normal circumstances；
Detect whether that hand information occurs in the picture；
Calculate the center for appearing in the hand in image.
Any of the above-described scheme is preferably, and the location information by detecting head in every frame image counts under normal circumstances
The center that head part is in.
Any of the above-described scheme is preferably, and head and hand center distance continuous a period of time are less than the distance set
Threshold value determines that driver is in a state of fatigue.
Any of the above-described scheme is preferably, and the scope of activities of hand center continuous a period of time is less than the threshold of setting
Value determines that driver is in a state of fatigue.
Any of the above-described scheme is preferably, and the scope of activities of head center position continuous a period of time is less than the threshold of setting
Value determines that driver is in a state of fatigue.
Any of the above-described scheme is preferably, and when the processing unit judges that driver is in a state of fatigue, is filled by output
Set output warning information.
Driver fatigue monitoring and pre-alarming method of the invention and system pass through the judgement of lightweight convolutional neural networks first and adopt
It whether there is human body image in the image of collection, when there are human body image, extract human skeleton model and by skeleton pattern
Attitudes vibration judges whether driver in a state of fatigue, and/or, by lightweight target detection network implementations head and/or
Hand detection and localization, and judge whether driver is in a state of fatigue according to the location information on head and/or hand, it is driven in judgement
When the person of sailing is in a state of fatigue, output warning information reminds driver.Driver fatigue monitoring and pre-alarming method of the invention and it is
The characteristics of system drives for the long boat equipment confined space in deep-sea, can effectively be monitored driver fatigue and early warning, overcome
The shortcomings that fatigue monitoring technology simply based on face-image processing can not adapt to deep-sea Changhai personnel's fatigue monitoring.Simultaneously
Neural network in the present invention has passed through the repetition training of a large amount of abundant samples, to the detection accuracy and speed of human body associated components
Du Genggao, robustness are more preferable.
Detailed description of the invention
Fig. 1 is the flow diagram of a preferred embodiment of driver fatigue monitoring and pre-alarming method according to the invention.
Fig. 2A-Fig. 2 C is three kinds of the embodiment as shown in Figure 1 of driver fatigue monitoring and pre-alarming method according to the invention
Different fatigue state human skeleton model detection effect figure.
Fig. 3 A- Fig. 3 C is three kinds of the embodiment as shown in Figure 1 of driver fatigue monitoring and pre-alarming method according to the invention
Different fatigue state human body head and/or hand locating effect figure.
Fig. 4 is the structural schematic diagram of a preferred embodiment of driver fatigue monitoring and warning system according to the invention.
Specific embodiment
For a better understanding of the present invention, the present invention will be described in detail below with reference to specific embodiments.
Embodiment 1
A kind of driver fatigue detection method for early warning, including acquisition image and output giving fatigue pre-warning information, further includes:
Human testing is carried out to the image of acquisition；
Carry out human skeleton model detection；
Carry out head and/or hand positioning；
The position of head and/or hand is judged；
Tired judgement is carried out according to human skeleton model and/or head and/or hand position.
Method starts its specific flow chart as shown in Figure 1:, executes step S1: acquisition image.Execute step 21: to acquisition
Image carry out human testing.It executes step S22: judging whether detect people in the image of acquisition according to the result of human testing
Body.
In step S21, human testing is carried out to the image of acquisition and is based on deep learning technology, using lightweight convolutional Neural
Network Fast R-CNN, lightweight convolutional neural networks Fast R-CNN have passed through end-to-end single phase training, the nerve net
There are network 2 different 3x3 convolution kernels to carry out convolution to characteristic pattern, and one of 3x3 convolution kernel is used to export the classification of human body
Probability, another 3x3 convolution kernel are used to export the location information of human body frame, finally use the result of 3x3 convolution kernel output non-
Maximum suppressing method is integrated, and human detection result is formed.
Fast R-CNN detects the step of human body are as follows:
1000-2000 candidate frame is determined in the picture using selective search；
CNN is input into whole picture, obtains characteristic pattern (feature map)；
Mapping range (patch) of each candidate frame on feature map is found, using this patch as each candidate
The convolution feature of frame is input to SPP layer (spatial pyramid pond layer) and layer later；
To the feature extracted in candidate frame, discriminate whether to belong to a certain kinds using classifier；
For belonging to the candidate frame of a certain feature, its position is further adjusted with device is returned.
When detecting human body in the image for judging acquisition in step S22, step S31 is executed: human body bone is carried out to image
Frame model inspection.
In step S31, carry out human skeleton model detection comprising steps of
S311 is handled input picture using depth convolutional neural networks, exports each position of human body upper body in picture
Location and corresponding confidence level；
S312 predicts the interconnection vector field between each position of human body upper body to indicate the connection relationship between each position；
S313 makes inferences to obtain human skeleton using greedy algorithm to the interconnection vector field between each site location and position
Model.
In step S311: output each position of human body upper body includes: the location of in picture
Extract at least one of human eye, ear, nose, neck, shoulder joint, elbow joint, wrist joint and hip joint
The center at position is as key point；
Export the position for the key point extracted.
As seen in figs. 2a-2c, three kinds of different fatigue state human skeleton models detection effect to obtain according to the method described above
Fruit figure, wherein Fig. 2A is that tested member occurs leaning on fatigue state on the seat, and Fig. 2 B one hand occurs for tested personnel and props up head
Fatigue state；Fig. 2 C is the fatigue state that another middle one hand that tested personnel occurs props up head.
When detecting human body in the image for judging acquisition in step S22, step S32 is executed, step S321 is first carried out:
Head and/or hand positioning are carried out to the image of acquisition, step S322 is then executed: judging the position of head and/or hand.
Head and/or hand positioning are carried out using vision detection technology is used to the image of acquisition in step S321, passed through
Lightweight target detection network implementations, the detection and localization to head and/or hand include bounding box (frame) return and
Classification confidence returns.When carrying out head and/or hand detection and localization to the image of acquisition, image is as target detection network
Input, is divided into multiple grids, is the multiple frames of each grid forecasting and corresponding classification confidence, shown grid frame packet
Four parameters are included, bounding box (x, y, w, h) is expressed as, wherein (x, y) represents the center of frame relevant to grid,
(w, h) is the width and height of frame relevant to full figure information.The classification confidence is the probability of each classification, is the general of object
The product of rate, degree of overlapping (IOU).
The lightweight target nerve network by training, the sample set that it is trained establish comprising steps of
It chooses enough images containing target and the target in image is manually marked；
Picture comprising target and containing different complex backgrounds is trained and check and evaluation；
According to trained and check and evaluation as a result, carrying out the processing of the update including increasing, revising to the image of selection；
Updated picture is trained again, check and evaluation and update processing, until selecting out figure the most suitable
As forming sample set.
It is likely to occur in view of driver and wears the case where making instruction cap, when selecting the image containing target, need to choose portion
Divide and is trained with the image for making instruction cap.
Using the sample set and lightweight target detection network, detection model is trained, to the detection model trained
It carries out repeatedly training and detects, choose check and evaluation confidence level height and the correct detection model of result, tied according to training and detection
Fruit is finely adjusted original training pattern and training parameter, obtains most suitable detection model, reaches best detection effect.
Fig. 3 A- Fig. 3 B show the three kinds of different fatigue state human body heads and/or hand obtained using above-mentioned localization method
Portion's locating effect figure.Fig. 3 A is that tested people the fatigue state to crouch in bridge of lying prone occurs, and detection model successfully detects human body
Head position, and providing confidence value is 0.94；Fig. 3 B is that tested people does not occur fatigue state, and detection model successfully detects
Human head location, and providing confidence level is 0.86；Fig. 3 C is the fatigue state that the singlehanded support cheek occurs in tested people, detection model
Successfully detect the position of human body head and hand.
In step S322, according to, to human body head and/or hand positioning result, judging head and/or hand in step S321
The position in portion, step S322 include:
Location information by detecting head in every frame image counts the center that head part under normal circumstances is in；
Detect whether that hand information occurs in the picture；
Calculate the center for appearing in the hand in image.
It executes step S4: fatigue state is judged whether it is according to human skeleton model and/or head and/or hand position.
By the attitudes vibration of skeleton pattern, judge whether driver is in a state of fatigue, such as pass through the human skeleton model detected,
Hand can be determined at a distance from head, determine whether to occur the fatigue movement for holding in the palm the cheek；For another example by determining head and shoulder joint
Positional relationship judge whether driver occurs tired movement.Head and hand center distance continuous a period of time, which are less than, to be set
Fixed distance threshold determines that driver is in a state of fatigue；The scope of activities of hand center continuous a period of time, which is less than, to be set
Fixed threshold value determines that driver is in a state of fatigue；The scope of activities of head center position continuous a period of time is less than setting
Threshold value determines that driver is in a state of fatigue.
When judging that fatigue state occurs in driver in step S4, step S5 is executed, exports giving fatigue pre-warning information.
Embodiment 2
As shown in figure 4, a kind of driver fatigue monitoring and warning system, for implementing the driver fatigue monitoring and warning side
Method, the system comprises: image collecting device 21, processing unit 22 and output device 23, the processing unit 22 execute described
Step in method:
Human testing is carried out to the image of acquisition；
Carry out human skeleton model detection；
Carry out head and/or hand positioning；
The position of head and/or hand is judged；
Tired judgement is carried out according to human skeleton model and/or head and/or hand position.
Described image acquisition device 21 is high-definition camera, is mounted on side on steering position, and just towards operator seat
It sets, is sent to processing unit 22 for acquiring the image of driver, and by the image information of acquisition.
When the processing unit 22 executes the step executed in the driver fatigue monitoring and pre-alarming method, based on deep
Learning art is spent, is realized using lightweight convolutional neural networks and human testing is carried out to the image of acquisition.
When detecting human body in the image of acquisition, human skeleton model detection is carried out to image.Carry out human skeleton mould
Type detection includes: to be handled using depth convolutional neural networks input picture, exports each position of human body upper body in picture
Location and corresponding confidence level；The interconnection vector field between each position of human body upper body is predicted to indicate the company between each position
Connect relationship；The interconnection vector field between each site location and position is made inferences to obtain human skeleton model using greedy algorithm.
When detecting human body in the image of acquisition, head is carried out to the image of acquisition and/or hand positions.Using vision
Detection technique passes through lightweight target detection network implementations head and/or hand detection and localization.It is fixed according to head and/or hand
Position testing result judges the position on head and/or hand comprising steps of calculating the centre bit that head part is under normal circumstances
It sets；Detect whether that hand information occurs in the picture；Calculate the center for appearing in the hand in image.Because with head
The different conversion angles in portion, the center on head will appear deflection, by the location information for detecting head in every frame image
Count the center that head part under normal circumstances is in.
By the attitudes vibration of skeleton pattern, judge whether driver is in a state of fatigue.Head and hand center
Distance continuous a period of time is less than the distance threshold of setting, determines that driver is in a state of fatigue.Hand center continuous one
The scope of activities of section time is less than the threshold value of setting, determines driver's head center position in a state of fatigue continuous a period of time
Scope of activities be less than setting threshold value, determine driver it is in a state of fatigue.
When the processing unit 22 judges that driver is in a state of fatigue, warning information is exported by output device 23.Institute
Stating warning information is at least one of voice reminder, early-warning lamp flashing, tactile alert.
Embodiment 3
A kind of driver fatigue monitoring and pre-alarming method, this method to the image of acquisition carry out Characteristics of drivers' behavior analysis and
Driver's facial-feature analysis, Characteristics of drivers' behavior analysis include that human skeleton model detection, right is carried out to driver
Driver carries out head and/or hand positions and judges its position；Driver's facial-feature analysis includes to driver's eye
At least one of portion's signature analysis and mouth feature analysis, judge whether driver frequency of wink variation, closed-eye time occurs
The too long or typical tired facial characteristics such as yawn, according to Characteristics of drivers' behavior analysis and driver's facial-feature analysis,
Whether comprehensive descision driver there is fatigue state.The system for executing the driver fatigue monitoring and pre-alarming method, Image Acquisition
Module includes at least two camera, and one is mounted on side on steering position, and just towards steering position, for adopting
Collect the image of driving behavior；Another is mounted on immediately ahead of driver, and towards driver's face, for acquiring driver face
The image transmitting of the image in portion, two image acquisition devices is handled to processing unit.
It should be noted that the above examples are only used to illustrate the technical scheme of the present invention, rather than its limitations；Although preceding
Stating embodiment, invention is explained in detail, it should be appreciated by those skilled in the art: it can be to previous embodiment
The technical solution of record is modified, or equivalent substitution of some or all of the technical features, and these are replaced,
The range for technical solution of the present invention that it does not separate the essence of the corresponding technical solution.
Claims (10)
1. a kind of driver fatigue monitoring and pre-alarming method, including acquisition image and output giving fatigue pre-warning information, it is characterised in that: also
Include:
Human testing is carried out to the image of acquisition；
Carry out human skeleton model detection；
Carry out head and/or hand positioning；
The position of head and/or hand is judged；
Tired judgement is carried out according to human skeleton model and/or head and/or hand position.
2. driver fatigue monitoring and pre-alarming method as described in claim 1, it is characterised in that: carry out human body to the image of acquisition
Detection is based on deep learning technology, using lightweight convolutional neural networks.
3. driver fatigue monitoring and pre-alarming method as claimed in claim 2, it is characterised in that: the lightweight convolutional Neural net
Network is by end-to-end single phase training.
4. driver fatigue monitoring and pre-alarming method as claimed in claim 3, it is characterised in that: the lightweight convolutional Neural net
There are network 2 different 3x3 convolution kernels to carry out convolution to characteristic pattern.
5. driver fatigue monitoring and pre-alarming method as claimed in claim 4, it is characterised in that: described 2 different 3x3 convolution
Core one is used to export the class probability of human body, and one for exporting the location information of human body frame.
6. driver fatigue monitoring and pre-alarming method as claimed in claim 5, it is characterised in that: use non-maxima suppression method
The result that 3x3 convolution kernel is exported is comprehensive, forms human detection result.
7. driver fatigue monitoring and pre-alarming method as described in claim 1, it is characterised in that: detect people in the image of acquisition
When body, human skeleton model detection is carried out to image.
8. driver fatigue monitoring and pre-alarming method as claimed in claim 7, it is characterised in that: carry out human skeleton model detection
Include:
Input picture is handled using depth convolutional neural networks, exports each position of human body upper body position locating in picture
It sets and corresponding confidence level；
The interconnection vector field between each position of human body upper body is predicted to indicate the connection relationship between each position；
The interconnection vector field between each site location and position is made inferences to obtain human skeleton model using greedy algorithm.
9. driver fatigue monitoring and pre-alarming method as claimed in claim 8, it is characterised in that: each portion of the output human body upper body
Position includes: the location of in picture
Extract at least one position in human eye, ear, nose, neck, shoulder joint, elbow joint, wrist joint and hip joint
Center as key point；
Export the position for the key point extracted.
10. a kind of driver fatigue monitoring and warning system, comprising: image collecting device, processing unit and output device, feature
Be: the system is for implementing such as the described in any item driver fatigue monitoring and pre-alarming methods of claim 1-9, the processing
Device executes the step of the method:
Human testing is carried out to the image of acquisition；
Carry out human skeleton model detection；
Carry out head and/or hand positioning；
The position of head and/or hand is judged；
Tired judgement is carried out according to human skeleton model and/or head and/or hand position.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910352155.4A CN110147738B (en) | 2019-04-29 | 2019-04-29 | Driver fatigue monitoring and early warning method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910352155.4A CN110147738B (en) | 2019-04-29 | 2019-04-29 | Driver fatigue monitoring and early warning method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110147738A true CN110147738A (en) | 2019-08-20 |
| CN110147738B CN110147738B (en) | 2021-01-22 |
Family
ID=67593830
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910352155.4A Active CN110147738B (en) | 2019-04-29 | 2019-04-29 | Driver fatigue monitoring and early warning method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110147738B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110717461A (en) * | 2019-10-12 | 2020-01-21 | 广东电网有限责任公司 | Fatigue state identification method, device and equipment |
| CN111243236A (en) * | 2020-01-17 | 2020-06-05 | 南京邮电大学 | Fatigue driving early warning method and system based on deep learning |
| CN111325872A (en) * | 2020-01-21 | 2020-06-23 | 和智信(山东)大数据科技有限公司 | Driver driving abnormity detection equipment and detection method based on computer vision |
| CN111476114A (en) * | 2020-03-20 | 2020-07-31 | 深圳追一科技有限公司 | Fatigue detection method, device, terminal equipment and storage medium |
| CN113743279A (en) * | 2021-08-30 | 2021-12-03 | 山东大学 | Ship pilot state monitoring method, system, storage medium and equipment |
| CN115035502A (en) * | 2022-07-08 | 2022-09-09 | 北京百度网讯科技有限公司 | Driver behavior monitoring method and device, electronic equipment and storage medium |
| CN115471826A (en) * | 2022-08-23 | 2022-12-13 | 中国航空油料集团有限公司 | Method and device for judging safe driving behavior of aircraft refueling truck and safe operation and maintenance system |
| CN116311181A (en) * | 2023-03-21 | 2023-06-23 | 重庆利龙中宝智能技术有限公司 | Method and system for rapidly detecting abnormal driving |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104013414A (en) * | 2014-04-30 | 2014-09-03 | 南京车锐信息科技有限公司 | Driver fatigue detecting system based on smart mobile phone |
| CN104574817A (en) * | 2014-12-25 | 2015-04-29 | 清华大学苏州汽车研究院（吴江） | Machine vision-based fatigue driving pre-warning system suitable for smart phone |
| JP2015156877A (en) * | 2012-05-18 | 2015-09-03 | 日産自動車株式会社 | Driver's physical state adaptation apparatus, and road map information construction method |
| CN106218405A (en) * | 2016-08-12 | 2016-12-14 | 深圳市元征科技股份有限公司 | Fatigue driving monitoring method and cloud server |
| CN106845430A (en) * | 2017-02-06 | 2017-06-13 | 东华大学 | Pedestrian detection and tracking based on acceleration region convolutional neural networks |
| CN107886069A (en) * | 2017-11-10 | 2018-04-06 | 东北大学 | A kind of multiple target human body 2D gesture real-time detection systems and detection method |
| CN108038453A (en) * | 2017-12-15 | 2018-05-15 | 罗派智能控制技术（上海）有限公司 | A kind of driver's state-detection and identifying system based on RGBD |
| CN108038469A (en) * | 2017-12-27 | 2018-05-15 | 百度在线网络技术（北京）有限公司 | Method and apparatus for detecting human body |
| CN108229390A (en) * | 2018-01-02 | 2018-06-29 | 济南中维世纪科技有限公司 | Rapid pedestrian detection method based on deep learning |
| CN108460362A (en) * | 2018-03-23 | 2018-08-28 | 成都品果科技有限公司 | A kind of system and method for detection human body |
-
2019
- 2019-04-29 CN CN201910352155.4A patent/CN110147738B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015156877A (en) * | 2012-05-18 | 2015-09-03 | 日産自動車株式会社 | Driver's physical state adaptation apparatus, and road map information construction method |
| CN104013414A (en) * | 2014-04-30 | 2014-09-03 | 南京车锐信息科技有限公司 | Driver fatigue detecting system based on smart mobile phone |
| CN104574817A (en) * | 2014-12-25 | 2015-04-29 | 清华大学苏州汽车研究院（吴江） | Machine vision-based fatigue driving pre-warning system suitable for smart phone |
| CN106218405A (en) * | 2016-08-12 | 2016-12-14 | 深圳市元征科技股份有限公司 | Fatigue driving monitoring method and cloud server |
| CN106845430A (en) * | 2017-02-06 | 2017-06-13 | 东华大学 | Pedestrian detection and tracking based on acceleration region convolutional neural networks |
| CN107886069A (en) * | 2017-11-10 | 2018-04-06 | 东北大学 | A kind of multiple target human body 2D gesture real-time detection systems and detection method |
| CN108038453A (en) * | 2017-12-15 | 2018-05-15 | 罗派智能控制技术（上海）有限公司 | A kind of driver's state-detection and identifying system based on RGBD |
| CN108038469A (en) * | 2017-12-27 | 2018-05-15 | 百度在线网络技术（北京）有限公司 | Method and apparatus for detecting human body |
| CN108229390A (en) * | 2018-01-02 | 2018-06-29 | 济南中维世纪科技有限公司 | Rapid pedestrian detection method based on deep learning |
| CN108460362A (en) * | 2018-03-23 | 2018-08-28 | 成都品果科技有限公司 | A kind of system and method for detection human body |
Non-Patent Citations (2)
| Title |
|---|
| CHAO YAN 等: "Recognizing driver inattention by convolutional neural networks", 《2015 8TH INTERNATIONAL CONGRESS ON IMAGE AND SIGNAL PROCESSING (CISP)》 * |
| 赵雪鹏 等: "基于级联卷积神经网络的疲劳检测", 《光电子·激光》 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110717461A (en) * | 2019-10-12 | 2020-01-21 | 广东电网有限责任公司 | Fatigue state identification method, device and equipment |
| WO2021068781A1 (en) * | 2019-10-12 | 2021-04-15 | 广东电网有限责任公司电力科学研究院 | Fatigue state identification method, apparatus and device |
| CN111243236A (en) * | 2020-01-17 | 2020-06-05 | 南京邮电大学 | Fatigue driving early warning method and system based on deep learning |
| CN111325872A (en) * | 2020-01-21 | 2020-06-23 | 和智信(山东)大数据科技有限公司 | Driver driving abnormity detection equipment and detection method based on computer vision |
| CN111325872B (en) * | 2020-01-21 | 2021-03-16 | 和智信(山东)大数据科技有限公司 | Driver driving abnormity detection method based on computer vision |
| CN111476114A (en) * | 2020-03-20 | 2020-07-31 | 深圳追一科技有限公司 | Fatigue detection method, device, terminal equipment and storage medium |
| CN113743279A (en) * | 2021-08-30 | 2021-12-03 | 山东大学 | Ship pilot state monitoring method, system, storage medium and equipment |
| CN113743279B (en) * | 2021-08-30 | 2023-10-13 | 山东大学 | Ship operator state monitoring method, system, storage medium and equipment |
| CN115035502A (en) * | 2022-07-08 | 2022-09-09 | 北京百度网讯科技有限公司 | Driver behavior monitoring method and device, electronic equipment and storage medium |
| CN115471826A (en) * | 2022-08-23 | 2022-12-13 | 中国航空油料集团有限公司 | Method and device for judging safe driving behavior of aircraft refueling truck and safe operation and maintenance system |
| CN115471826B (en) * | 2022-08-23 | 2024-03-26 | 中国航空油料集团有限公司 | Method and device for judging safe driving behavior of aviation fueller and safe operation and maintenance system |
| CN116311181A (en) * | 2023-03-21 | 2023-06-23 | 重庆利龙中宝智能技术有限公司 | Method and system for rapidly detecting abnormal driving |
| CN116311181B (en) * | 2023-03-21 | 2023-09-12 | 重庆利龙中宝智能技术有限公司 | Method and system for rapidly detecting abnormal driving |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110147738B (en) | 2021-01-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110147738A (en) | A kind of driver fatigue monitoring and pre-alarming method and system | |
| CN107704805B (en) | Method for detecting fatigue driving, automobile data recorder and storage device | |
| CN104794855B (en) | Driver attention's comprehensive evaluating device | |
| CN105069976B (en) | A kind of fatigue detecting and traveling record integrated system and fatigue detection method | |
| CN104574817A (en) | Machine vision-based fatigue driving pre-warning system suitable for smart phone | |
| CN105956548A (en) | Driver fatigue state detection method and device | |
| CN105286802B (en) | Driver Fatigue Detection based on video information | |
| CN107038422A (en) | The fatigue state recognition method of deep learning is constrained based on space geometry | |
| CN110368275A (en) | A kind of blind-guidance robot and blind guiding system, blind-guiding method | |
| CN104574819B (en) | Fatigued drive detection method based on mouth features | |
| CN104361321A (en) | Methods of judging fall behaviors and body balance for old people | |
| CN107229922A (en) | A kind of fatigue driving monitoring method and device | |
| CN106965675A (en) | A kind of lorry swarm intelligence safety work system | |
| CN109334563A (en) | A kind of anticollision method for early warning based on road ahead pedestrian and bicyclist | |
| CN113033503A (en) | Multi-feature fusion dangerous driving behavior detection method and system | |
| CN110448316A (en) | Data processing equipment and method, wake up system and recording medium at monitoring system | |
| Celona et al. | A multi-task CNN framework for driver face monitoring | |
| Pech et al. | Head tracking based glance area estimation for driver behaviour modelling during lane change execution | |
| CN109460704A (en) | A kind of fatigue detection method based on deep learning, system and computer equipment | |
| Lee et al. | A study on feature extraction methods used to estimate a driver’s level of drowsiness | |
| CN106295474A (en) | The fatigue detection method of deck officer, system and server | |
| CN115937830A (en) | Special vehicle-oriented driver fatigue detection method | |
| CN116259002A (en) | Human body dangerous behavior analysis method based on video | |
| CN107967944A (en) | A kind of outdoor environment big data measuring of human health method and platform based on Hadoop | |
| CN109308467A (en) | Traffic accident prior-warning device and method for early warning based on machine learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |