CN108764034A - A kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera - Google Patents
A kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera Download PDFInfo
- Publication number
- CN108764034A CN108764034A CN201810351258.4A CN201810351258A CN108764034A CN 108764034 A CN108764034 A CN 108764034A CN 201810351258 A CN201810351258 A CN 201810351258A CN 108764034 A CN108764034 A CN 108764034A
- Authority
- CN
- China
- Prior art keywords
- driving behavior
- driver
- dangerous driving
- interest
- infrared camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
- G06V40/166—Detection; Localisation; Normalisation using acquisition arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Biomedical Technology (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Traffic Control Systems (AREA)
Abstract
The present invention relates to a kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera, includes the following steps：1. driver's Face datection, driver's cabin scene near-infrared image data are acquired offline by vehicle-mounted near infrared camera, image data human face region is labeled, off-line training Adaboost graders search for face area-of-interest as Face datection grader, using Face datection grader；2. dangerous driving behavior detects, it is based on face area-of-interest, increases background area, dangerous driving behavior detection is carried out using depth convolutional neural networks；3. dangerous driving behavior early warning carries out multiframe confirmation to dangerous driving behavior using time series data, such as reaches the continuous frame number threshold value of setting, then send out vision pre-warning signal and/or sense of hearing pre-warning signal.The class dangerous driving behavior of diverting attention of driver can be identified in the present invention and early warning remains to normal specification driving condition, to improve travel safety to prompt driver to change behavior of diverting attention immediately.
Description
Technical field
The present invention relates to field of vehicle control more particularly to a kind of driving behaviors of diverting attention based on driver's cabin near infrared camera
Method for early warning.
Background technology
According to road traffic accident data statistics, the traffic accident for having more than half be by driver hazardous act or
Caused by errant vehicle operation, however, being largely the row of diverting attention due to fatigue driving or driver in such human accident
For caused by.Existing passenger car and the active safety system of commercial car seldom relate to driving behavior analysis and remind work(
Can, for commercial vehicle, long-duration driving and drives over a long distance and make the generation of above-mentioned dangerous driving conditions
Probability higher.Existing most of commercial commerial vehicle does not have the driving behavior monitoring system of perfect in shape and function, though some have
Vehicle-mounted video recording and operation note function, but do not have fatigue driving or dangerous driving behavior warning function, thus can not be effective
Guarantee driving safety.
Invention content
In order to solve the above-mentioned technical problem the present invention, provides a kind of driving behavior of diverting attention based on driver's cabin near infrared camera
Method for early warning carries out danger of the driver in driving procedure driving behavior of diverting attention using driver's cabin near-infrared vision data
Intellectual analysis, can be identified the dangerous driving behavior diverted attention and early warning is protected with prompting driver to change behavior of diverting attention immediately
Normal specification driving behavior is held, to improve travel safety.
The above-mentioned technical problem of the present invention is mainly to be addressed by following technical proposals：One kind of the invention is based on driving
The driving behavior method for early warning of diverting attention for sailing room near infrared camera, includes the following steps：
Driver's Face datection：Driver's cabin scene near-infrared image data are acquired offline by vehicle-mounted near infrared camera, it is right
Image data human face region is labeled, and off-line training Adaboost graders are examined as Face datection grader using face
Survey the grader search face region (ROI) interested；
2. dangerous driving behavior detects：Based on face area-of-interest, increases background area, utilize depth convolutional Neural net
Network carries out dangerous driving behavior detection；
3. dangerous driving behavior early warning：Multiframe confirmation is carried out to dangerous driving behavior using time series data, such as reaches setting
Continuous frame number threshold value, then send out vision pre-warning signal and/or sense of hearing pre-warning signal.
Dangerous driving behavior in the present invention refers to driver in the cigarette smoking occurred on the run and takes phone row
For.The present invention utilizes driver's cabin near-infrared vision data, and intelligence is carried out to dangerous driving behavior of the driver in driving procedure
Analysis, can be identified the dangerous driving behavior diverted attention and early warning is remained to prompting driver to change behavior of diverting attention immediately
Normal specification driving behavior, to improve travel safety.The present invention can be integrated in the cab system of view-based access control model, be formed tired
Please it sails and behavior joint-detection early warning system of diverting attention, there is ductility.
Preferably, 1. the step includes the following steps：
(1.1) driver's cabin scene near-infrared image data are acquired by vehicle-mounted near infrared camera offline, to image data people
Face region is labeled, and is generated positive sample image 12000 and is opened, and negative sample image 20000 is opened, off-line training Cascade H aar features or
The Adaboost graders of LBP features are as Face datection grader；
(1.2) Face datection grader is utilized, is slided by setting step-length and setting scale in driver's cabin area-of-interest
Window searches for face area-of-interest, for each sliding window, calculates the Haar feature operators applied when off-line training or LBP features
Operator, and gained feature vector is sent into Soft-Cascade graders and carries out two classification, it is consistent to classification results using NMS
Sliding window region is overlapped to merge, it is online to obtain driver's face area-of-interest；
(1.3) the driver's face area-of-interest obtained according to on-line checking, extraction angle point calculate, and utilize Lucas-
Kanade optical flow approach calculates the movable information of corner feature, and updates corner feature position, is based on similitude projection relation
It is assumed that calculating the location updating of present frame face using RANSAC methods, and merged, is obtained with present frame Face datection result
Obtain final face area-of-interest.
In the technical program, positive and negative samples image number can change as needed.When training Face datection grader,
It is also excavated using difficult example and the methods of Active Learning training for promotion effect, grader cascades the number of plies as configurable parameter.It drives
Room area-of-interest range and sliding window step-length and scale are configurable parameter, should be adjusted according to different camera installation parameters
It is whole.
Preferably, 2. the step is：Final face area-of-interest based on acquisition, by 1.2 proportionality coefficient,
Increase background area, generate the smoking area-of-interest for analyzing dangerous driving behavior and takes phone area-of-interest, profit
Dangerous driving behavior detection is carried out with depth convolutional neural networks, detection method includes the following steps：
(2.1) depth convolutional neural networks design：Behavioral value network structure, packet are designed using depth convolutional neural networks
It includes region and suggests network structure and region Recurrent networks structure；
(2.2) off-line training depth convolutional neural networks structure：Smoking is respectively trained and takes the inspection of two kinds of behaviors of phone
Survey grid network, the training for detecting network are divided into two steps, and the first step trains region to suggest that network structure, second step train region to return
Return network structure；
(2.3) front end applications depth convolutional neural networks structure：In smoking area-of-interest and take phone region of interest
Dangerous driving behavior detection is carried out in domain.
Preferably, in the step (2.1), the region suggests that network structure is made of convolutional layer, inputs and is
16 × 16 × 3 image datas export the confidence level for region Suggestion box and rough vertex position；The region Recurrent networks
Structure is made of convolutional layer and full articulamentum, is inputted as 32 × 32 × 3 image datas, is exported the confidence level for dangerous driving behavior
And accurate vertex position.
Preferably, step (2.2) off-line training depth convolutional neural networks structure includes：Acquisition includes smoking
The image pattern of behavior and each 50000 comprising the image pattern for taking phone behavior, artificial mark generate corresponding training label,
Cigarette smoking detection network is respectively trained and takes phone behavioral value network, each training for detecting network is divided into two steps
Suddenly, Classification Loss function L_cls is set as cross entropy, returns the Euclidean distance that loss function L_reg is set as bounding box vertex；
The first step trains region to suggest that network structure, training sample are generated according to label, and training loss function setting is as follows：
Loss1=α 1 × L_cls+ β 1 × L_reg, α 1, β 1 are configurable parameter；
Second step trains region Recurrent networks structure, training sample to suggest network structure in original training sample according to region
Collection output result generates, and training loss function setting is as follows：
Loss2=α 2 × L_cls+ β 2 × L_reg, α 2, β 2 are configurable parameter；
It is trained by the way of stochastic gradient descent.Learning rate, batch size etc. are configurable parameter (default value
For 64).
The number of the image pattern of acquisition can change as needed.Using image gamut, it is several how convert, can also to figure
Decent is expanded.If the image pattern quantity of acquisition is sufficiently large, the step of being expanded by transformation can be omitted.It examines
Consider cigarette smoking and takes two kinds of behavior area-of-interests when existing characteristics otherness and forward direction are applied between phone behavior
Independence, therefore need to be respectively trained cigarette smoking detection network and take phone behavioral value network.The default value point of α 1, β 1
Not Wei 0.6 and 0.4, α 2, β 2 default value be respectively 0.4 and 0.6.
Preferably, step (2.3) front end applications depth convolutional neural networks structure includes：To step (2.2)
The cigarette smoking detection network and take the progress rarefaction of phone behavioral value network and quantization compression behaviour that middle off-line training obtains
Make, verify compressed detection neural network accuracy, the dangerous driving behavior detection smoked in smoking area-of-interest is connecing
Make a phone call to carry out in area-of-interest taking the dangerous driving behavior detection of phone.
Preferably, 3. the step is：Define the value that current time driver takes telephone state S1, normal driving
When S1=0, S1=1 when taking phone；The value for defining current time driver's smoking state S2, S2=0 when normal driving, smoking
When S2=1；Multiframe confirmation is carried out to dangerous driving behavior using time series data, is included the following steps：
(3.1) it is C1 to take phone hazardous act confidence level variable, and smoking hazardous act confidence level variable is C2, calculating side
Formula is as follows：
C1t+1=max (0, C1t+(S2-1)*K1+S2*K2)
C2t+1=max (0, C2t+(S2-1)*K1′+S2*K2′)
Wherein, t is discrete sampling periodicity, and the value of t is related near infrared camera frequency acquisition in practical application；K1,
K2 and K1 ', K2 ' are configurable threshold parameter；Such as when taking phone, default value is：K1=0.8 × K2=0.3 × T1；
(3.2) setting takes phone hazardous act confidence level threshold value of warning T1 and smoking hazardous act confidence level threshold value of warning
T2 sends out vision pre-warning signal and/or sense of hearing pre-warning signal if C1 >=T1 or C2 >=T2 by way of event triggering.
Distinguish icon type, prompt tone and the advisory frequency of the different corresponding pre-warning signals of behavior of diverting attention.That is it when C1 >=T1, sends out and connects
It makes a phone call early warning；When C2 >=T2, smoking early warning is sent out.Driver is reminded to pay attention to simultaneously changing hazard driving behavior in time.
Preferably, the driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera, for commercial car,
Including step, 4. dangerous driving behavior is recorded a video and is reported：When detecting the dangerous driving behavior of driver, to dangerous driving
Behavior is recorded a video, and Video data is reported to distal end control platform.
Preferably, 4. the step is：When detecting the dangerous driving behavior of driver, caching record early warning
Signal sent out before the moment 10 seconds and in the rear 10 second time (i.e. totally 20 seconds) driver's cabin video data, pass through real-time recording and event
The mode of record records dangerous driving behavior, and dangerous driving behavior record is reported to distal tube by communication module
Control platform.It is data cached to be named with time and early warning type, hard disk is written with the coding mode of H264 or H265.
According to vehicle bus data access situation, preferably, recorded video can be corresponded to the speed at moment and turn to letter
Number it is incorporated into video stream file in a manner of intelligent frame or character adding, so as to follow-up other application.
The beneficial effects of the invention are as follows：Using driver's cabin near-infrared vision data, to danger of the driver in driving procedure
Danger driving behavior of diverting attention carries out intellectual analysis, the dangerous driving behavior diverted attention can be identified and early warning, to prompt driver
Change behavior of diverting attention immediately, normal specification driving behavior is remained to, to improve travel safety.
Description of the drawings
Fig. 1 is a kind of system structure diagram of driving behavior early warning system of diverting attention in the present invention.
Fig. 2 is a kind of flow chart of the present invention.
Fig. 3 is a kind of schematic diagram of each area-of-interest in driver's cabin scene near-infrared image in the present invention.
Fig. 4 is a kind of structural schematic diagram that network is suggested in region in the present invention.
Fig. 5 is a kind of structural schematic diagram of region Recurrent networks in the present invention.
1. face area-of-interest in figure, 2. smoking area-of-interests, 3. take phone area-of-interest.
Specific implementation mode
Below with reference to the embodiments and with reference to the accompanying drawing the technical solutions of the present invention will be further described.
Embodiment：A kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera of the present embodiment, it is used
Divert attention driving behavior early warning system as shown in Figure 1, system input be driver's cabin image, export and (taken for violation driving behavior
Phone and smoke) pre-warning signal and related early warning segment (video flowing or image) data.The driving row of diverting attention of this system realization
It can be turned on and off as needed for warning function.
Driving behavior method for early warning divert attention as shown in Fig. 2, including the following steps：
1. driver's Face datection：
(1.1) driver's cabin scene near-infrared image data are acquired by vehicle-mounted near infrared camera offline, to image data people
Face region is labeled, and is generated positive sample image 12000 and is opened, and negative sample image 20000 is opened, off-line training Cascade H aar features or
The Adaboost graders of LBP features are as Face datection grader；
(1.2) Face datection grader is utilized, is slided by setting step-length and setting scale in driver's cabin area-of-interest
Window searches for face area-of-interest, for each sliding window, calculates the Haar feature operators applied when off-line training or LBP features
Operator, and gained feature vector is sent into Soft-Cascade graders and carries out two classification, it is consistent to classification results using NMS
Sliding window region is overlapped to merge, it is online to obtain driver's face area-of-interest；
(1.3) the driver's face area-of-interest obtained according to on-line checking, extraction angle point (Harris or FAST) are counted
It calculates, the movable information of corner feature is calculated using Lucas-Kanade optical flow approach, and update corner feature position, be based on phase
Like property projection relation it is assumed that using RANSAC methods calculate present frame face location updating, and with present frame Face datection knot
Fruit is merged, and final face area-of-interest is obtained.
2. dangerous driving behavior detects：Final face area-of-interest 1 based on acquisition is increased by 1.2 proportionality coefficient
Part background area, as shown in figure 3, generating the smoking area-of-interest 2 for analyzing dangerous driving behavior and taking phone sense
Interest region 3, face area-of-interest includes head and the face of people, including the region at left and right ear position is respectively to smoke
Area-of-interest, including the region at nose and face position are to take phone area-of-interest, utilize depth convolutional neural networks
Dangerous driving behavior detection is carried out, detection method includes the following steps：
(2.1) depth convolutional neural networks design：Behavioral value network structure, packet are designed using depth convolutional neural networks
It includes region and suggests network structure and region Recurrent networks structure；The region suggests that network structure is made of convolutional layer, such as Fig. 4
It is shown, it inputs as 16 × 16 × 3 image datas, exports the confidence level for region Suggestion box and rough vertex position；Described
Region Recurrent networks structure is made of convolutional layer and full articulamentum, as shown in figure 5, input is 32 × 32 × 3 image datas, output
Confidence level for dangerous driving behavior and accurate vertex position；
(2.2) off-line training depth convolutional neural networks structure：Image pattern of the acquisition comprising cigarette smoking and comprising connecing
It makes a phone call each 50000 of the image pattern of behavior, artificial mark generates corresponding training label, cigarette smoking detection net is respectively trained
Network and phone behavioral value network is taken, each training for detecting network is divided into two steps, the L_cls settings of Classification Loss function
For cross entropy, the Euclidean distance that loss function L_reg is set as bounding box vertex is returned；
The first step trains region to suggest that network structure, training sample are generated according to label, and training loss function setting is as follows：
Loss1=α 1 × L_cls+ β 1 × L_reg, α 1, β 1 are configurable parameter, and α 1,1 default values of β are respectively 0.6 and
0.4；
Second step trains region Recurrent networks structure, training sample to suggest network structure in original training sample according to region
Collection output result generates, and training loss function setting is as follows：
Loss2=α 2 × L_cls+ β 2 × L_reg, α 2, β 2 are configurable parameter, and 2 default value of d2, β is respectively 0.6 and
0.4；
It is trained by the way of stochastic gradient descent；
(2.3) front end applications depth convolutional neural networks structure：The cigarette smoking that off-line training in step (2.2) is obtained
It detects network and takes phone behavioral value network and carry out rarefaction and quantization squeeze operation, it is smart to verify compressed detection network
Degree, the dangerous driving behavior detection smoked in smoking area-of-interest, is connect taking in phone area-of-interest
The dangerous driving behavior detection made a phone call.
3. dangerous driving behavior early warning：It defines current time driver and takes the value of telephone state S1, S1 when normal driving
=0, S1=1 when taking phone；The value for defining current time driver's smoking state S2, S2=0 when normal driving, S2 when smoking
=1；Multiframe confirmation is carried out to dangerous driving behavior using time series data, is included the following steps：
(3.1) it is C1 to take phone hazardous act confidence level variable, and smoking hazardous act confidence level variable is C2, calculating side
Formula is as follows：
C1t+1=max (0, C1t+(S2-1)*K1+S2*K2)
C2t+1=max (0, C2t+(S2-1)*K1′+S2*K2′)
Wherein, t is discrete sampling periodicity, and the value of t is related near infrared camera frequency acquisition in practical application；K1,
K2 and K1 ', K2 ' are configurable threshold parameter；
(3.2) setting takes phone hazardous act confidence level threshold value of warning T1 and smoking hazardous act confidence level threshold value of warning
T2 sends out vision pre-warning signal and sense of hearing pre-warning signal by way of event triggering, distinguishes if C1 >=T1 or C2 >=T2
Difference divert attention the corresponding pre-warning signal of behavior icon type, prompt tone and advisory frequency, i.e. C1 >=T1 when, send out and take electricity
Early warning is talked about, when C2 >=T2, smoking early warning is sent out, driver is reminded to pay attention to simultaneously changing hazard driving behavior in time.
For commercial car platform, can also setting steps 4. dangerous driving behavior record a video and report：When detecting that driver deposits
In dangerous driving behavior, caching record pre-warning signal was sent out before the moment 10 seconds and the driver's cabin video counts in the rear 10 second time
According to being recorded to dangerous driving behavior by way of real-time recording and logout, and driven danger by communication module
It sails behavior record and is reported to distal end control platform.
The present invention utilizes driver's cabin near-infrared vision data, diverts attention driving behavior to danger of the driver in driving procedure
Intellectual analysis is carried out, the dangerous driving behavior diverted attention can be identified and early warning, to prompt driver to change row of diverting attention immediately
Normal specification driving behavior to be remained to, to improve travel safety.The present invention can be integrated in the driver's cabin system of view-based access control model
System, forms fatigue driving and behavior joint-detection early warning system of diverting attention, and system has certain ductility.The present invention is according to sequential
The dangerous driving behavior pre-warning signal of confirmation has the videograph and upload function of event trigger type concurrently, has commercial operation
Vehicle platform application prospect.
Claims (9)
1. a kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera, it is characterised in that include the following steps：
1. driver's Face datection：Driver's cabin scene near-infrared image data are acquired offline by vehicle-mounted near infrared camera, to figure
As data human face region is labeled, off-line training Adaboost graders utilize Face datection as Face datection grader
Grader searches for face area-of-interest；
2. dangerous driving behavior detects：Based on face area-of-interest, increase background area, using depth convolutional neural networks into
Row dangerous driving behavior detects；
3. dangerous driving behavior early warning：Multiframe confirmation is carried out to dangerous driving behavior using time series data, such as reaches the company of setting
Continuous frame number threshold value, then send out vision pre-warning signal and/or sense of hearing pre-warning signal.
2. a kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera according to claim 1, special
Sign is 1. the step includes the following steps：
(1.1) driver's cabin scene near-infrared image data are acquired by vehicle-mounted near infrared camera offline, to image data face area
Domain is labeled, and is generated positive sample image 12000 and is opened, negative sample image 20000 is opened, off-line training Cascade H aar features or LBP
The Adaboost graders of feature are as Face datection grader；
(1.2) Face datection grader is utilized, is searched by setting step-length and setting scale sliding window in driver's cabin area-of-interest
Rope face area-of-interest calculates the Haar feature operators applied when off-line training or LBP features is calculated for each sliding window
Son, and gained feature vector is sent into Soft-Cascade graders and carries out two classification, utilize the NMS weights consistent to classification results
Sliding window region is closed to merge, it is online to obtain driver's face area-of-interest；
(1.3) the driver's face area-of-interest obtained according to on-line checking, extraction angle point calculate, and utilize Lucas-Kanade
Optical flow approach calculates the movable information of corner feature, and updates corner feature position, based on similitude projection relation it is assumed that profit
The location updating of present frame face is calculated with RANSAC methods, and is merged with present frame Face datection result, is obtained final
Face area-of-interest.
3. a kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera according to claim 1, special
Sign is 2. the step is：Final face area-of-interest based on acquisition increases background area by 1.2 proportionality coefficient
Domain generates the smoking area-of-interest for analyzing dangerous driving behavior and takes phone area-of-interest, utilizes depth convolution
Neural network carries out dangerous driving behavior detection, and detection method includes the following steps：
(2.1) depth convolutional neural networks design：Behavioral value network structure, including area are designed using depth convolutional neural networks
Suggest network structure and region Recurrent networks structure in domain；
(2.2) off-line training depth convolutional neural networks structure：Smoking is respectively trained and takes the detection net of two kinds of behaviors of phone
Network, the training for detecting network are divided into two steps, and the first step trains region to suggest that network structure, second step train region to return net
Network structure；
(2.3) front end applications depth convolutional neural networks structure：In smoking area-of-interest and take in phone area-of-interest
Carry out dangerous driving behavior detection.
4. a kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera according to claim 3, special
Sign is in the step (2.1) that the region suggests that network structure is made of convolutional layer, inputs as 16 × 16 × 3 figures
As data, the confidence level for region Suggestion box and rough vertex position are exported；The region Recurrent networks structure is by convolution
Layer and full articulamentum are constituted, and input confidence level and the accurate top exported for 32 × 32 × 3 image datas as dangerous driving behavior
Point position.
5. a kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera according to claim 3, special
Sign is that step (2.2) off-line training depth convolutional neural networks structure includes：Acquisition includes the image of cigarette smoking
Sample and each 50000 comprising the image pattern for taking phone behavior, artificial mark generate corresponding training label, suction are respectively trained
Cigarette behavioral value network and phone behavioral value network is taken, each training for detecting network is divided into two steps, Classification Loss
Function L_cls is set as cross entropy, returns the Euclidean distance that loss function L_reg is set as bounding box vertex；
The first step trains region to suggest that network structure, training sample are generated according to label, and training loss function setting is as follows：
Loss1=α 1 × L_cls+ β 1 × L_reg, α 1, β 1 are configurable parameter；
Second step trains region Recurrent networks structure, training sample to suggest that network structure is defeated in original training sample collection according to region
Go out result generation, training loss function setting is as follows：
Loss2=α 2 × L_cls+ β 2 × L_reg, α 2, β 2 are configurable parameter；
It is trained by the way of stochastic gradient descent.
6. a kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera according to claim 3, special
Sign is that step (2.3) front end applications depth convolutional neural networks structure includes：Off-line training in step (2.2) is obtained
The cigarette smoking obtained, which detects network and takes phone behavioral value network, carries out rarefaction and quantization squeeze operation, verifies after compressing
Detection neural network accuracy, smoking area-of-interest in smoke dangerous driving behavior detection, it is interested taking phone
Carry out taking the dangerous driving behavior detection of phone in region.
7. a kind of pre- police of driving behavior of diverting attention based on driver's cabin near infrared camera according to claim 1 or 2 or 3
Method, it is characterised in that the 3. step is：It defines current time driver and takes the value of telephone state S1, S1 when normal driving
=0, S1=1 when taking phone；The value for defining current time driver's smoking state S2, S2=0 when normal driving, S2 when smoking
=1；Multiframe confirmation is carried out to dangerous driving behavior using time series data, is included the following steps：
(3.1) it is C1 to take phone hazardous act confidence level variable, and smoking hazardous act confidence level variable is C2, and calculation is such as
Under：
C1t+1=max (0, C1t+(S2-1)*K1+S2*K2)
C2t+1=max (0, C2t+(S2-1)*K1′+52*K2′)
Wherein, t be discrete sampling periodicity, K1, K2 and K1 ', K2 ' be configurable threshold parameter；
(3.2) setting takes phone hazardous act confidence level threshold value of warning T1 and smoking hazardous act confidence level threshold value of warning T2,
If C1 >=T1 or C2 >=T2, vision pre-warning signal and/or sense of hearing pre-warning signal are sent out by way of event triggering.
8. a kind of pre- police of driving behavior of diverting attention based on driver's cabin near infrared camera according to claim 1 or 2 or 3
Method, it is characterised in that for commercial car, including step 4. record a video and report by dangerous driving behavior：It endangers when detecting that driver exists
It when dangerous driving behavior, records a video to dangerous driving behavior, and Video data is reported to distal end control platform.
9. a kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera according to claim 8, special
Sign is 4. the step is：When detecting the dangerous driving behavior of driver, when caching record pre-warning signal is sent out
First 10 seconds and the driver's cabin video data in the rear 10 second time are carved, to dangerous driving by way of real-time recording and logout
Behavior is recorded, and dangerous driving behavior record is reported to distal end control platform by communication module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810351258.4A CN108764034A (en) | 2018-04-18 | 2018-04-18 | A kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810351258.4A CN108764034A (en) | 2018-04-18 | 2018-04-18 | A kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108764034A true CN108764034A (en) | 2018-11-06 |
Family
ID=64011044
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810351258.4A Pending CN108764034A (en) | 2018-04-18 | 2018-04-18 | A kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108764034A (en) |
Cited By (24)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109902560A (en) * | 2019-01-15 | 2019-06-18 | 浙江师范大学 | A kind of fatigue driving method for early warning based on deep learning |
| CN110046560A (en) * | 2019-03-28 | 2019-07-23 | 青岛小鸟看看科技有限公司 | A kind of dangerous driving behavior detection method and camera |
| CN110059650A (en) * | 2019-04-24 | 2019-07-26 | 京东方科技集团股份有限公司 | Information processing method, device, computer storage medium and electronic equipment |
| CN110222596A (en) * | 2019-05-20 | 2019-09-10 | 浙江零跑科技有限公司 | A kind of driving behavior analysis anti-cheating method of view-based access control model |
| CN110309760A (en) * | 2019-06-26 | 2019-10-08 | 深圳市微纳集成电路与系统应用研究院 | The method that the driving behavior of driver is detected |
| CN110334627A (en) * | 2019-06-26 | 2019-10-15 | 深圳市微纳集成电路与系统应用研究院 | The device and system that the behavior of personnel is detected |
| CN110766131A (en) * | 2019-05-14 | 2020-02-07 | 北京嘀嘀无限科技发展有限公司 | Data processing device and method and electronic equipment |
| CN110807352A (en) * | 2019-08-29 | 2020-02-18 | 浙江零跑科技有限公司 | In-vehicle and out-vehicle scene visual analysis method for dangerous driving behavior early warning |
| CN111090723A (en) * | 2019-11-18 | 2020-05-01 | 清远电力规划设计院有限公司 | Power grid safety production content recommendation method based on knowledge graph |
| CN111222449A (en) * | 2020-01-02 | 2020-06-02 | 上海中安电子信息科技有限公司 | Driver behavior detection method based on fixed camera image |
| CN111242015A (en) * | 2020-01-10 | 2020-06-05 | 同济大学 | Method for predicting driving danger scene based on motion contour semantic graph |
| CN111243235A (en) * | 2020-01-13 | 2020-06-05 | 惠龙易通国际物流股份有限公司 | Driving assistance method and device |
| CN111274881A (en) * | 2020-01-10 | 2020-06-12 | 中国平安财产保险股份有限公司 | Driving safety monitoring method and device, computer equipment and storage medium |
| CN111325058A (en) * | 2018-12-14 | 2020-06-23 | 长沙智能驾驶研究院有限公司 | Driving behavior detection method, device and system and storage medium |
| CN111723602A (en) * | 2019-03-19 | 2020-09-29 | 杭州海康威视数字技术股份有限公司 | Driver behavior recognition method, device, equipment and storage medium |
| CN111753585A (en) * | 2019-03-28 | 2020-10-09 | 北京市商汤科技开发有限公司 | Motion tracking processing method and device, medium, and apparatus |
| CN111746808A (en) * | 2019-03-28 | 2020-10-09 | 空中客车运营有限公司 | Condition recognition device, aircraft cabin and method for monitoring an aircraft cabin |
| CN111860316A (en) * | 2020-07-20 | 2020-10-30 | 上海汽车集团股份有限公司 | Driving behavior recognition method and device and storage medium |
| CN112016457A (en) * | 2020-08-27 | 2020-12-01 | 青岛慕容信息科技有限公司 | Driver distraction and dangerous driving behavior recognition method, device and storage medium |
| CN112017155A (en) * | 2020-07-13 | 2020-12-01 | 浙江大华汽车技术有限公司 | Method, device and system for measuring health sign data and storage medium |
| CN112434612A (en) * | 2020-11-25 | 2021-03-02 | 创新奇智(上海)科技有限公司 | Smoking detection method and device, electronic equipment and computer readable storage medium |
| CN112861608A (en) * | 2020-12-30 | 2021-05-28 | 浙江万里学院 | Detection method and system for distracted driving behaviors |
| WO2021159269A1 (en) * | 2020-02-11 | 2021-08-19 | 云图技术有限公司 | Vision-based safe driving early warning method and system, and storage medium |
| CN116912808A (en) * | 2023-09-14 | 2023-10-20 | 四川公路桥梁建设集团有限公司 | Bridge girder erection machine control method, electronic equipment and computer readable medium |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103593654A (en) * | 2013-11-13 | 2014-02-19 | 智慧城市系统服务（中国）有限公司 | Method and device for face location |
| CN104125636A (en) * | 2013-04-27 | 2014-10-29 | 中国移动通信集团公司 | Base station positioning method and device |
| CN104598892A (en) * | 2015-01-30 | 2015-05-06 | 广东威创视讯科技股份有限公司 | Dangerous driving behavior alarming method and system |
| CN104637246A (en) * | 2015-02-02 | 2015-05-20 | 合肥工业大学 | Driver multi-behavior early warning system and danger evaluation method |
| CN106056071A (en) * | 2016-05-30 | 2016-10-26 | 北京智芯原动科技有限公司 | Method and device for detection of driver' behavior of making call |
| WO2016170786A1 (en) * | 2015-04-21 | 2016-10-27 | パナソニックＩｐマネジメント株式会社 | Information processing system, information processing method, and program |
| CN106156725A (en) * | 2016-06-16 | 2016-11-23 | 江苏大学 | A kind of method of work of the identification early warning system of pedestrian based on vehicle front and cyclist |
| CN106503673A (en) * | 2016-11-03 | 2017-03-15 | 北京文安智能技术股份有限公司 | A kind of recognition methodss of traffic driving behavior, device and a kind of video acquisition device |
| CN106611169A (en) * | 2016-12-31 | 2017-05-03 | 中国科学技术大学 | Dangerous driving behavior real-time detection method based on deep learning |
| CN107133568A (en) * | 2017-03-31 | 2017-09-05 | 浙江零跑科技有限公司 | A kind of speed limit prompting and hypervelocity alarm method based on vehicle-mounted forward sight camera |
| CN107169421A (en) * | 2017-04-20 | 2017-09-15 | 华南理工大学 | A kind of car steering scene objects detection method based on depth convolutional neural networks |
| CN107194346A (en) * | 2017-05-19 | 2017-09-22 | 福建师范大学 | A kind of fatigue drive of car Forecasting Methodology |
| CN107301384A (en) * | 2017-06-09 | 2017-10-27 | 湖北天业云商网络科技有限公司 | A kind of driver takes phone behavioral value method and system |
| CN107491769A (en) * | 2017-09-11 | 2017-12-19 | 中国地质大学（武汉） | Method for detecting fatigue driving and system based on AdaBoost algorithms |
| CN107766829A (en) * | 2017-10-27 | 2018-03-06 | 浙江大华技术股份有限公司 | A kind of method and apparatus of Articles detecting |
-
2018
- 2018-04-18 CN CN201810351258.4A patent/CN108764034A/en active Pending
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104125636A (en) * | 2013-04-27 | 2014-10-29 | 中国移动通信集团公司 | Base station positioning method and device |
| CN103593654A (en) * | 2013-11-13 | 2014-02-19 | 智慧城市系统服务（中国）有限公司 | Method and device for face location |
| CN104598892A (en) * | 2015-01-30 | 2015-05-06 | 广东威创视讯科技股份有限公司 | Dangerous driving behavior alarming method and system |
| CN104637246A (en) * | 2015-02-02 | 2015-05-20 | 合肥工业大学 | Driver multi-behavior early warning system and danger evaluation method |
| WO2016170786A1 (en) * | 2015-04-21 | 2016-10-27 | パナソニックＩｐマネジメント株式会社 | Information processing system, information processing method, and program |
| CN106056071A (en) * | 2016-05-30 | 2016-10-26 | 北京智芯原动科技有限公司 | Method and device for detection of driver' behavior of making call |
| CN106156725A (en) * | 2016-06-16 | 2016-11-23 | 江苏大学 | A kind of method of work of the identification early warning system of pedestrian based on vehicle front and cyclist |
| CN106503673A (en) * | 2016-11-03 | 2017-03-15 | 北京文安智能技术股份有限公司 | A kind of recognition methodss of traffic driving behavior, device and a kind of video acquisition device |
| CN106611169A (en) * | 2016-12-31 | 2017-05-03 | 中国科学技术大学 | Dangerous driving behavior real-time detection method based on deep learning |
| CN107133568A (en) * | 2017-03-31 | 2017-09-05 | 浙江零跑科技有限公司 | A kind of speed limit prompting and hypervelocity alarm method based on vehicle-mounted forward sight camera |
| CN107169421A (en) * | 2017-04-20 | 2017-09-15 | 华南理工大学 | A kind of car steering scene objects detection method based on depth convolutional neural networks |
| CN107194346A (en) * | 2017-05-19 | 2017-09-22 | 福建师范大学 | A kind of fatigue drive of car Forecasting Methodology |
| CN107301384A (en) * | 2017-06-09 | 2017-10-27 | 湖北天业云商网络科技有限公司 | A kind of driver takes phone behavioral value method and system |
| CN107491769A (en) * | 2017-09-11 | 2017-12-19 | 中国地质大学（武汉） | Method for detecting fatigue driving and system based on AdaBoost algorithms |
| CN107766829A (en) * | 2017-10-27 | 2018-03-06 | 浙江大华技术股份有限公司 | A kind of method and apparatus of Articles detecting |
Non-Patent Citations (3)
| Title |
|---|
| T. HOANG NGAN LE ET AL: "Multiple Scale Faster-RCNN Approach to Driver’s Cell-phone Usage and Hands on Steering Wheel Detection", 《2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS》 * |
| 宋保卫等: "《城市轨道交通通信与信号控制》", 28 February 2015 * |
| 张俊友等: "《智能交通系统及应用》", 31 July 2017 * |
Cited By (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111325058A (en) * | 2018-12-14 | 2020-06-23 | 长沙智能驾驶研究院有限公司 | Driving behavior detection method, device and system and storage medium |
| CN111325058B (en) * | 2018-12-14 | 2023-12-01 | 长沙智能驾驶研究院有限公司 | Driving behavior detection method, device, system and storage medium |
| CN109902560A (en) * | 2019-01-15 | 2019-06-18 | 浙江师范大学 | A kind of fatigue driving method for early warning based on deep learning |
| CN111723602A (en) * | 2019-03-19 | 2020-09-29 | 杭州海康威视数字技术股份有限公司 | Driver behavior recognition method, device, equipment and storage medium |
| CN111723602B (en) * | 2019-03-19 | 2023-08-08 | 杭州海康威视数字技术股份有限公司 | Method, device, equipment and storage medium for identifying driver behavior |
| CN110046560B (en) * | 2019-03-28 | 2021-11-23 | 青岛小鸟看看科技有限公司 | Dangerous driving behavior detection method and camera |
| CN111746808A (en) * | 2019-03-28 | 2020-10-09 | 空中客车运营有限公司 | Condition recognition device, aircraft cabin and method for monitoring an aircraft cabin |
| CN110046560A (en) * | 2019-03-28 | 2019-07-23 | 青岛小鸟看看科技有限公司 | A kind of dangerous driving behavior detection method and camera |
| CN111753585A (en) * | 2019-03-28 | 2020-10-09 | 北京市商汤科技开发有限公司 | Motion tracking processing method and device, medium, and apparatus |
| CN110059650A (en) * | 2019-04-24 | 2019-07-26 | 京东方科技集团股份有限公司 | Information processing method, device, computer storage medium and electronic equipment |
| CN110766131A (en) * | 2019-05-14 | 2020-02-07 | 北京嘀嘀无限科技发展有限公司 | Data processing device and method and electronic equipment |
| CN110222596A (en) * | 2019-05-20 | 2019-09-10 | 浙江零跑科技有限公司 | A kind of driving behavior analysis anti-cheating method of view-based access control model |
| CN110334627A (en) * | 2019-06-26 | 2019-10-15 | 深圳市微纳集成电路与系统应用研究院 | The device and system that the behavior of personnel is detected |
| CN110309760A (en) * | 2019-06-26 | 2019-10-08 | 深圳市微纳集成电路与系统应用研究院 | The method that the driving behavior of driver is detected |
| CN110807352A (en) * | 2019-08-29 | 2020-02-18 | 浙江零跑科技有限公司 | In-vehicle and out-vehicle scene visual analysis method for dangerous driving behavior early warning |
| CN110807352B (en) * | 2019-08-29 | 2023-08-25 | 浙江零跑科技股份有限公司 | In-vehicle scene visual analysis method for dangerous driving behavior early warning |
| CN111090723A (en) * | 2019-11-18 | 2020-05-01 | 清远电力规划设计院有限公司 | Power grid safety production content recommendation method based on knowledge graph |
| CN111090723B (en) * | 2019-11-18 | 2023-05-05 | 清远电力规划设计院有限公司 | Knowledge graph-based recommendation method for safe production content of power grid |
| CN111222449A (en) * | 2020-01-02 | 2020-06-02 | 上海中安电子信息科技有限公司 | Driver behavior detection method based on fixed camera image |
| CN111274881A (en) * | 2020-01-10 | 2020-06-12 | 中国平安财产保险股份有限公司 | Driving safety monitoring method and device, computer equipment and storage medium |
| CN111274881B (en) * | 2020-01-10 | 2024-05-31 | 中国平安财产保险股份有限公司 | Driving safety monitoring method and device, computer equipment and storage medium |
| CN111242015A (en) * | 2020-01-10 | 2020-06-05 | 同济大学 | Method for predicting driving danger scene based on motion contour semantic graph |
| CN111242015B (en) * | 2020-01-10 | 2023-05-02 | 同济大学 | Method for predicting driving dangerous scene based on motion profile semantic graph |
| CN111243235A (en) * | 2020-01-13 | 2020-06-05 | 惠龙易通国际物流股份有限公司 | Driving assistance method and device |
| WO2021159269A1 (en) * | 2020-02-11 | 2021-08-19 | 云图技术有限公司 | Vision-based safe driving early warning method and system, and storage medium |
| CN112017155A (en) * | 2020-07-13 | 2020-12-01 | 浙江大华汽车技术有限公司 | Method, device and system for measuring health sign data and storage medium |
| CN112017155B (en) * | 2020-07-13 | 2023-12-26 | 浙江华锐捷技术有限公司 | Method, device, system and storage medium for measuring health sign data |
| CN111860316A (en) * | 2020-07-20 | 2020-10-30 | 上海汽车集团股份有限公司 | Driving behavior recognition method and device and storage medium |
| CN111860316B (en) * | 2020-07-20 | 2024-03-19 | 上海汽车集团股份有限公司 | Driving behavior recognition method, device and storage medium |
| CN112016457A (en) * | 2020-08-27 | 2020-12-01 | 青岛慕容信息科技有限公司 | Driver distraction and dangerous driving behavior recognition method, device and storage medium |
| CN112434612A (en) * | 2020-11-25 | 2021-03-02 | 创新奇智(上海)科技有限公司 | Smoking detection method and device, electronic equipment and computer readable storage medium |
| CN112861608A (en) * | 2020-12-30 | 2021-05-28 | 浙江万里学院 | Detection method and system for distracted driving behaviors |
| CN116912808A (en) * | 2023-09-14 | 2023-10-20 | 四川公路桥梁建设集团有限公司 | Bridge girder erection machine control method, electronic equipment and computer readable medium |
| CN116912808B (en) * | 2023-09-14 | 2023-12-01 | 四川公路桥梁建设集团有限公司 | Bridge girder erection machine control method, electronic equipment and computer readable medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108764034A (en) | A kind of driving behavior method for early warning of diverting attention based on driver's cabin near infrared camera | |
| CN108791299B (en) | Driving fatigue detection and early warning system and method based on vision | |
| CN112686090B (en) | Intelligent monitoring system for abnormal behavior in bus | |
| CN110119676A (en) | A kind of Driver Fatigue Detection neural network based | |
| CN112633057B (en) | Intelligent monitoring method for abnormal behavior in bus | |
| CN111353471A (en) | Safe driving monitoring method, device, equipment and readable storage medium | |
| CN114026611A (en) | Detecting driver attentiveness using heatmaps | |
| CN110532925B (en) | Driver fatigue detection method based on space-time graph convolutional network | |
| CN105769120A (en) | Fatigue driving detection method and device | |
| CN106898346A (en) | A kind of freeway tunnel safety monitoring system | |
| CN110222596B (en) | Driver behavior analysis anti-cheating method based on vision | |
| CN109740477A (en) | Study in Driver Fatigue State Surveillance System and its fatigue detection method | |
| CN110780356A (en) | Subway platform clearance foreign matter detecting system | |
| KR20100121020A (en) | Apparatus for detecting a obstacle in a road crossing and controlling method thereof using a moving track of an object | |
| CN115761537B (en) | Power transmission line foreign matter intrusion identification method oriented to dynamic feature supplementing mechanism | |
| CN113591659B (en) | Gesture control intention recognition method and system based on multi-mode input | |
| Thiruppathiraj et al. | Automatic pothole classification and segmentation using android smartphone sensors and camera images with machine learning techniques | |
| CN112052829B (en) | Pilot behavior monitoring method based on deep learning | |
| CN114299473A (en) | Driver behavior identification method based on multi-source information fusion | |
| CN106611165A (en) | Automobile window detection method and device based on correlation filtering and color matching | |
| CN111175852B (en) | Airport fog forecast early warning method based on long-time memory algorithm | |
| CN106128105A (en) | A kind of traffic intersection pedestrian behavior monitoring system | |
| CN116563801A (en) | Traffic accident detection method, device, electronic equipment and medium | |
| CN109927730B (en) | Real-time fatigue driving behavior scoring system and method based on DMS system | |
| CN114387310A (en) | Urban trunk road traffic flow statistical method based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20181106 |
|
| RJ01 | Rejection of invention patent application after publication |