CN109953763A - A kind of vehicle carried driving behavioral value early warning system and method based on deep learning - Google Patents
A kind of vehicle carried driving behavioral value early warning system and method based on deep learning Download PDFInfo
- Publication number
- CN109953763A CN109953763A CN201910151313.XA CN201910151313A CN109953763A CN 109953763 A CN109953763 A CN 109953763A CN 201910151313 A CN201910151313 A CN 201910151313A CN 109953763 A CN109953763 A CN 109953763A
- Authority
- CN
- China
- Prior art keywords
- module
- measured
- behavior
- early warning
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/024—Detecting, measuring or recording pulse rate or heart rate
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/168—Evaluating attention deficit, hyperactivity
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
- A61B5/18—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
- A61B5/316—Modalities, i.e. specific diagnostic methods
- A61B5/369—Electroencephalography [EEG]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/7405—Details of notification to user or communication with user or patient ; user input means using sound
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/7455—Details of notification to user or communication with user or patient ; user input means characterised by tactile indication, e.g. vibration or electrical stimulation
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/746—Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
Abstract
The invention discloses a kind of vehicle carried driving behavioral value early warning system and method based on deep learning, system include information acquisition module, transmission module, control processing module, warning module, and method includes: to acquire signal source information by information acquisition module first；Collected signal source information is sent to control processing module by transmission module later；Then control processing module judges the fatigue state and behavior of object to be measured according to signal source information；The fatigue state of acquisition and behavior outcome are finally transmitted to warning module and cloud, warning module carries out corresponding acousto-optic, vibration early warning according to fatigue state and behavior outcome.Image recognition technology, bio-signal acquisition algorithm and the driving behavior detection algorithm of present invention combination deep learning, carry out multisignal source acquisition, and acquisition accuracy is higher；And the data anastomosing algorithm based on machine learning is utilized, the judgement of object fatigue to be measured and behavior is carried out, the judgement result of acquisition is more accurate, and early warning is more efficient reliable.
Description
Technical field
The invention belongs to driving behavior detection field, especially a kind of vehicle carried driving behavioral value based on deep learning is pre-
Alert system and method.
Background technique
With China's economic society sustained and rapid development, China's motor vehicle total data is sharply increased.By the end of the end of the year 2018,
China's vehicle guaranteeding organic quantity is up to 3.22 hundred million.The sharp increase of vehicle, mileage in highway open to traffic increase year by year, also along with unexpected traffic thing
Therefore take place frequently.Accelerate simultaneously with modern life rhythm, phenomenon of staying up late is serious, do not get enough athletic exercise, irregular etc. the factor of diet
It is poor to cause the easy tired energy of modern, the fatigue driving occurred therewith.According to statistics, in the factor for influencing safe driving,
Driving tired, absent minded and lack of standardization etc. is the main reason for causing traffic accident to occur.Currently, driving behavior is examined
Surveying main method is detected for some specific index of physiological driver's psychology in driving process, and machine vision is utilized
Technology or the external change feature and behavior of other sensors technology detection driver, such as: eyelid is blinked, nods, yawns, is taken out
Cigarette sees the mobile phone etc..But the method determines that speed and accuracy are still to be improved.Therefore how effectively to be driven by vehicle-mounted auxiliary
The system of sailing come traffic accident brought by preventing fatigue driving, absent minded and driving lack of standardization have become it is worldwide
Problem, there are many scholars to be studied using different technologies both at home and abroad.
Traditional driving behavior detection algorithm based on physiological signal and behavioural characteristic has its advantage and defect.It is based on
The detection algorithm of physiological signal feature precision with higher but since sensor is contacted with driver, influences to drive；Based on row
The detection algorithm being characterized does not need the direct contact detecting apparatus of driver, and to equipment on the basis of automobile existing apparatus
Demand is lower, and practicability is very strong, but detection accuracy is not high.
Summary of the invention
Technical problem solved by the invention is to provide a kind of according to object heart rate to be measured, EEG signals and behavior view
Frequently, Comprehensive Evaluation object fatigue state to be measured and the system and method for driving behavior lack of standardization.
The technical solution for realizing the aim of the invention is as follows: a kind of vehicle carried driving behavioral value early warning based on deep learning
System, including sequentially connected information acquisition module, transmission module, control processing module, warning module；
The information acquisition module, for acquiring signal source information, the signal source include object to be measured EEG signals,
Heart rate and behavior video；
The transmission module, for the collected signal source information of information acquisition module to be transmitted to control processing module；
The control processing module for judging fatigue state, the behavior of object to be measured according to signal source information, and will be sentenced
Disconnected result is transmitted to warning module and cloud；
The warning module, for carrying out acousto-optic, vibration early warning according to the judging result of control processing module.
Based on the detection method for early warning of the above-mentioned vehicle carried driving behavioral value early warning system based on deep learning, including it is following
Step:
Step 1 acquires signal source information by information acquisition module, including being absorbed in for the EEG signals acquisition by object to be measured
Degree, heart rate and behavior video；
The collected signal source information of step 1 is sent to control processing module by transmission module by step 2；
Step 3, control processing module judge the fatigue state and behavior of object to be measured according to signal source information；
Fatigue state and behavior outcome that step 3 obtains are transmitted to warning module and cloud, warning module root by step 4
Corresponding acousto-optic, vibration early warning are carried out according to fatigue state and behavior outcome.
Compared with prior art, the present invention its remarkable advantage are as follows: 1) present invention is in the image recognition skill based on deep learning
In art, in conjunction with bio-signal acquisition algorithm and driving behavior detection algorithm, the acquisition of multisignal source is carried out, is maximized favourable factors and minimized unfavourable ones, sufficiently
Respective advantage is played, the signal source of selection is more fully reliable, and it is more objective that signal source acquires accuracy；2) present invention utilizes
Data anastomosing algorithm based on machine learning carries out the judgement of fatigue and every object behavior to be measured, so that the result determined is more
To be accurate, so that early warning is more efficient reliable；3) present invention can be used for the fatigue state and driving behavior of all kinds of objects to be measured
Detection and the early warning of vehicle drive safety；4) the corresponding judgement result of onboard system acquisition data is uploaded to cloud by the present invention
Database can check the historical behavior of object to be measured, be responsible for convenient for traffic police department or logistics, passenger company it
Numerous drivers be monitored management.
Present invention is further described in detail with reference to the accompanying drawing.
Detailed description of the invention
Fig. 1 is that the present invention is based on the vehicle carried driving behavioral value early warning systems of deep learning to constitute schematic diagram.
Fig. 2 is eeg signal acquisition, processing and early warning flow diagram in the present invention.
Fig. 3 is heart rate acquisition, processing and early warning flow diagram in the present invention.
Fig. 4 is face picture acquisition, processing and early warning flow diagram in the present invention.
Fig. 5 is data anastomosing algorithm implementation flow chart in the present invention.
Specific embodiment
In conjunction with Fig. 1, a kind of vehicle carried driving behavioral value early warning system based on deep learning of the present invention, including it is sequentially connected
Information acquisition module, transmission module, control processing module, warning module；
Information acquisition module, for acquiring signal source information, the signal source includes the EEG signals of object to be measured, heart rate
And behavior video；
Transmission module, for the collected signal source information of information acquisition module to be transmitted to control processing module；
Processing module is controlled, is tied for judging fatigue state, the behavior of object to be measured according to signal source information, and by judgement
Fruit is transmitted to warning module and cloud；
Warning module, for carrying out acousto-optic, vibration early warning according to the judging result of control processing module.
Further, information acquisition module includes brain wave acquisition module, heart rate acquisition module and image capture module；
Brain wave acquisition module, for acquiring the EEG signals of object to be measured and then obtaining its focus；
Heart rate acquisition module, for acquiring the heart rate of object to be measured；
Image capture module, for acquiring the behavior video of object to be measured.
It is further preferred that brain wave acquisition module is wear-type device, comprising:
Forehead EEG pole, ear-clip electrodes, for acquiring the EEG signals of object to be measured and being transmitted to TGAM chip, the brain
Electric signal includes α wave, β wave, θ wave；
TGAM chip, for receiving EEG signals and carrying out preliminary treatment to it, and according to treated EEG signals meter
Calculate the focus of object to be measured；Wherein, the preliminary treatment includes filtering, amplification, A/D conversion.
It is further preferred that heart rate acquisition module is bracelet body-worn device, comprising:
Optics heart rate sensor is transmitted to for measuring the transmittance data in object blood to be measured, and by the data
DA14580 chip；
DA14580 chip is carried out for pre-processing to transmittance data, and by the ADC module of the built-in chip type
Analog-to-digital conversion seeks the heart rate of object to be measured later；The pretreatment includes filtering, except making an uproar.
It is further preferred that image capture module includes:
Camera, for acquiring the behavior video of object to be measured.
Further, transmission module includes:
Bluetooth HC-06 slave module, the focus of the object to be measured for receiving the acquisition of TGAM chip processing, and passed
Transport to the bluetooth HC-06 host module to match with bluetooth HC-06 slave module；
Bluetooth HC-06 host module, for the focus of the object to be measured of bluetooth HC-06 slave module transfer to be transmitted to
Control processing module；
The bluetooth module of DA14580 built-in chip type, for heart rate to be transmitted to control processing module；
Usb communication module, for by the behavior transmission of video of the collected object to be measured of camera to controlling processing module.
Further, control processing module includes:
The vehicle-mounted chip of raspberry pie 3B+, for judged according to received signal source information object to be measured fatigue state,
Behavior, and judging result is transmitted to warning module and cloud.
Further, warning module includes:
Display module, for showing the visualization alert interface of object fatigue state to be measured, behavior；
Vibration module, for realizing driver's seat vibration；
Voice module, for playing the audio signal of prerecording.
It is further preferred that vibration module includes L298N pressure stabilizing driving circuit, the vibrating motor being sequentially connected, Neng Goushi
The vibration of existing different frequency.
Based on the detection method for early warning of the above-mentioned vehicle carried driving behavioral value early warning system based on deep learning, including it is following
Step:
Step 1 acquires signal source information by information acquisition module, including being absorbed in for the EEG signals acquisition by object to be measured
Degree, heart rate and behavior video；
The collected signal source information of step 1 is sent to control processing module by transmission module by step 2；
Step 3, control processing module judge the fatigue state and behavior of object to be measured according to signal source information；
Fatigue state and behavior outcome that step 3 obtains are transmitted to warning module and cloud, warning module root by step 4
Corresponding acousto-optic, vibration early warning are carried out according to fatigue state and behavior outcome.
Further, in conjunction with Fig. 2, Fig. 3, Fig. 4, control processing module described in step 3 judges to be measured according to signal source information
The behavior of object, specifically:
Step 3-1, several images are intercepted from the behavior video of collected object to be measured with interval t, wherein the unit of t
For ms；
Step 3-2, the object in each image in the trained object detection model API detecting step 3-1 of EasyDL is utilized
Body；
If jobbie belongs to the object for being preset as dangerous driving factor in object detection model, determined according to the object
The behavior of object to be measured determines the behavior of object to be measured for dangerous driving if the corresponding image number of the object is greater than threshold value p
Otherwise behavior is determined as normal driving behavior；
Otherwise execute step 3-3；
Step 3-3, the facial image for each image that extraction step 3-1 is obtained；
Step 3-4, the classification of each image in the trained image classification model API detecting step 3-3 of EasyDL is utilized；
If a certain image category belongs to the image category for being preset as dangerous driving factor in image classification model, basis should
The number of classification image determines the behavior of object to be measured, if the number of category image is greater than threshold value p, determines object to be measured
Behavior is dangerous driving behavior, is otherwise determined as normal driving behavior.
Further, in conjunction with Fig. 2, Fig. 3, Fig. 4, control processing module described in step 3 judges to be measured according to signal source information
The fatigue state of object, specifically:
The eye image for every width facial image that step 3-1 ', extraction step 3-3 are obtained；
Step 3-2 ', all eye images of step 3-1 ' are pre-processed；The pretreatment includes successively executing
Median filtering, laplacian spectral radius, image gray processing, binaryzation；
Step 3-3 ', according to perclos algorithm, seek human eye perclos index value f, specifically:
Step 3-3 ' -1, according to all people eye image, record four time intervals: opened completely from human eye to closure
p1The time interval t1 of %, it is opened completely from human eye to closure p2The time interval t2 of %, it is opened completely from human eye to opening next time
Open p1The interval time t3 of %, it is opened completely from human eye to opening p next time2The time interval t4 of %；Wherein, p1%, p2% is equal
The ratio of entire eyes is accounted for for eyeball；
Step 3-3 ' -2, human eye perclos index value f, formula used are sought according to t1, t2, t3, t4 are as follows:
Step 3-4 ', according to the focus r of object to be measured, heart rate h, human eye perclos index f, using being based on depth
The data anastomosing algorithm of habit is as shown in figure 5, seek the fatigue state degree value h of object to be measuredθ(x), specifically:
Data when step 3-4 ' -1, acquisition object to be measured are awake and tired when carrying out driver training, and Separating hyper surface
According to training set and test set, training set is trained using logistic regression analysis model, and true using stochastic gradient descent method
Optimized parameter θ in the fixed model；
Step 3-4 ' -2, according to the focus r of object to be measured, heart rate h, human eye perclos index f and optimized parameter θ, ask
Take the fatigue state value h of object to be measuredθ(x):
In formula, x is the matrix of focus r, heart rate h, human eye perclos index the f composition of object to be measured, x=[r h
f]；
Wherein, the calculation formula of focus r are as follows:
R=(α+θ)/β
In formula, α is the EEG signals α wave power of acquisition, and β is EEG signals β wave power, and θ is EEG signals θ wave power；
The calculation formula of heart rate h are as follows:
H=6000/period
In formula, time interval of the period between heartbeat twice；
Step 3-5 ', according to fatigue state degree value hθ(x) size determines the fatigue state of object to be measured；Specifically:
If 0 < hθ(x) < q1, then the fatigue state of object to be measured is severe fatigue；
If q1≤hθ(x) < q2, then the fatigue state of object to be measured is slight fatigue；
If q2≤hθ(x) < q3, then the fatigue state of object to be measured is awake；
Wherein, 0≤q1,q2,q3≤1。
Preferably, p in step 3-3 ' -11=20, p2Q in=80, step 3-5 '1=0.3, q2=0.6, q3=1.
Further, warning module described in step 4 carries out corresponding according to the fatigue state and behavior outcome of object to be measured
Acousto-optic, vibration early warning, specifically:
If the behavior of object to be measured is judged as dangerous driving behavior or fatigue state to be judged as severe tired, slight tired
Labor, then display module flashing display current dangerous driving behavior or fatigue state result, vibration module work so that driver's seat production
Raw vibration, voice module play the audio signal of prerecording simultaneously, realize forewarning function jointly；
Wherein, tired for severe, slight tired two kinds of fatigue states, the frequency of vibration module work is high when severe fatigue
In the corresponding frequency of slight fatigue.
Invention is described in further detail below with reference to embodiment.
Embodiment
In conjunction with Fig. 1, a kind of vehicle carried driving behavioral value early warning system based on deep learning of the invention, including it is following interior
Hold:
What image capture module was selected in the embodiment of the present invention is camera, acquires the behavior video of driver, and pass through
Usb communication module in transmission module, by the collected driving behavior transmission of video of camera to controlling processing module.Because
The object to be measured behaviors such as make a phone call, smoke, one hand drive, yawn, bow are easy to go to perceive from vision level, and a period of time
The closing time of interior human eye accounts for the ratio of total time, can be used to determine fatigue, can be used as control processing module and carries out fatigue synthesis
The information source of judgement.
Heart rate acquisition module, selects optics heart rate sensor and DA14580 chip, and optics heart rate sensor measures driver
Transmittance data in blood, and the data are transmitted to DA14580 chip, DA14580 chip filters transmittance data
Wave carries out analog-to-digital conversion except making an uproar, and by the ADC module of the built-in chip type, the heart rate of driver is sought later, finally by biography
In defeated module, heart rate is transmitted to control processing module by the bluetooth module of DA14580 built-in chip type.When object to be measured is in tired
When labor state, heart rate can be significantly risen, and can be used as the information source that control processing module carries out tired comprehensive judgement.
Brain wave acquisition module selects the wear-type device being made of forehead EEG pole, ear-clip electrodes, TGAM chip, forehead
Brain electrode, ear-clip electrodes acquire the α wave of driver, β wave, θ wave and are transmitted to TGAM chip, and TGAM chip is filtered, puts
Greatly, A/D conversion, calculate the focus of driver, finally by transmission module, bluetooth HC-06 slave module, bluetooth HC-06
Focus is transmitted to control processing module by host module.When driver is in a state of fatigue, human brain liveness is reduced, thus
β wave is caused to reduce, but α wave increases, when fatigue state is changed into sleep state, the main frequency of brain electricity can be down to θ wave, therefore
It can be used as the information source that control processing module carries out tired comprehensive judgement by α wave, β wave, the calculated focus of θ wave.
Processing module is controlled, raspberry pie 3B+ chip is selected, which has the basic function of all PC, and rich interface is full
The functional requirement of sufficient this system, for handling behavior video, heart rate, focus data, judgement driver fatigue state and behavior,
And early warning is carried out by distribution pin level height control warning module, while by data and determining that result is uploaded to cloud.
In warning module, display module selects LCD touching screen, for show object fatigue state to be measured, behavior it is visual
Change alert interface；Vibration module selects L298N pressure stabilizing driving circuit, the vibrating motor being sequentially connected, and can be realized driver's seat not
The vibration of same frequency；Voice module selects SYN6288 voice playing module, for playing the audio signal of prerecording.
The present invention is based on the detection methods of the vehicle carried driving behavioral value early warning system of deep learning, including the following contents:
1, focus, heart rate and behavior video that the EEG signals of driver obtain are acquired, in which:
Brain electricity focus calculation formula are as follows:
R=(α+θ)/β
In formula, α is the EEG signals α wave power of acquisition, and β is EEG signals β wave power, and θ is EEG signals θ wave power；
It is handled by normalization data, R ∈ (0,100).
The calculation formula of heart rate h are as follows:
H=6000/period
In formula, time interval of the period between heartbeat twice；H ∈ (40,120) beat/min.
2, the collected signal source information of step 1 is sent to control processing module by transmission module.
3, control processing module judges the fatigue state and behavior of object to be measured according to signal source information, specifically:
Step 3-1, every 100ms is to interception image in the behavior video of collected object to be measured；
Step 3-2, the object in each image in the trained object detection model API detecting step 3-1 of EasyDL is utilized
Body；If jobbie belongs to the object for being preset as dangerous driving factor in object detection model, determined according to the object to be measured
The behavior of object determines the behavior of object to be measured for dangerous driving row if the corresponding image number of the object is greater than threshold value 30
To be otherwise determined as normal driving behavior；The object of dangerous driving factor includes cigarette, mobile phone etc.；
Step 3-3, the facial image for each image that extraction step 3-1 is obtained；
Step 3-4, the classification of each image in the trained image classification model API detecting step 3-3 of EasyDL is utilized；
If a certain image category belongs to the image category for being preset as dangerous driving factor in image classification model, according to category image
Number determine the behavior of object to be measured, if the number of category image is greater than threshold value 30, determine that the behavior of object to be measured is
Otherwise dangerous driving behavior is determined as normal driving behavior.
Step 3-5, extraction step 3-3 obtain every width facial image eye image and carry out image preprocessing, comprising:
Median filtering, laplacian spectral radius, image gray processing, binaryzation.
Step 3-6, human eye perclos index value f, formula used are sought according to t1, t2, t3, t4 are as follows:
In formula,
T1: it is opened completely from human eye to the time interval of closure 20%；
T2: it is opened completely from human eye to the time interval of closure 80%；
T3: it is opened completely from human eye to the time interval for opening 20% next time；
T4: it is opened completely from human eye to the time interval for opening 80% next time；
Wherein, 20%, 80% is ratio that eyeball accounts for entire eyes；
Step 3-7, it according to the focus r of object to be measured, heart rate h, human eye perclos index f and optimized parameter θ, seeks
The fatigue state value h of object to be measuredθ(x):
In formula, x is the matrix of focus r, heart rate h, human eye perclos index the f composition of object to be measured, x=[r h
f]；θ is optimized parameter of the model after training, hθ(x) (0,1) ∈
Step 3-8, according to fatigue state degree value hθ(x) size determines the fatigue state of object to be measured；Specifically:
If 0 < hθ(x) < q1, then the fatigue state of object to be measured is severe fatigue；
If q1≤hθ(x) < q2, then the fatigue state of object to be measured is slight fatigue；
If q2≤hθ(x) < q3, then the fatigue state of object to be measured is awake；
Wherein, 0≤q1,q2,q3≤ 1, q in the present embodiment1=0.3, q2=0.6, q3=1.
The present embodiment describes the deterministic process of fatigue state with the data instance of degree of fatigue known to three kinds, such as the following table 1 institute
Show:
The data of 1 three kinds of degree of fatigues of table
| R (brain electricity focus) | H (heart rate) | F (human eye perclos index) | |
| It is awake | 90 | 68 | 0.12 |
| Slight fatigue | 41 | 66 | 0.375 |
| Severe fatigue | 21 | 66 | 0.416667 |
In the present embodiment, data when object to be measured regains consciousness when carrying out driver training and is tired, and Separating hyper surface are acquired
According to training set and test set, training set is trained using logistic regression analysis model, and true using stochastic gradient descent method
Optimized parameter θ in cover half type is [0.0408,0.0741, -5.3655, -5.6617].
(1) r=90, h=68, f=0.12 are taken, then after x is multiplied with θ, as a result are as follows:
Y=0.0408*90+0.0741*68+ (- 5.3655) * 0.12-5.6617=2.40524
hθ(x)=1/ (1+e-y), it can be calculated, hθ(x)=0.9172, in q2≤hθ(x) < q3Section, therefore be awake；
(2) r=41, h=66, f=0.375 are taken, then after x is multiplied with θ, as a result are as follows:
Y=0.0408*41+0.0741*66+ (- 5.3655) * 0.375-5.6617=-0.36036
hθ(x)=1/ (1+e-y), it can be calculated, hθ(x)=0.4108, in q1≤hθ(x) < q2Section, therefore be slight tired
Labor；
(3) r=21, h=66, f=0.416667 are taken, then after x is multiplied with θ, as a result are as follows:
Y=0.0408*21+0.0741*66+ (- 5.3655) * 0.416667-5.6617=-2.14992
hθ(x)=1/ (1+e-y), it can be calculated, hθ(x)=0.10433, in 0 < hθ(x) < q1Section, thus it is tired for severe
Labor.
4, fatigue state and behavior outcome that above-mentioned 3 obtain are transmitted to warning module and cloud, warning module is according to tired
Labor state and behavior outcome carry out corresponding acousto-optic, vibration early warning.
If the behavior of object to be measured is judged as dangerous driving behavior or fatigue state to be judged as severe tired, slight tired
Labor, then display module flashing display current dangerous driving behavior or fatigue state result, vibration module work so that driver's seat production
Raw vibration, voice module play the audio signal of prerecording simultaneously, realize forewarning function jointly；
Wherein, tired for severe, slight tired two kinds of fatigue states, the frequency of vibration module work is high when severe fatigue
In the corresponding frequency of slight fatigue.
To sum up, system and method for the invention are combined based on physiological parameter fatigue detecting method, based on computer vision
Driving behavior detection method, and the data anastomosing algorithm based on deep learning, carried out object to be measured behavior and
The judgement of fatigue state, on the one hand improves the accuracy of fatigue judgement, and the portability of another aspect system equipment allows driver
More easily receive.Driver's EEG signals, heart rate and behavior video are acquired by information acquisition module；It will by transmission module
These three signal sources are transmitted to control processing module, carry out fatigue behaviour judgement, on the one hand control warning module and carry out early warning, separately
On the one hand data are uploaded to cloud, the numerous drivers being responsible for convenient for traffic police department or logistics, passenger company it carry out
Monitoring management.
Claims (10)
1. a kind of vehicle carried driving behavioral value early warning system based on deep learning, which is characterized in that including sequentially connected letter
Cease acquisition module, transmission module, control processing module, warning module；
The information acquisition module, for acquiring signal source information, the signal source includes the EEG signals of object to be measured, heart rate
And behavior video；
The transmission module, for the collected signal source information of information acquisition module to be transmitted to control processing module；
The control processing module is tied for judging fatigue state, the behavior of object to be measured according to signal source information, and by judgement
Fruit is transmitted to warning module and cloud；
The warning module, for carrying out acousto-optic, vibration early warning according to the judging result of control processing module.
2. the vehicle carried driving behavioral value early warning system according to claim 1 based on deep learning, which is characterized in that institute
Stating information acquisition module includes brain wave acquisition module, heart rate acquisition module and image capture module；
Brain wave acquisition module, for acquiring the EEG signals of object to be measured and then obtaining its focus；
Heart rate acquisition module, for acquiring the heart rate of object to be measured；
Image capture module, for acquiring the behavior video of object to be measured.
3. the vehicle carried driving behavioral value early warning system according to claim 2 based on deep learning, which is characterized in that institute
Stating brain wave acquisition module is wear-type device, comprising:
Forehead EEG pole, ear-clip electrodes, for acquiring the EEG signals of object to be measured and being transmitted to TGAM chip, the brain telecommunications
Number include α wave, β wave, θ wave；
TGAM chip, for receiving EEG signals and carrying out preliminary treatment to it, and according to treated EEG signals calculate to
Survey the focus of object；Wherein, the preliminary treatment includes filtering, amplification, A/D conversion；
The heart rate acquisition module is bracelet body-worn device, comprising:
Optics heart rate sensor is transmitted to DA14580 for measuring the transmittance data in object blood to be measured, and by the data
Chip；
DA14580 chip carries out modulus for pre-processing to transmittance data, and by the ADC module of the built-in chip type
Conversion, seeks the heart rate of object to be measured later；The pretreatment includes filtering, except making an uproar；
Described image acquisition module includes:
Camera, for acquiring the behavior video of object to be measured.
4. the vehicle carried driving behavioral value early warning system according to claim 3 based on deep learning, which is characterized in that institute
Stating transmission module includes:
Bluetooth HC-06 slave module, the focus of the object to be measured for receiving the acquisition of TGAM chip processing, and transmit it to
The bluetooth HC-06 host module to match with bluetooth HC-06 slave module；
Bluetooth HC-06 host module, for the focus of the object to be measured of bluetooth HC-06 slave module transfer to be transmitted to control
Processing module；
The bluetooth module of DA14580 built-in chip type, for heart rate to be transmitted to control processing module；
Usb communication module, for by the behavior transmission of video of the collected object to be measured of camera to controlling processing module；
The control processing module includes:
The vehicle-mounted chip of raspberry pie 3B+, for judging fatigue state, the row of object to be measured according to received signal source information
For, and judging result is transmitted to warning module and cloud；
The warning module includes:
Display module, for showing the visualization alert interface of object fatigue state to be measured, behavior；
Vibration module, for realizing driver's seat vibration；
Voice module, for playing the audio signal of prerecording.
5. the vehicle carried driving behavioral value early warning system according to claim 4 based on deep learning, which is characterized in that institute
Stating vibration module includes L298N pressure stabilizing driving circuit, the vibrating motor being sequentially connected, and can be realized the vibration of different frequency.
6. the pre- police of detection based on the vehicle carried driving behavioral value early warning system described in claim 1 based on deep learning
Method, which comprises the following steps:
Step 1 acquires signal source information by information acquisition module, including by object to be measured EEG signals acquisition focus,
Heart rate and behavior video；
The collected signal source information of step 1 is sent to control processing module by transmission module by step 2；
Step 3, control processing module judge the fatigue state and behavior of object to be measured according to signal source information；
Fatigue state and behavior outcome that step 3 obtains are transmitted to warning module and cloud by step 4, and warning module is according to tired
Labor state and behavior outcome carry out corresponding acousto-optic, vibration early warning.
7. the vehicle carried driving behavioral value method for early warning according to claim 6 based on deep learning, which is characterized in that step
The rapid 3 control processing module judges the behavior of object to be measured according to signal source information, specifically:
Step 3-1, several images are intercepted from the behavior video of collected object to be measured with interval t, wherein the unit of t is
ms；
Step 3-2, the object in each image in the trained object detection model API detecting step 3-1 of EasyDL is utilized；
If jobbie belongs to the object for being preset as dangerous driving factor in object detection model, determined according to the object to be measured
The behavior of object determines the behavior of object to be measured for dangerous driving row if the corresponding image number of the object is greater than threshold value p
To be otherwise determined as normal driving behavior；
Otherwise execute step 3-3；
Step 3-3, the facial image for each image that extraction step 3-1 is obtained；
Step 3-4, the classification of each image in the trained image classification model API detecting step 3-3 of EasyDL is utilized；
If a certain image category belongs to the image category for being preset as dangerous driving factor in image classification model, according to the category
The number of image determines the behavior of object to be measured, if the number of category image is greater than threshold value p, determines the behavior of object to be measured
For dangerous driving behavior, otherwise it is determined as normal driving behavior.
8. the vehicle carried driving behavioral value method for early warning according to claim 7 based on deep learning, which is characterized in that step
The rapid 3 control processing module judges the fatigue state of object to be measured according to signal source information, specifically:
The eye image for every width facial image that step 3-1 ', extraction step 3-3 are obtained；
Step 3-2 ', all eye images of step 3-1 ' are pre-processed；The pretreatment includes the intermediate value successively executed
Filtering, laplacian spectral radius, image gray processing, binaryzation；
Step 3-3 ', according to perclos algorithm, seek human eye perclos index value f, specifically:
Step 3-3 ' -1, according to all people eye image, record four time intervals: opened completely from human eye to closure p1%'s
Time interval t1, it is opened completely from human eye to closure p2The time interval t2 of %, it is opened completely from human eye to opening p next time1%
Interval time t3, open completely from human eye to opening p next time2The time interval t4 of %；Wherein, p1%, p2% is eyeball
Account for the ratio of entire eyes；
Step 3-3 ' -2, human eye perclos index value f, formula used are sought according to t1, t2, t3, t4 are as follows:
Step 3-4 ', according to the focus r of object to be measured, heart rate h, human eye perclos index f, using based on deep learning
Data anastomosing algorithm seeks the fatigue state degree value h of object to be measuredθ(x), specifically:
Data when step 3-4 ' -1, acquisition object to be measured are awake and tired when carrying out driver training, and Separating hyper surface is according to instruction
Practice collection and test set, training set is trained using logistic regression analysis model, and institute is determined using stochastic gradient descent method
State the optimized parameter θ in model；
Step 3-4 ' -2, according to the focus r of object to be measured, heart rate h, human eye perclos index f and optimized parameter θ, seek to
Survey the fatigue state value h of objectθ(x):
In formula, x is the matrix of focus r, heart rate h, human eye perclos index the f composition of object to be measured, x=[r h f]；
Wherein, the calculation formula of focus r are as follows:
R=(α+θ)/β
In formula, α is the EEG signals α wave power of acquisition, and β is EEG signals β wave power, and θ is EEG signals θ wave power；
The calculation formula of heart rate h are as follows:
H=6000/period
In formula, time interval of the period between heartbeat twice；
Step 3-5 ', according to fatigue state degree value hθ(x) size determines the fatigue state of object to be measured；Specifically:
If 0 < hθ(x) < q1, then the fatigue state of object to be measured is severe fatigue；
If q1≤hθ(x) < q2, then the fatigue state of object to be measured is slight fatigue；
If q2≤hθ(x) < q3, then the fatigue state of object to be measured is awake；
Wherein, 0≤q1,q2,q3≤1。
9. the vehicle carried driving behavioral value method for early warning according to claim 8 based on deep learning, which is characterized in that step
Rapid 3-3 ' -1 p1=20, p2=80, step 3-5 ' q1=0.3, q2=0.6, q3=1.
10. the vehicle carried driving behavioral value method for early warning according to claim 8 based on deep learning, which is characterized in that
Warning module described in step 4 carries out corresponding acousto-optic, vibration early warning according to the fatigue state and behavior outcome of object to be measured, specifically
Are as follows:
If the behavior of object to be measured is judged as dangerous driving behavior or fatigue state is judged as that severe is tired, slight fatigue,
Then display module flashing display current dangerous driving behavior or fatigue state result, vibration module work so that driver's seat generation vibration
Dynamic, voice module plays the audio signal of prerecording simultaneously, realizes forewarning function jointly；
Wherein, tired for severe, slight tired two kinds of fatigue states, the frequency of vibration module work is higher than light when severe fatigue
The corresponding frequency of degree fatigue.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910151313.XA CN109953763A (en) | 2019-02-28 | 2019-02-28 | A kind of vehicle carried driving behavioral value early warning system and method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910151313.XA CN109953763A (en) | 2019-02-28 | 2019-02-28 | A kind of vehicle carried driving behavioral value early warning system and method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109953763A true CN109953763A (en) | 2019-07-02 |
Family
ID=67023893
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910151313.XA Pending CN109953763A (en) | 2019-02-28 | 2019-02-28 | A kind of vehicle carried driving behavioral value early warning system and method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109953763A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110491091A (en) * | 2019-09-08 | 2019-11-22 | 湖北汽车工业学院 | A kind of commercial vehicle driver fatigue state monitoring and warning system |
| CN110807898A (en) * | 2019-11-04 | 2020-02-18 | 中国第一汽车股份有限公司 | Fatigue driving prevention method and system |
| CN111882827A (en) * | 2020-07-27 | 2020-11-03 | 复旦大学 | Fatigue driving monitoring method, system and device and readable storage medium |
| CN112641445A (en) * | 2021-01-19 | 2021-04-13 | 深圳市阿尔法车联网技术有限公司 | Automobile user health and behavior monitoring system and method based on Internet of vehicles data informatization |
| CN113116352A (en) * | 2021-03-24 | 2021-07-16 | 深兰科技（上海）有限公司 | Fatigue state prediction method, fatigue state prediction device, electronic equipment and storage medium |
| CN113331846A (en) * | 2021-06-30 | 2021-09-03 | 易念科技（深圳）有限公司 | Driving state detection method, detection device and computer readable storage medium |
| TWI823577B (en) * | 2021-11-26 | 2023-11-21 | 國立成功大學 | Exercise training system able to recognize fatigue of user |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN207637164U (en) * | 2017-11-10 | 2018-07-20 | 厦门大学 | A kind of vehicle-mounted fatigue driving monitoring system |
| CN108995531A (en) * | 2018-08-09 | 2018-12-14 | 爱驰汽车有限公司 | fatigue warning device and method for automobile |
| CN109101891A (en) * | 2018-07-17 | 2018-12-28 | 哈尔滨理工大学 | A kind of rice pest detection system and its detection method merging artificial intelligence |
-
2019
- 2019-02-28 CN CN201910151313.XA patent/CN109953763A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN207637164U (en) * | 2017-11-10 | 2018-07-20 | 厦门大学 | A kind of vehicle-mounted fatigue driving monitoring system |
| CN109101891A (en) * | 2018-07-17 | 2018-12-28 | 哈尔滨理工大学 | A kind of rice pest detection system and its detection method merging artificial intelligence |
| CN108995531A (en) * | 2018-08-09 | 2018-12-14 | 爱驰汽车有限公司 | fatigue warning device and method for automobile |
Non-Patent Citations (4)
| Title |
|---|
| 周凌霄: "多源生理信号融合的驾驶疲劳检测预警系统研究", 《中国优秀硕士学位论文全文数据库》 * |
| 孙宇平: "基于稀疏表征和自相似性的视觉数据识别关键技术及应用", 《中国博士学位论文全文数据库》 * |
| 搜狐新闻（HTTPS://WWW.SOHU.COM/A/255283777_500254）: "又是一等奖！这群扬大人，明明可以拼颜值，确偏偏要靠才华……", 《搜狐新闻》 * |
| 王军: "驾驶员疲劳检测算法研究", 《中国优秀硕士学位论文全文数据库》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110491091A (en) * | 2019-09-08 | 2019-11-22 | 湖北汽车工业学院 | A kind of commercial vehicle driver fatigue state monitoring and warning system |
| CN110807898A (en) * | 2019-11-04 | 2020-02-18 | 中国第一汽车股份有限公司 | Fatigue driving prevention method and system |
| CN111882827A (en) * | 2020-07-27 | 2020-11-03 | 复旦大学 | Fatigue driving monitoring method, system and device and readable storage medium |
| CN112641445A (en) * | 2021-01-19 | 2021-04-13 | 深圳市阿尔法车联网技术有限公司 | Automobile user health and behavior monitoring system and method based on Internet of vehicles data informatization |
| CN112641445B (en) * | 2021-01-19 | 2021-11-09 | 深圳市阿尔法车联网技术有限公司 | Automobile user health and behavior monitoring system and method based on Internet of vehicles data informatization |
| CN113116352A (en) * | 2021-03-24 | 2021-07-16 | 深兰科技（上海）有限公司 | Fatigue state prediction method, fatigue state prediction device, electronic equipment and storage medium |
| CN113331846A (en) * | 2021-06-30 | 2021-09-03 | 易念科技（深圳）有限公司 | Driving state detection method, detection device and computer readable storage medium |
| CN113331846B (en) * | 2021-06-30 | 2024-01-02 | 易念科技（深圳）有限公司 | Driving state detection method, detection device and computer readable storage medium |
| TWI823577B (en) * | 2021-11-26 | 2023-11-21 | 國立成功大學 | Exercise training system able to recognize fatigue of user |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109953763A (en) | A kind of vehicle carried driving behavioral value early warning system and method based on deep learning | |
| CN108765876A (en) | Driving fatigue depth analysis early warning system based on multimode signal and method | |
| CN111166357A (en) | Fatigue monitoring device system with multi-sensor fusion and monitoring method thereof | |
| CN107536617A (en) | The apparatus and method of bio-identification signal detection driver status based on driver | |
| CN107979985B (en) | Vehicle-mounted healthy and safe driving auxiliary device | |
| CN108464839A (en) | A kind of vehicle-mounted fatigue monitoring and early warning of driving system based on machine learning | |
| CN109717853A (en) | Vehicle carried driving person's fatigue detecting system based on raspberry pie | |
| CN104068868A (en) | Method and device for monitoring driver fatigue on basis of machine vision | |
| CN106128032A (en) | A kind of fatigue state monitoring and method for early warning and system thereof | |
| CN101551934B (en) | Device and method for monitoring fatigue driving of driver | |
| CN109389806A (en) | Fatigue driving detection method for early warning, system and medium based on multi-information fusion | |
| CN107669283A (en) | Spectacle driving fatigue monitoring warning device based on multi-feature fusion and method | |
| CN111631697A (en) | Intelligent sleep and fatigue state information monitoring control system and method and monitor | |
| CN208126407U (en) | Anti-fatigue-driving system based on software-hardware synergism image procossing | |
| CN209252893U (en) | The vehicle-mounted detection of one kind and interfering system and vehicle-mounted detecting system | |
| CN107563346A (en) | One kind realizes that driver fatigue sentences method for distinguishing based on eye image processing | |
| CN109567832A (en) | A kind of method and system of the angry driving condition of detection based on Intelligent bracelet | |
| CN110367975A (en) | A kind of fatigue driving detection method for early warning based on brain-computer interface | |
| CN113288168A (en) | Wearable fatigue monitoring of intelligence and early warning system | |
| CN109875584A (en) | The detection method and its warning system of physiological driver's fatigue | |
| CN106446822B (en) | Blink detection method based on circle fitting | |
| CN112107295A (en) | Data processing method and system of wearable device, storage medium and wearable device | |
| CN109907756A (en) | Driving Fatigue Monitoring System based on head pose information and eye information | |
| CN109858178A (en) | A kind of commercial vehicle drivers giving fatigue pre-warning method based on Intelligent bracelet | |
| CN106384096A (en) | Fatigue driving monitoring method based on blink detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |