CN109886241A - Driver fatigue detection based on shot and long term memory network - Google Patents
Driver fatigue detection based on shot and long term memory network Download PDFInfo
- Publication number
- CN109886241A CN109886241A CN201910156037.6A CN201910156037A CN109886241A CN 109886241 A CN109886241 A CN 109886241A CN 201910156037 A CN201910156037 A CN 201910156037A CN 109886241 A CN109886241 A CN 109886241A
- Authority
- CN
- China
- Prior art keywords
- driver
- shot
- fatigue
- long term
- eye
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 28
- 230000007787 long-term memory Effects 0.000 title claims abstract description 14
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 25
- 238000013528 artificial neural network Methods 0.000 claims abstract description 9
- 230000001815 facial effect Effects 0.000 claims abstract description 8
- 238000000034 method Methods 0.000 claims abstract description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 10
- 239000011521 glass Substances 0.000 claims description 7
- 238000012545 processing Methods 0.000 claims description 4
- 230000008447 perception Effects 0.000 claims description 2
- 238000012549 training Methods 0.000 claims description 2
- 230000000007 visual effect Effects 0.000 claims description 2
- 238000009412 basement excavation Methods 0.000 claims 1
- 230000001629 suppression Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 5
- 239000000284 extract Substances 0.000 description 5
- 230000002123 temporal effect Effects 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 206010039203 Road traffic accident Diseases 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 210000001747 pupil Anatomy 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The present invention relates to a kind of Driver Fatigue Detections based on shot and long term memory network, comprising: 1) acquires driver's facial video image by infrared collecting equipment；2) Face datection and positioning feature point are carried out using the cascade convolutional neural networks of multitask, driver's eye image sequence is obtained according to the geometrical relationship between characteristic point；3) a kind of convolution loop neural network end to end is devised, eye space feature is extracted, while analyzing context relation between adjacent image frame, judges whether driver is in a state of fatigue in conjunction with the timing variations of eye image feature in a period of time.The result shows that, this method is when light condition difference and driver wear sunglasses, also eye feature can accurately be extracted, compared to the fatigue detection method based on CNN combination PERCLOS standard, higher fatigue detecting accuracy rate is achieved, is realized to the other prediction of driver's driving condition videl stage.
Description
Technical field
The present invention relates to a kind of Driver Fatigue Detections based on shot and long term memory network, in sensitivity, robustness
And it is more excellent than the prior art in terms of accuracy, there is good detection performance, belong to image procossing, deep learning field.
Background technique
Studies have shown that fatigue driving is to cause the one of the major reasons of traffic accident, national laws regulation is all prohibited
Fatigue driving.But due to its apparent gradual change, hidden and subjective rejection characteristic, regulatory agency is difficult to monitor in time.If driving
Reminding in time when fatigue state occurs in member, will be effectively prevented from traffic accident.Therefore, fatigue detecting is carried out to driver
With great research significance and social value.
The fatigue detection method of view-based access control model feature is by extensive concern, and presently, there are some difficult points: (1) current driver's
Fatigue detecting associated data set is relatively deficient；(2) in actual driving procedure, the complicated multiplicity of driving environment, the variation pair of light
It extracts visual signature and causes very big interference；(3) variation of face orientation and head pose will affect to a certain extent drives
The integrality of the person's of sailing face image acquisition.In addition when driver's wearing spectacles or sunglasses, eye areas can be caused to block.
There are many kinds of traditional Driver Fatigue Detections, but there are many drawbacks.Fatigue based on physiological signal
Detection method Detection accuracy theoretically with higher, but be easy in practical applications by environmental factor, detector
The influence of device, the accuracy of apparatus measures and driver personal driving habit etc., poor anti jamming capability, detection device price are high
Expensive, in actual use, the measurement of physiological signal needs wearing device, usually the detection of contact, can be to driver
It makes troubles and interferes, operability is not strong, it is difficult to is a wide range of to promote；Fatigue detection method based on driving behavior can be by
The influence of the factors such as driving habit difference, vehicle model and condition of road surface to different drivers, it is difficult since there are individual differences
To determine the threshold value of the standard diagrams of driver fatigue, the correlation between the parameter and degree of fatigue of fatigue characteristic be not easy by
Quantization.
In recent years, deep learning algorithm achieves important breakthrough, and deep learning is introduced fatigue detecting field, passes through convolution
Neural network scheduling algorithm is extracted and is analyzed to driver ocular characteristics, finally makes tired judgement, and this way is not only protected
Conventional machines vision algorithm and the contactless advantage of human body have been stayed, the dependence to special parameter threshold value and individual difference are also reduced
Different bring influences, and greatly enhances the accuracy rate of fatigue detecting.
Summary of the invention
The invention proposes a kind of fatigue detection method based on deep learning and driver eye's feature examines fatigue
Survey problem regards the identification problem based on eye image sequence as, is driven first by the cascade convolutional neural networks of multitask
Member's Face datection and positioning feature point obtain driver's eye image sequence according to the geometrical relationship between characteristic point, extract one
The space temporal aspect of driver eye's state in the section time, finally according to the context relation between the feature of eye consecutive frame
Make tired judgement.
Technical solution of the present invention, including the following steps:
Step 1: building infrared video acquisition system and acquire driver respectively and do not wearing glasses, wear dark glasses, wearing spectacles
Facial video image in the case of three kinds forms fatigue detecting data set；
Step 2: being passed through using the detection algorithm based on the cascade convolutional neural networks of multitask to three convolutional Neural nets
Network is cascaded, and can be realized simultaneously Face datection and face feature point location；
Step 3: the facial image in step 2 being handled, driver's eyes region can be according to obtained characteristic point
The symmetry and experimental results of coordinate combination eyes distribution determine position constraint condition to solve；
Step 4: driver's eye image sequence data obtained in step 3 is input to convolution loop nerve end to end
In network, by deep layer convolutional coding structure in conjunction with shot and long term memory unit, by analyze eye image sequence in a period of time when
Order relation carries out tired judgement.
Compared with prior art, the beneficial effects of the present invention are:
The present invention carries out the study for having supervision on the basis of low volume data, avoids complicated image processing process.?
It is more excellent than the prior art in sensitivity, robustness and accuracy, and accuracy rate has reached 98.96%, illustrates that the present invention has
Good detection performance.
In addition, also can accurately extract eye spy for when light condition difference and driver wear sunglasses
Sign, compared to the fatigue detection method based on CNN combination PERCLOS standard, achieves higher fatigue detecting accuracy rate, therefore
The present invention has preferable testing result for driver tired driving.
Detailed description of the invention
Fig. 1 overall framework schematic diagram, i.e. Figure of abstract；
Fig. 2 Face datection and positioning feature point partial detection figure；
Fig. 3 ocular extracts schematic diagram；
Fig. 4 fatigue detecting frame diagram；
Fig. 5 (a) SE function structure chart；
Fig. 5 (b) Res-SE function structure chart；
Fig. 6 FDRNet network structure.
Specific embodiment
The present invention is described in further detail With reference to embodiment.
Overall framework schematic diagram of the invention as shown in Figure 1, acquire driving by infrared video acquisition system first respectively
Member in the case that do not wear glasses, wear dark glasses, wear three kinds of spectacles facial video image；Then pass through the cascade volume of multitask
Product neural network carries out Face datection and positioning feature point, obtains driver's eye image according to the geometrical relationship between characteristic point
Sequence；Next, devising a kind of convolution loop neural network model end to end, driver eye's shape in a period of time is extracted
The space temporal aspect of state, finally makes tired judgement according to the context relation between the feature of eye adjacent image frame.
With reference to the accompanying drawing, the specific implementation process of technical solution of the present invention is illustrated.
1. experimental subjects
Fatigue detecting image data set of the invention by 30 experimenters respectively do not wear glasses, wear spectacles with
And wear the experimental situation Imitating fatigue of three kinds of situations of sunglasses and two kinds of driving conditions of regaining consciousness, while utilizing infrared collecting system
Experimenter face video is acquired, fatigue detecting data set TJPU-FDD is formed, amounts to a length of 6 seconds views of 500 mean times
Frequency segment, screen resolution are 1920 × 1080.
2. Face datection and positioning feature point
Present invention employs the detection algorithms based on the cascade convolutional neural networks of multitask to pass through to three convolutional Neurals
Network is cascaded, and can be realized simultaneously Face datection and face feature point location.First in data processing stage, to cope with mesh
Issues On Multi-scales are marked, original facial image is zoomed into different sizes, image pyramid is constructed, as the defeated of three rank cascades
Enter.Then (raw similar to the region of Faster-RNN using full convolutional network P-Net on the basis of building image is pyramidal
At network) generate candidate window and frame regression vector.Then it is generated on last stage using a convolutional neural networks R-Net is selected
Face candidate frame, further give up the candidate forms of overlapping.Finally by another convolutional neural networks O-Net, obtain most
Whole face frame and five characteristic point positions, as shown in Figure 2.
3. driver eye's extracted region
Face frame and five characteristic point positions, eyes are detected with the cascade convolutional neural networks algorithm of multitask
Area-of-interest positioning can be determined according to the symmetry and experimental results of obtained characteristic point coordinate combination eyes distribution
Position constraint condition solves: (1) two interpupillary distances are about 1.7 times of single eye widths；(2) eyes the ratio of width to height is chosen for
5:4；(3) between eye areas vertex to be extracted and pupil position the range difference of transverse and longitudinal coordinate be respectively half the wide height of eyes
Value.Driver's eyes region, midpoint A, B equal generation are determined according to the geometrical relationship between above-mentioned 3 middle left and rights eye characteristic point
Table driver human eye feature point, coordinate are respectively (xA, yA)、(xB, yB), the distance between two o'clock is d.Point C represents left side eyes
Region top left corner apex, coordinate are (xC, yC), w, h respectively represent the width of corresponding eye areas, height, as shown in Figure 3.
The 4 driver fatigue detections based on shot and long term memory network
4.1 fatigue detecting frames end to end
The depth of deep layer convolutional neural networks embodies the quantity of spatially network layer number, is held very much by profound CNN
The space characteristics for easily extracting image obtain the layer-stepping expression of characteristics of image, but cannot be in the time domain to the variation of image sequence
It is modeled.Relative to CNN, the depth of Recognition with Recurrent Neural Network, which is embodied in its network structure, to be unfolded in the time domain.In time domain
The network structure of upper chain shows that RNN is a kind of neural network of suitable processing time series data.LSTM is a kind of special RNN,
It can be realized distance learning and prevent gradient from disappearing and exploding, be more suitable for fatigue detecting task.
The present invention by end-to-end, trainable fatigue detecting frame (as shown in Figure 4), realize convolutional neural networks with
The mutual supplement with each other's advantages of two kinds of structures of shot and long term memory unit.The space characteristics that eyes are extracted using profound convolutional layer, are passed through
LSTM unit analyzes the sequential relationship of the adjacent eyes image of driver in driving procedure.Compared with single CNN model, the fatigue
Detection framework may learn eye feature deeper modular table on room and time and reach, to improve fatigue detecting
Accuracy rate.Firstly, by being obtained from video clip based on the cascade convolutional neural networks human eye area location algorithm of multitask
Take the continuous frame sequence of eyes image.Then regard continuous 16 frame of each video as a time step, in each time step
Input { x1, x2..., x15, x16By a CNN module with deep layer convolutional layer, learn the profound convolution of human eye
Perception expression, generates 1 × 1 × 512 vector, and is input in the Sequence Learning module that one is made of LSTM unit, to
The correlation of capture time sequence finally generates a two-dimensional vector forecasting result.In order to prevent the size of model parameter with
Sequence length proportionally increases, and the weight of fatigue detecting frame Optimized model in the form of end to end, these weights are each
It is reused in time step.Space characteristics extraction module is connected directly with temporal aspect extraction module, can pass through joint
The driving condition to judge driver in each time step is trained, the recognition result in each time step is then integrated.Tool
For body, in order to carry out single Tag Estimation to entire video clip, in the end of network, using softmax layers to sometimes
Between step-length feature output carry out consolidated forecast as final recognition result.
4.2 convolution loop neural network FDRNet
Fatigue detecting is considered as two classification problems to driving condition by the present invention, extracts space characteristics in conjunction with deep layer convolution
The advantage of ability and LSTM modular learning temporal aspect devises a kind of end-to-end, trainable convolution loop neural network
Structure (FDRNet), which is a kind of variant of ResNet-10, in FDRNet, by SE Module-embedding residual error structure, from
While simplifying e-learning on Spatial Dimension, and the performance of network is promoted by measuring feature channel interdependency.
Fig. 5 (b) be embedded in SE module after residual error structure Res-SE, wherein Conv_1, Conv_2 be convolution kernel be 3 ×
3 convolutional layer, the step-length that the step-length of Conv_1 is 2, Conv_2 are 1.SE module acts on Conv_2, by the study of residual error branch
To feature carry out recalibration after export.Branch where convolutional layer Conv_3 is the quick connection branch of residual error structure, convolution
Core is having a size of 1 × 1, step-length 2.If input dimension is H × W × C, after Res-SE module, output dimension becomes H/2 × W/
2 × 2C, e-learning have arrived the residual error feature by feature recalibration, have improved the information useful to fatigue detecting task, press down
The little information of effect is made.The input channel number of Res-SE module is indicated with C.In addition, BN layers, ReLU layers have been put into pre- sharp
In work, the regularization of model can be improved in this way, so that a possibility that reducing over-fitting, further promotes network performance.Fig. 6
The network structure of FDRNet is depicted, wherein space characteristics extractor is by convolutional layer Conv, the maximum pond with residual error mapping
Layer Pool, three Res-SE modules and global pool layer composition.Conv selects 7 × 7 big convolution kernels, step-length 2.Three Res-
SE module input feature vector port number C is respectively 64,128,256, and wherein the convolution kernel size of residual error branch road is 3 × 3, fast
The convolution that branch road is 1 × 1.Eye image is exported via the vector that space characteristics extractor obtains regular length, exports dimension
It is 1 × 1 × 512.Then by the reshape layer of entitled " Reshape_CNN " by the output dimension of CNN be adjusted to T × N ×
512, one of the input as LSTM unit is subsequently communicated to temporal aspect extractor and is handled.Here it is mono- that T represents LSTM
The time step of member, represents the video frame number of a time step, T=16 herein in a network.N represents LSTM unit while locating
The number of the independent data stream of reason indicates the quantity of video-frequency band, in network training in N be set as 16；In model verifying, N
It is set as 4.Another input of LSTM is Sequential Continuity mark, indicates and represents currently processed picture frame for " 0 " as time step
Beginning, indicate the continuity for representing current image frame for " 1 " as previous frame.Pass through another entitled " Resahpe_markers "
Reshape layer the dimension of continuity mark is adjusted to T × N.It is two-dimensional that each video-frequency band is finally obtained on full articulamentum
Character representation, the probability of two category features is calculated by softmax layers, and the classification of maximum probability is final tired court verdict.
Invention is a kind of Driver Fatigue Detection based on shot and long term memory network, achieves higher fatigue inspection
Accuracy rate is surveyed, is realized to the other prediction of driver's driving condition videl stage.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention, should
Understand, the present invention is not limited to implementation as described herein, the purpose of these implementations description is to help this field
In technical staff practice the present invention.Any those of skill in the art are easy to do not departing from spirit and scope of the invention
In the case of be further improved and perfect, therefore the present invention is only by the content of the claims in the present invention and the limit of range
System, intention, which covers, all to be included the alternative in the spirit and scope of the invention being defined by the appended claims and waits
Same scheme.
Claims (5)
1. a kind of Driver Fatigue Detection based on shot and long term memory network, including the following steps:
Step 1: building infrared video acquisition system and acquire driver respectively and do not wearing glasses, wear dark glasses, wearing three kinds of spectacles
In the case of facial video image form fatigue detecting data set；
Step 2: using the detection algorithm based on the cascade convolutional neural networks of multitask by three convolutional neural networks into
Row cascade, can be realized simultaneously Face datection and face feature point location；
Step 3: the facial image in step 2 being handled, driver's eyes region can be according to obtained characteristic point coordinate
The symmetry and experimental results being distributed in conjunction with eyes determine position constraint condition to solve；
Step 4: eye image sequence data obtained in step 3 is input in convolution loop neural network end to end, it will
In conjunction with shot and long term memory unit, the sequential relationship by analyzing eye image sequence in a period of time carries out deep layer convolutional coding structure
Fatigue judgement.
2. the Driver Fatigue Detection according to claim 1 based on shot and long term memory network, which is characterized in that step
In rapid 1, driver's face image, and the infrared light for the 850nm that arranges in pairs or groups are acquired using the infrared camera of 3.6mm focal length, 70 degree of visual angles
Source carries out light filling, while the interference of other wavelength lights is reduced by the narrow band filter of 850nm, and can obtain through sunglasses
Clearly ocular image is got, while being also able to satisfy the demand of night use, improves the robustness of algorithm.
3. the Driver Fatigue Detection according to claim 1 based on shot and long term memory network, which is characterized in that step
In rapid 2, original facial image is zoomed into different sizes, structure figures to cope with target Issues On Multi-scales in data processing stage
As pyramid, as the input of three rank cascades, then on the basis of building image is pyramidal, full convolutional network P- is utilized
Net generates candidate window and frame regression vector, then uses the selected people generated on last stage of a convolutional neural networks R-Net
Face candidate frame further gives up the candidate forms of overlapping, finally by another convolutional neural networks O-Net, obtains final
Face frame and five characteristic point positions.
4. the Driver Fatigue Detection according to claim 1 based on shot and long term memory network, which is characterized in that step
In rapid 3, in order to obtain driver's eye image sequence, pass through the cascade convolutional neural networks locating human face frame of multitask and spy
A sign point position, two problems of Face datection and positioning feature point are unified under multitask frame, and using three sub-networks and
Non-maxima suppression completes the excavation of sample in the training process, and the coordinate of eyes area-of-interest can pass through characteristic point coordinate
Binding site constraint condition solves.
5. the Driver Fatigue Detection according to claim 1 based on shot and long term memory network, which is characterized in that step
In rapid 4, by obtaining eye figure from video clip based on the cascade convolutional neural networks human eye area location algorithm of multitask
The continuous frame sequence of picture then regards continuous 16 frame of each video as a time step, and the input in each time step is logical
Cross the CNN module with deep layer convolutional layer, learn the profound convolution perception expression of human eye, generate dimension be 1 × 1 ×
512 vector, and be input in the Sequence Learning module that one is made of LSTM unit, the correlation to capture time sequence
Property, finally generate a two-dimensional vector forecasting result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910156037.6A CN109886241A (en) | 2019-03-05 | 2019-03-05 | Driver fatigue detection based on shot and long term memory network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910156037.6A CN109886241A (en) | 2019-03-05 | 2019-03-05 | Driver fatigue detection based on shot and long term memory network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109886241A true CN109886241A (en) | 2019-06-14 |
Family
ID=66930300
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910156037.6A Pending CN109886241A (en) | 2019-03-05 | 2019-03-05 | Driver fatigue detection based on shot and long term memory network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109886241A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110363093A (en) * | 2019-06-19 | 2019-10-22 | 深圳大学 | A kind of driver's action identification method and device |
| CN110427871A (en) * | 2019-07-31 | 2019-11-08 | 长安大学 | A kind of method for detecting fatigue driving based on computer vision |
| CN110490299A (en) * | 2019-07-25 | 2019-11-22 | 南京信息工程大学 | Sensitive shot and long term accumulating method based on state change differential |
| CN110633665A (en) * | 2019-09-05 | 2019-12-31 | 卓尔智联(武汉)研究院有限公司 | Recognition method, device and storage medium |
| CN110717389A (en) * | 2019-09-02 | 2020-01-21 | 东南大学 | Driver fatigue detection method based on generation of countermeasure and long-short term memory network |
| CN110738190A (en) * | 2019-10-28 | 2020-01-31 | 北京经纬恒润科技有限公司 | fatigue driving judgment method, device and equipment |
| CN111027391A (en) * | 2019-11-12 | 2020-04-17 | 湖南大学 | Fatigue state identification method based on CNN pyramid characteristics and LSTM |
| CN111243236A (en) * | 2020-01-17 | 2020-06-05 | 南京邮电大学 | Fatigue driving early warning method and system based on deep learning |
| CN111645695A (en) * | 2020-06-28 | 2020-09-11 | 北京百度网讯科技有限公司 | Fatigue driving detection method and device, computer equipment and storage medium |
| CN111723694A (en) * | 2020-06-05 | 2020-09-29 | 广东海洋大学 | Abnormal driving behavior identification method based on CNN-LSTM space-time feature fusion |
| CN111723707A (en) * | 2020-06-09 | 2020-09-29 | 天津大学 | Method and device for estimating fixation point based on visual saliency |
| CN111814956A (en) * | 2020-06-23 | 2020-10-23 | 哈尔滨工程大学 | Multi-task learning air quality prediction method based on multi-dimensional secondary feature extraction |
| CN112434611A (en) * | 2020-11-25 | 2021-03-02 | 中山大学 | Early fatigue detection method and system based on eye movement subtle features |
| CN112580527A (en) * | 2020-12-22 | 2021-03-30 | 之江实验室 | Facial expression recognition method based on convolution long-term and short-term memory network |
| CN112906631A (en) * | 2021-03-17 | 2021-06-04 | 南京邮电大学 | Dangerous driving behavior detection method and detection system based on video |
| CN113128360A (en) * | 2021-03-30 | 2021-07-16 | 苏州乐达纳米科技有限公司 | Driver driving behavior detection and identification method based on deep learning |
| CN117079255A (en) * | 2023-10-17 | 2023-11-17 | 江西开放大学 | Fatigue driving detection method based on face recognition and voice interaction |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107748858A (en) * | 2017-06-15 | 2018-03-02 | 华南理工大学 | A kind of multi-pose eye locating method based on concatenated convolutional neutral net |
| US20180150684A1 (en) * | 2016-11-30 | 2018-05-31 | Shenzhen AltumView Technology Co., Ltd. | Age and gender estimation using small-scale convolutional neural network (cnn) modules for embedded systems |
| CN108294759A (en) * | 2017-01-13 | 2018-07-20 | 天津工业大学 | A kind of Driver Fatigue Detection based on CNN Eye state recognitions |
| CN108921051A (en) * | 2018-06-15 | 2018-11-30 | 清华大学 | Pedestrian's Attribute Recognition network and technology based on Recognition with Recurrent Neural Network attention model |
| CN109034159A (en) * | 2018-05-28 | 2018-12-18 | 北京捷通华声科技股份有限公司 | image information extracting method and device |
| CN109376603A (en) * | 2018-09-25 | 2019-02-22 | 北京周同科技有限公司 | A kind of video frequency identifying method, device, computer equipment and storage medium |
-
2019
- 2019-03-05 CN CN201910156037.6A patent/CN109886241A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180150684A1 (en) * | 2016-11-30 | 2018-05-31 | Shenzhen AltumView Technology Co., Ltd. | Age and gender estimation using small-scale convolutional neural network (cnn) modules for embedded systems |
| CN108294759A (en) * | 2017-01-13 | 2018-07-20 | 天津工业大学 | A kind of Driver Fatigue Detection based on CNN Eye state recognitions |
| CN107748858A (en) * | 2017-06-15 | 2018-03-02 | 华南理工大学 | A kind of multi-pose eye locating method based on concatenated convolutional neutral net |
| CN109034159A (en) * | 2018-05-28 | 2018-12-18 | 北京捷通华声科技股份有限公司 | image information extracting method and device |
| CN108921051A (en) * | 2018-06-15 | 2018-11-30 | 清华大学 | Pedestrian's Attribute Recognition network and technology based on Recognition with Recurrent Neural Network attention model |
| CN109376603A (en) * | 2018-09-25 | 2019-02-22 | 北京周同科技有限公司 | A kind of video frequency identifying method, device, computer equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 张国基等: "《生物辨识系统与深度学习》", 31 December 2018 * |
| 耿磊等: "基于多形态红外特征与深度学习的实时驾驶员疲劳检测", 《红外与激光工程》 * |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110363093A (en) * | 2019-06-19 | 2019-10-22 | 深圳大学 | A kind of driver's action identification method and device |
| CN110490299A (en) * | 2019-07-25 | 2019-11-22 | 南京信息工程大学 | Sensitive shot and long term accumulating method based on state change differential |
| CN110490299B (en) * | 2019-07-25 | 2022-07-29 | 南京信息工程大学 | Sensitive long-short term memory method based on state change differential |
| CN110427871A (en) * | 2019-07-31 | 2019-11-08 | 长安大学 | A kind of method for detecting fatigue driving based on computer vision |
| CN110427871B (en) * | 2019-07-31 | 2022-10-14 | 长安大学 | Fatigue driving detection method based on computer vision |
| CN110717389B (en) * | 2019-09-02 | 2022-05-13 | 东南大学 | Driver fatigue detection method based on generation countermeasure and long-short term memory network |
| CN110717389A (en) * | 2019-09-02 | 2020-01-21 | 东南大学 | Driver fatigue detection method based on generation of countermeasure and long-short term memory network |
| CN110633665A (en) * | 2019-09-05 | 2019-12-31 | 卓尔智联(武汉)研究院有限公司 | Recognition method, device and storage medium |
| CN110633665B (en) * | 2019-09-05 | 2023-01-10 | 卓尔智联(武汉)研究院有限公司 | Identification method, device and storage medium |
| CN110738190A (en) * | 2019-10-28 | 2020-01-31 | 北京经纬恒润科技有限公司 | fatigue driving judgment method, device and equipment |
| CN111027391A (en) * | 2019-11-12 | 2020-04-17 | 湖南大学 | Fatigue state identification method based on CNN pyramid characteristics and LSTM |
| CN111243236A (en) * | 2020-01-17 | 2020-06-05 | 南京邮电大学 | Fatigue driving early warning method and system based on deep learning |
| CN111723694A (en) * | 2020-06-05 | 2020-09-29 | 广东海洋大学 | Abnormal driving behavior identification method based on CNN-LSTM space-time feature fusion |
| CN111723707A (en) * | 2020-06-09 | 2020-09-29 | 天津大学 | Method and device for estimating fixation point based on visual saliency |
| CN111723707B (en) * | 2020-06-09 | 2023-10-17 | 天津大学 | Gaze point estimation method and device based on visual saliency |
| CN111814956A (en) * | 2020-06-23 | 2020-10-23 | 哈尔滨工程大学 | Multi-task learning air quality prediction method based on multi-dimensional secondary feature extraction |
| CN111814956B (en) * | 2020-06-23 | 2022-04-08 | 哈尔滨工程大学 | Multi-task learning air quality prediction method based on multi-dimensional secondary feature extraction |
| CN111645695A (en) * | 2020-06-28 | 2020-09-11 | 北京百度网讯科技有限公司 | Fatigue driving detection method and device, computer equipment and storage medium |
| CN112434611A (en) * | 2020-11-25 | 2021-03-02 | 中山大学 | Early fatigue detection method and system based on eye movement subtle features |
| CN112580527A (en) * | 2020-12-22 | 2021-03-30 | 之江实验室 | Facial expression recognition method based on convolution long-term and short-term memory network |
| CN112906631B (en) * | 2021-03-17 | 2022-07-29 | 南京邮电大学 | Dangerous driving behavior detection method and detection system based on video |
| CN112906631A (en) * | 2021-03-17 | 2021-06-04 | 南京邮电大学 | Dangerous driving behavior detection method and detection system based on video |
| CN113128360A (en) * | 2021-03-30 | 2021-07-16 | 苏州乐达纳米科技有限公司 | Driver driving behavior detection and identification method based on deep learning |
| CN117079255A (en) * | 2023-10-17 | 2023-11-17 | 江西开放大学 | Fatigue driving detection method based on face recognition and voice interaction |
| CN117079255B (en) * | 2023-10-17 | 2024-01-05 | 江西开放大学 | Fatigue driving detection method based on face recognition and voice interaction |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109886241A (en) | Driver fatigue detection based on shot and long term memory network | |
| CN111259850B (en) | Pedestrian re-identification method integrating random batch mask and multi-scale representation learning | |
| CN106951867B (en) | Face identification method, device, system and equipment based on convolutional neural networks | |
| CN110569772B (en) | Method for detecting state of personnel in swimming pool | |
| CN106503687B (en) | Merge the monitor video system for identifying figures and its method of face multi-angle feature | |
| CN109819208A (en) | A kind of dense population security monitoring management method based on artificial intelligence dynamic monitoring | |
| CN104063719B (en) | Pedestrian detection method and device based on depth convolutional network | |
| CN104378582B (en) | A kind of intelligent video analysis system and method cruised based on Pan/Tilt/Zoom camera | |
| CN103824070B (en) | A kind of rapid pedestrian detection method based on computer vision | |
| CN110378259A (en) | A kind of multiple target Activity recognition method and system towards monitor video | |
| CN107451568A (en) | Use the attitude detecting method and equipment of depth convolutional neural networks | |
| CN106529419B (en) | The object automatic testing method of saliency stacking-type polymerization | |
| CN101853399B (en) | Method for realizing blind road and pedestrian crossing real-time detection by utilizing computer vision technology | |
| CN109034134A (en) | Abnormal driving behavioral value method based on multitask depth convolutional neural networks | |
| Li et al. | Sign language recognition based on computer vision | |
| CN113239822A (en) | Dangerous behavior detection method and system based on space-time double-current convolutional neural network | |
| CN108960047A (en) | Face De-weight method in video monitoring based on the secondary tree of depth | |
| Hwang et al. | Multi-modal human action recognition using deep neural networks fusing image and inertial sensor data | |
| Elihos et al. | Comparison of image classification and object detection for passenger seat belt violation detection using NIR & RGB surveillance camera images | |
| CN103902989A (en) | Human body motion video recognition method based on non-negative matrix factorization | |
| CN113378649A (en) | Identity, position and action recognition method, system, electronic equipment and storage medium | |
| CN103049748B (en) | Behavior monitoring method and device | |
| Pramita et al. | Mask wearing classification using CNN | |
| CN111860457A (en) | Fighting behavior recognition early warning method and recognition early warning system thereof | |
| CN115761618A (en) | Key site security monitoring image identification method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190614 |
|
| WD01 | Invention patent application deemed withdrawn after publication |