CN108446645A - Vehicle-mounted face identification method based on deep learning - Google Patents
Vehicle-mounted face identification method based on deep learning Download PDFInfo
- Publication number
- CN108446645A CN108446645A CN201810252216.5A CN201810252216A CN108446645A CN 108446645 A CN108446645 A CN 108446645A CN 201810252216 A CN201810252216 A CN 201810252216A CN 108446645 A CN108446645 A CN 108446645A
- Authority
- CN
- China
- Prior art keywords
- driver
- master control
- control end
- identification method
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
A kind of vehicle-mounted face identification method based on deep learning：Obtain image, structure Driver data's collection；Model is built, including extraction visual signature and semantic feature and constitutive characteristic handle model；Training characteristics handle model；According to test result Optimal Experimental as a result, the characteristic processing model after optimization is transmitted to master control end；Warning lamp and camera are onboard installed, the operational circumstances of driver are uploaded into master control end in real time, master control end judges whether driver has according to the characteristic processing model after optimization and drives in violation of rules and regulations, it is driven in violation of rules and regulations when having, master control end sends out signal, excites warning lamp, reminds driver's civilization traveling.The present invention by analyzing face variation in real time, extract face characteristic, detect whether it has violation operation, analysis comparison is carried out with data set, once it was found that phenomena such as having unlawful practice, such as finding fatigue driving and receive calls, see the mobile phone, alarm can be sent out automatically, prevent the bad behavior of driver in time.Reduce the possibility that traffic accident occurs.
Description
Technical field
The present invention relates to a kind of face identification methods.More particularly to a kind of vehicle-mounted recognition of face side based on deep learning
Method.
Background technology
With the fast development of acquisition of information and the information processing technology, how computer vision utilizes computer technology
Efficiently and accurately obtain relevant information from ambient image or video, so in objective world things and there is a phenomenon where
It analyzed, judged and decision, have become a very important research topic.With the fast development of deep learning, meter
Calculation machine vision is quickly grown in recent years, and deep layer convolutional neural networks play the research and development in terms of progress computer vision
Very important effect.
With the fast development of computer vision, face recognition technology growth momentum is good, and the application of recognition of face is also got over
Come more extensive, recognition of face is a kind of biological identification technology that the facial characteristics based on face is identified, and passes through video camera etc.
Image capture device acquires image or video flowing containing face, then carries out Face datection and face tracking in the picture,
And then the face to detecting carries out a series of relevant the relevant technologies of face.Face recognition technology includes mainly that (1) is several at present
The face identification method of what feature；(2) the recognition of face side of face identification method (3) neural network of feature based face (PCA)
The face identification method (6) of face identification method (5) line segment Hausdorff distance (LHD) of method (4) elastic graph matching support to
The face identification method of amount machine (SVM).
Traffic accident takes place frequently at present, the problem of being greatly driver itself, main to wrap other than natural cause
It sees the mobile phone and diverts one's attention when including fatigue driving, receiving calls, drive, it is unexpected countless caused by the above reason.
At present although the vehicle-mounted camera in China has certain development, but focus primarily upon test car speed, target following,
The fields such as pedestrian detection, detection of obstacles drive not excessive research, in the past due to technology for the security civilization of driver
Limitation, how to judge driver whether in violation of rules and regulations traveling lack real-time high-efficiency judgment method.Face based on deep learning is known
Other technology can exercise supervision and judge to the real-time operation of driver, can effectively reduce driver's violation operation and cause
Traffic accident.
Invention content
The technical problem to be solved by the invention is to provide one kind judging driver by analyzing face variation in real time
Whether there is phenomena such as fatigue driving, and provides the vehicle-mounted face identification method based on deep learning reminded in time.
The technical solution adopted in the present invention is：A kind of vehicle-mounted face identification method based on deep learning, including it is as follows
Step：
1) image, structure Driver data's collection are obtained；
2) model is built, including：
(1) by convolutional neural networks and LSTM networks, extraction visual signature and semanteme are concentrated from Driver data respectively
Feature；
(2) visual signature of extraction and semantic feature are input in the LSTM networks with attention mechanism and are constituted
Characteristic processing model；
3) training characteristics handle model, and 60% image that Driver data concentrates is used to train, and 20% image is used
In verification, 20% image is for testing；
4) according to test result respectively to parameter Wz、Wr, W be finely adjusted, Optimal Experimental is as a result, by the feature after optimization
Reason model is transmitted to master control end；
5) warning lamp and camera are onboard installed, the operational circumstances of driver are uploaded into master control end, master control in real time
End processed judges whether driver has according to the characteristic processing model after optimization and drives in violation of rules and regulations, is driven in violation of rules and regulations when having, master control end hair
Go out signal, excite warning lamp, reminds driver's civilization traveling.
Step 1) obtains script including the use of the network picture based on python, and obtaining different drivers by internet schemes
Picture, to image making label, label indicates picture material, then summarizes in detail, collects as Driver data, in the image
Appearance includes：Driver's normal driving, bowing sees the mobile phone, looks about, chatting and fatigue driving.
(1) in step 2) includes：It is extracted and is driven using the conv5_3 layers of the VGG-19 networks in convolutional neural networks
14 × 14 × 512 dimension visual signatures of member's data set, obtain feature vector ai, visual information is generated by attention mechanism
Context vector zvt；The semantic feature of Driver data's collection is extracted by LSTM networks, and obtains semantic context vectors zst。
(2) step in step 2) includes：
(1) by visual information context vector zvtWith semantic context vectors zstIt can be more by affine transformation formation
The adequately context vector z of expression image informationt；
(2) the context vector z that will be obtainedtThe LSTM networks with attention mechanism are input to, analyze driver's
Behavior.
The characteristic processing model, it is as follows：
et=fatt(ai,ht-1)
zt=σ (Wz·[ht-1,xt])
rt=σ (Wr·[ht-1,xt])
Wherein, zvtIndicate visual information context vector, aiIndicate visual feature vector, αtIndicate weight, ztIt indicates up and down
Literary vector, xtIndicate the input at current time, htAnd ht-1The hiding layer state at current time and last moment is indicated respectively,For
The candidate state of current time hidden layer, Wz、Wr, W be parameter.
The vehicle-mounted face identification method based on deep learning of the present invention, is being obtained by vehicle-mounted camera on the run
Driver's real time information extracts face characteristic, detects whether it has by convolutional neural networks by analyzing face variation in real time
Violation operation carries out analysis comparison with data set, once it finds there is unlawful practice, such as find fatigue driving and answers electricity
Words, phenomena such as seeing the mobile phone, can send out alarm automatically, prevent the bad behavior of driver in time.What reduction traffic accident occurred can
It can property.
Description of the drawings
Fig. 1 is the flow chart of the vehicle-mounted face identification method the present invention is based on deep learning.
Specific implementation mode
The vehicle-mounted face identification method based on deep learning of the present invention is made in detail with reference to embodiment and attached drawing
Explanation.
As shown in Figure 1, the vehicle-mounted face identification method based on deep learning of the present invention, includes the following steps：
1) image is obtained, structure Driver data collects, including：
Script is obtained using the network picture based on python, different driver's images are obtained by internet, to image
Label is made, label indicates picture material, then summarizes in detail, collects as Driver data, and the picture material includes：It drives
The person's of sailing normal driving, bowing sees the mobile phone, looks about, chatting and fatigue driving.
2) model is built, including：
(1) in order to preferably differentiate the behavior of driver, the present invention extracts visual signature and semantic feature.Pass through volume
Product neural network and LSTM networks concentrate extraction visual signature and semantic feature from Driver data respectively；Including：Utilize convolution
14 × 14 × 512 dimension visual signatures of conv5_3 layers extraction Driver data's collection of the VGG-19 networks in neural network, obtain
Feature vector ai, visual information context vector z is generated by attention mechanismvt；Driver's number is extracted by LSTM networks
According to the semantic feature of collection, and obtain semantic context vectors zst。
(2) visual signature of extraction and semantic feature are input in the LSTM networks with attention mechanism and are constituted
Characteristic processing model；Including：
(2.1) by visual information context vector zvtWith semantic context vectors zstIt can be more by affine transformation formation
Add the context vector z of adequately expression image informationt；
(2.2) the context vector z that will be obtainedtThe LSTM networks with attention mechanism are input to, driver is analyzed
Behavior.Such as：Nozzle type changes when chat, the variation of eyes when tired.
LSTM (Long Short Term) is a kind of special RNN models, can learn long-term Dependency Specification.LSTM energy
It is enough by one forget door come it is selective remember or forget before information.Attention mechanism is current very popular
A kind of image procossing method, it is to distribute degree of concern by calculating the weight of image relevant range.The feature
Model is handled, it is as follows：
et=fatt(ai,ht-1)
zt=σ (Wz·[ht-1,xt])
rt=σ (Wr·[ht-1,xt])
Wherein, zvtIndicate visual information context vector, aiIndicate visual feature vector, αtIndicate weight, ztIt indicates up and down
Literary vector, xtIndicate the input at current time, htAnd ht-1The hiding layer state at current time and last moment is indicated respectively,For
The candidate state of current time hidden layer, Wz、Wr, W be parameter.
3) training characteristics handle model, and 60% image that Driver data concentrates is used to train, and 20% image is used
In verification, 20% image is for testing；
4) according to test result respectively to parameter Wz、Wr, W be finely adjusted, Optimal Experimental is as a result, by the feature after optimization
Reason model is transmitted to master control end；
5) warning lamp and camera are onboard installed, the operational circumstances of driver are uploaded into master control end, master control in real time
End processed judges whether driver has according to the characteristic processing model after optimization and drives in violation of rules and regulations, is driven in violation of rules and regulations when having, master control end hair
Go out signal, excite warning lamp, reminds driver's civilization traveling.
Claims (5)
1. a kind of vehicle-mounted face identification method based on deep learning, which is characterized in that include the following steps：
1) image, structure Driver data's collection are obtained；
2) model is built, including：
(1) by convolutional neural networks and LSTM networks, extraction visual signature and semantic feature are concentrated from Driver data respectively；
(2) visual signature of extraction and semantic feature are input to constitutive characteristic in the LSTM networks with attention mechanism
Handle model；
3) training characteristics handle model, and 60% image that Driver data concentrates is used to train, and 20% image is for testing
Card, 20% image is for testing；
4) according to test result respectively to parameter Wz、Wr, W be finely adjusted, Optimal Experimental is as a result, by the characteristic processing mould after optimization
Type is transmitted to master control end；
5) warning lamp and camera are onboard installed, the operational circumstances of driver are uploaded into master control end, master control end in real time
Judge whether driver has according to the characteristic processing model after optimization to drive in violation of rules and regulations, be driven in violation of rules and regulations when having, master control end sends out letter
Number, warning lamp is excited, driver's civilization traveling is reminded.
2. the vehicle-mounted face identification method according to claim 1 based on deep learning, which is characterized in that step 1) includes
Script is obtained using the network picture based on python, different driver's images are obtained by internet, to image making label,
Label indicates picture material in detail, then summarizes, and collects as Driver data, and the picture material includes：Driver is normal
It drives, bowing sees the mobile phone, looks about, chatting and fatigue driving.
3. the vehicle-mounted face identification method according to claim 1 based on deep learning, which is characterized in that in step 2)
(1) includes：Using the VGG-19 networks in convolutional neural networks conv5_3 layers extraction Driver data collection 14 × 14 ×
512 dimension visual signatures, obtain feature vector ai, visual information context vector z is generated by attention mechanismvt；Pass through
LSTM networks extract the semantic feature of Driver data's collection, and obtain semantic context vectors zst。
4. the vehicle-mounted face identification method according to claim 1 based on deep learning, which is characterized in that in step 2)
(2) step includes：
(1) by visual information context vector zvtWith semantic context vectors zstIt can be more fully by affine transformation formation
Express the context vector z of image informationt；
(2) the context vector z that will be obtainedtThe LSTM networks with attention mechanism are input to, the behavior of driver is analyzed.
5. the vehicle-mounted face identification method according to claim 4 based on deep learning, which is characterized in that the feature
Model is handled, it is as follows：
et=fatt(ai,ht-1)
zt=σ (Wz·[ht-1,xt])
rt=σ (Wr·[ht-1,xt])
Wherein, zvtIndicate visual information context vector, aiIndicate visual feature vector, αtIndicate weight, ztIndicate context to
Amount, xtIndicate the input at current time, htAnd ht-1The hiding layer state at current time and last moment is indicated respectively,It is current
The candidate state of moment hidden layer, Wz、Wr, W be parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810252216.5A CN108446645B (en) | 2018-03-26 | 2018-03-26 | Vehicle-mounted face recognition method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810252216.5A CN108446645B (en) | 2018-03-26 | 2018-03-26 | Vehicle-mounted face recognition method based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108446645A true CN108446645A (en) | 2018-08-24 |
| CN108446645B CN108446645B (en) | 2021-12-31 |
Family
ID=63197048
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810252216.5A Active CN108446645B (en) | 2018-03-26 | 2018-03-26 | Vehicle-mounted face recognition method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108446645B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109124625A (en) * | 2018-09-04 | 2019-01-04 | 大连理工大学 | A kind of driver fatigue state horizontal mipmap method |
| CN109614303A (en) * | 2018-12-05 | 2019-04-12 | 国网北京市电力公司 | A kind of violation information processing method and processing device |
| CN109886209A (en) * | 2019-02-25 | 2019-06-14 | 成都旷视金智科技有限公司 | Anomaly detection method and device, mobile unit |
| CN110059541A (en) * | 2019-02-28 | 2019-07-26 | 长江大学 | A kind of mobile phone usage behavior detection method and device in driving |
| CN110334614A (en) * | 2019-06-19 | 2019-10-15 | 腾讯科技（深圳）有限公司 | A kind of fatigue state method for early warning, device, equipment and storage medium |
| CN111738337A (en) * | 2020-06-23 | 2020-10-02 | 吉林大学 | Driver distraction state detection and identification method in mixed traffic environment |
| CN113378851A (en) * | 2020-02-25 | 2021-09-10 | 阿里巴巴集团控股有限公司 | Visual recognition method and device for image data, storage medium and processor |
| CN113688822A (en) * | 2021-09-07 | 2021-11-23 | 河南工业大学 | Time sequence attention mechanism scene image identification method |
| CN114267206A (en) * | 2021-12-28 | 2022-04-01 | 上汽大众汽车有限公司 | Security alarm method, security alarm device, security alarm system, and computer-readable storage medium |
| CN115071725A (en) * | 2022-08-02 | 2022-09-20 | 广东车卫士信息科技有限公司 | Driving behavior analysis method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170192956A1 (en) * | 2015-12-31 | 2017-07-06 | Google Inc. | Generating parse trees of text segments using neural networks |
| CN107038221A (en) * | 2017-03-22 | 2017-08-11 | 杭州电子科技大学 | A kind of video content description method guided based on semantic information |
| CN107330362A (en) * | 2017-05-25 | 2017-11-07 | 北京大学 | A kind of video classification methods based on space-time notice |
| CN107563498A (en) * | 2017-09-08 | 2018-01-09 | 中国石油大学(华东) | View-based access control model is combined the Image Description Methods and system of strategy with semantic notice |
| CN107608943A (en) * | 2017-09-08 | 2018-01-19 | 中国石油大学(华东) | Merge visual attention and the image method for generating captions and system of semantic notice |
-
2018
- 2018-03-26 CN CN201810252216.5A patent/CN108446645B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170192956A1 (en) * | 2015-12-31 | 2017-07-06 | Google Inc. | Generating parse trees of text segments using neural networks |
| CN107038221A (en) * | 2017-03-22 | 2017-08-11 | 杭州电子科技大学 | A kind of video content description method guided based on semantic information |
| CN107330362A (en) * | 2017-05-25 | 2017-11-07 | 北京大学 | A kind of video classification methods based on space-time notice |
| CN107563498A (en) * | 2017-09-08 | 2018-01-09 | 中国石油大学(华东) | View-based access control model is combined the Image Description Methods and system of strategy with semantic notice |
| CN107608943A (en) * | 2017-09-08 | 2018-01-19 | 中国石油大学(华东) | Merge visual attention and the image method for generating captions and system of semantic notice |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109124625A (en) * | 2018-09-04 | 2019-01-04 | 大连理工大学 | A kind of driver fatigue state horizontal mipmap method |
| CN109124625B (en) * | 2018-09-04 | 2021-07-20 | 大连理工大学 | Driver fatigue state level grading method |
| CN109614303A (en) * | 2018-12-05 | 2019-04-12 | 国网北京市电力公司 | A kind of violation information processing method and processing device |
| CN109886209A (en) * | 2019-02-25 | 2019-06-14 | 成都旷视金智科技有限公司 | Anomaly detection method and device, mobile unit |
| CN110059541A (en) * | 2019-02-28 | 2019-07-26 | 长江大学 | A kind of mobile phone usage behavior detection method and device in driving |
| CN110334614A (en) * | 2019-06-19 | 2019-10-15 | 腾讯科技（深圳）有限公司 | A kind of fatigue state method for early warning, device, equipment and storage medium |
| CN113378851A (en) * | 2020-02-25 | 2021-09-10 | 阿里巴巴集团控股有限公司 | Visual recognition method and device for image data, storage medium and processor |
| CN111738337A (en) * | 2020-06-23 | 2020-10-02 | 吉林大学 | Driver distraction state detection and identification method in mixed traffic environment |
| CN111738337B (en) * | 2020-06-23 | 2022-03-25 | 吉林大学 | Driver distraction state detection and identification method in mixed traffic environment |
| CN113688822A (en) * | 2021-09-07 | 2021-11-23 | 河南工业大学 | Time sequence attention mechanism scene image identification method |
| CN114267206A (en) * | 2021-12-28 | 2022-04-01 | 上汽大众汽车有限公司 | Security alarm method, security alarm device, security alarm system, and computer-readable storage medium |
| CN115071725A (en) * | 2022-08-02 | 2022-09-20 | 广东车卫士信息科技有限公司 | Driving behavior analysis method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108446645B (en) | 2021-12-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108446645A (en) | Vehicle-mounted face identification method based on deep learning | |
| Ding et al. | Semantic segmentation with context encoding and multi-path decoding | |
| CN104637246B (en) | Driver multi-behavior early warning system and danger evaluation method | |
| CN109460699B (en) | Driver safety belt wearing identification method based on deep learning | |
| Anagnostopoulos et al. | A license plate-recognition algorithm for intelligent transportation system applications | |
| CN109194612B (en) | Network attack detection method based on deep belief network and SVM | |
| Zhang et al. | Multi-task SE-network for image splicing localization | |
| CN111680613B (en) | Method for detecting falling behavior of escalator passengers in real time | |
| CN110378314A (en) | A kind of human face region image archiving method, device, electronic equipment and storage medium | |
| CN110210382A (en) | A kind of face method for detecting fatigue driving and device based on space-time characteristic identification | |
| CN103886279B (en) | Real-time rider detection using synthetic training data | |
| CN109460704A (en) | A kind of fatigue detection method based on deep learning, system and computer equipment | |
| Potdar et al. | A convolutional neural network based live object recognition system as blind aid | |
| CN107944398A (en) | Based on depth characteristic association list diagram image set face identification method, device and medium | |
| CN109670457A (en) | A kind of driver status recognition methods and device | |
| CN105574489A (en) | Layered stack based violent group behavior detection method | |
| Yan et al. | Recognizing driver inattention by convolutional neural networks | |
| CN114155512A (en) | Fatigue detection method and system based on multi-feature fusion of 3D convolutional network | |
| CN108960175A (en) | A kind of licence plate recognition method based on deep learning | |
| Liu et al. | Development of face recognition system based on PCA and LBP for intelligent anti-theft doors | |
| CN108108651B (en) | Method and system for detecting driver non-attentive driving based on video face analysis | |
| Fang et al. | Traffic police gesture recognition by pose graph convolutional networks | |
| CN111860117A (en) | Human behavior recognition method based on deep learning | |
| CN114429126A (en) | True and false message identification method based on reinforcement learning and affair knowledge graph | |
| CN111753684B (en) | Pedestrian re-recognition method using target posture for generation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |