CN113591615A - Multi-model-based driver smoking detection method - Google Patents
Multi-model-based driver smoking detection method Download PDFInfo
- Publication number
- CN113591615A CN113591615A CN202110795174.1A CN202110795174A CN113591615A CN 113591615 A CN113591615 A CN 113591615A CN 202110795174 A CN202110795174 A CN 202110795174A CN 113591615 A CN113591615 A CN 113591615A
- Authority
- CN
- China
- Prior art keywords
- smoking
- driver
- image
- cigarette
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000000391 smoking effect Effects 0.000 title claims abstract description 79
- 238000001514 detection method Methods 0.000 title claims abstract description 35
- 235000019504 cigarettes Nutrition 0.000 claims abstract description 44
- 238000000034 method Methods 0.000 claims abstract description 18
- 239000000779 smoke Substances 0.000 claims description 9
- 238000013135 deep learning Methods 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 5
- 230000018109 developmental process Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a multi-model-based method for detecting smoking of a driver, which comprises the steps of acquiring an image of a smoking detection area of the driver, inputting the image of the smoking detection area of the driver into a smoking gesture recognition model and a smoking area recognition model to acquire a probability map of smoking gesture recognition and a probability map of smoking recognition, analyzing the results of the probability map of smoking gesture recognition and the probability map of smoking recognition, and detecting that the driver smokes when a smoking gesture and a cigarette are recognized at the same time. According to the multi-model-based driver smoking detection method, the false alarm rate of smoking detection can be reduced by detecting the smoking gesture and detecting the cigarette, and the cigarette detection adopts case segmentation, so that for the situation of a slender target such as a cigarette, the case segmentation avoids the influence caused by background pixels, and the stability of final detection is improved.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a multi-model-based method for detecting smoking of a driver.
Background
With the development of economy and automobile technology, more and more automobiles are arranged around the world, and the traffic is developed more and more. But the safety precaution consciousness of people is not correspondingly enhanced, which causes more frequent traffic accidents; so that the safety driving assisting technology is more and more emphasized. The smoking behavior of the driver can lead the driver to drive distractedly, so the smoking detection technology is an important component of the safe driving technology. Conventional smoking detection methods include applying conventional machine learning methods and applying deep learning methods. The traditional machine learning method is applied to extract LBP characteristics, color characteristics and mouth region characteristics of the smoke of the driver with irregular smoke contour for fusion, and then SVM is used for classifying whether smoke is pumped again or not. The method is characterized in that the method is applied to carry out target detection on cigarettes through a target detection model to obtain the classification and position information of the cigarettes, and the method is also easy to misjudge due to the fact that the cigarette area occupies a small smoking area.
Disclosure of Invention
The invention aims to provide a multi-model-based method for detecting the smoking of a driver, which can reduce the false alarm rate of smoking detection.
The invention provides a multi-model-based method for detecting smoking of a driver, which comprises the following steps:
acquiring an image of a smoking detection area of a driver;
inputting the image of the smoking detection area of the driver into a smoking gesture recognition model and a cigarette area recognition model to obtain a probability chart of smoking gesture recognition and a probability chart of cigarette recognition;
and analyzing the results of the probability chart of the smoking gesture recognition and the probability chart of the cigarette recognition, and detecting that the driver smokes when the smoking gesture and the cigarette are recognized at the same time.
According to the multi-model-based method for detecting the smoking of the driver, by detecting the smoking gesture and detecting the cigarette, the false alarm rate of smoking detection can be reduced, and the cigarette detection adopts case segmentation, namely the probability that each pixel of an image belongs to the cigarette is identified, and for the case of a slender target of the cigarette, the case segmentation avoids the influence caused by background pixels, so that the stability of final detection is improved; .
Drawings
FIG. 1 is a schematic flow chart of a method for detecting a driver's smoking based on multiple models according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a smoking gesture recognition model according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a cigarette region identification model according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a sample image generation process including a smoking gesture according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a flowchart of a process for generating an image of a sample of a cigarette area according to an embodiment of the present invention;
fig. 6 is a schematic diagram of an example of cigarette detection according to an embodiment of the present invention.
Detailed Description
As shown in fig. 1, a method for detecting a smoking of a driver based on multiple models according to an embodiment of the present invention includes acquiring an image through an image capturing device, performing full-scale face detection on the image to obtain face regions, calculating the area of each face region, extracting a face image with the largest area, inputting the face image into a face alignment model, extracting a series of face images with face feature points, extracting a region image around the mouth of the driver according to the face image with the face feature points, acquiring a smoking detection region image of the driver according to the extracted region image around the mouth of the driver, inputting the smoking detection region image of the driver into a smoking gesture recognition model and a smoking region recognition model to acquire a probability map of a smoking gesture recognition and a probability map of a smoking recognition, and analyzing the results of the probability map of the smoking gesture recognition and the probability map of the smoking recognition, when the smoking gesture and the cigarette are recognized at the same time, it is detected that the driver is smoking.
Inputting the image of the smoking detection area of the driver into the smoking gesture recognition model and the cigarette area recognition model to obtain the probability chart of the smoking gesture recognition and the probability chart of the cigarette recognition specifically as follows:
the smoking gesture recognition model is composed of a convolution module, a MobileNetV2 module and a full connection module, and a small area image for smoking detection is firstly zoomed to
Wherein
And
represents height and width; and then, finally obtaining a probability chart of the smoking gesture recognition through a series of operations of a convolution module, a MobileNet V2 module, a full connection module and softmax.
The cigarette region identification model is composed of a convolution module, a MobileNetV2 module and a MobileNetV2 funnel module, and a large region image for smoking detection is firstly zoomed to
(ii) a And then, calculating through a series of convolution modules, a MobileNet V2 module and a MobileNet V2 funnel module to finally obtain a probability map of cigarette identification.
The smoking gesture recognition model is obtained through training and learning of a deep learning framework, a large number of samples and sample marks are needed for training and learning, and therefore the process of generating the samples needed by the training and smoking gesture recognition model is specifically as shown in fig. 4, firstly, a human face image, coordinates of human face characteristic points, a smoking gesture containing smoke and a smoke-containing anchor point are obtained; calculating the mean value and the variance of pixels C and D in cheek areas on two sides of the face according to the characteristic points of the face:
，
wherein the content of the first and second substances,
、
respectively, the pixel values and the number of pixels of the region C, D. Similarly, calculate the mean of the smoking gesture
Sum variance
The following are:
，
then, according to the four formulas, the smoking gesture is normalized so that the variance and mean are equal to the area C, D, then
Wherein, in the step (A),
a pixel value representing the normalization of the smoke gesture. Then, the standardized smoking gesture is superposed on the face image through scaling rotation, and a smoke-containing anchor point meeting the smoking gesture is in the mouth area; then, Gaussian filtering is carried out on the edge of the smoking gesture, and the sawtooth is eliminated; and finally obtaining a sample containing the smoking gesture.
The cigarette area recognition model is obtained through training and learning of a deep learning framework, and the process of synthesizing samples required by training the cigarette area recognition model is specifically as shown in fig. 5, firstly, a human face image containing feature points, a cigarette and a cigarette-containing anchor point mark are obtained; then, after the cigarette image is zoomed and rotated, the cigarette image is superposed on the face image, and the requirement that the cigarette-containing anchor point is in the mouth area is met; then Gaussian filtering is carried out on the edges of the cigarettes; finally, a face image containing cigarettes and cigarette area marks are obtained. The sample image containing the smoking gesture and the sample image of the cigarette area are generated without manual collection and marking, so that the time development cost is greatly reduced. Fig. 6 is an example of detection of the cigarette area recognition model, in which columns 2, 4, 6, and 8 are input images, and columns 1, 3, 5, and 7 are probability maps of cigarette recognition output by the cigarette area recognition model.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.
Claims (7)
1. A method for detecting the smoking of a driver based on multiple models is characterized by comprising the following steps:
acquiring an image of a smoking detection area of a driver;
inputting the image of the smoking detection area of the driver into a smoking gesture recognition model and a cigarette area recognition model to obtain a probability chart of smoking gesture recognition and a probability chart of cigarette recognition;
and analyzing the results of the probability chart of the smoking gesture recognition and the probability chart of the cigarette recognition, and detecting that the driver smokes when the smoking gesture and the cigarette are recognized at the same time.
2. The multi-model based driver smoking detection method according to claim 1, wherein obtaining the driver smoking detection area image comprises:
acquiring a face image with face characteristic points;
extracting an image of a region around the mouth of the driver according to the face image with the face characteristic points;
and acquiring an image of the smoking detection area of the driver according to the extracted image of the area around the mouth of the driver.
3. The multi-model based driver smoking detection method of claim 2, wherein obtaining a face image having face feature points comprises:
acquiring an image through a camera device;
carrying out full-image face detection on the image to obtain a face region;
calculating the area of each face region, and extracting a face image with the largest area;
and inputting the face image into a face alignment model, and extracting a series of face images with face characteristic points.
4. The multi-model based driver smoke detection method according to claim 1, wherein the smoke gesture recognition model is composed of a convolution module, a MobileNetV2 module, and a fully connected module.
5. The multi-model based driver smoking detection method of claim 1, wherein the cigarette zone identification model is comprised of a convolution module, a MobileNetV2 module, and a MobileNetV2 funnel module.
6. The method for detecting the smoking of the driver based on the multiple models as claimed in claim 4, wherein the smoking gesture recognition model is obtained through training and learning of a deep learning framework, and is obtained through training and supervision of a sample image set containing a smoking gesture.
7. The method for detecting the smoking of the driver based on the multiple models as claimed in claim 5, wherein the cigarette region identification model is obtained through training learning of a deep learning framework, and is obtained through training supervision of an image set containing cigarette samples.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110795174.1A CN113591615A (en) | 2021-07-14 | 2021-07-14 | Multi-model-based driver smoking detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110795174.1A CN113591615A (en) | 2021-07-14 | 2021-07-14 | Multi-model-based driver smoking detection method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113591615A true CN113591615A (en) | 2021-11-02 |
Family
ID=78247378
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110795174.1A Pending CN113591615A (en) | 2021-07-14 | 2021-07-14 | Multi-model-based driver smoking detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113591615A (en) |
Citations (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105260703A (en) * | 2015-09-15 | 2016-01-20 | 西安邦威电子科技有限公司 | Detection method suitable for smoking behavior of driver under multiple postures |
| CN106682603A (en) * | 2016-12-19 | 2017-05-17 | 陕西科技大学 | Real time driver fatigue warning system based on multi-source information fusion |
| CN106709420A (en) * | 2016-11-21 | 2017-05-24 | 厦门瑞为信息技术有限公司 | Method for monitoring driving behaviors of driver of commercial vehicle |
| CN107977607A (en) * | 2017-11-20 | 2018-05-01 | 安徽大学 | A kind of fatigue driving monitoring method based on machine vision |
| CN109872160A (en) * | 2019-02-01 | 2019-06-11 | 广州逗号智能科技有限公司 | Electric paying method and device |
| CN110705357A (en) * | 2019-09-02 | 2020-01-17 | 深圳中兴网信科技有限公司 | Face recognition method and face recognition device |
| CN110728234A (en) * | 2019-10-12 | 2020-01-24 | 爱驰汽车有限公司 | Driver face recognition method, system, device and medium |
| CN110738186A (en) * | 2019-10-23 | 2020-01-31 | 德瑞姆创新科技(深圳)有限公司 | driver smoking detection method and system based on computer vision technology |
| CN110837815A (en) * | 2019-11-15 | 2020-02-25 | 济宁学院 | Driver state monitoring method based on convolutional neural network |
| CN111160288A (en) * | 2019-12-31 | 2020-05-15 | 北京奇艺世纪科技有限公司 | Gesture key point detection method and device, computer equipment and storage medium |
| CN111178341A (en) * | 2020-04-10 | 2020-05-19 | 支付宝(杭州)信息技术有限公司 | Living body detection method, device and equipment |
| CN111325058A (en) * | 2018-12-14 | 2020-06-23 | 长沙智能驾驶研究院有限公司 | Driving behavior detection method, device and system and storage medium |
| CN210983433U (en) * | 2019-10-23 | 2020-07-10 | 德瑞姆创新科技(深圳)有限公司 | Driver smoking detection system based on computer vision technology |
| CN111488841A (en) * | 2020-04-15 | 2020-08-04 | 上海眼控科技股份有限公司 | Smoking behavior detection method and device, computer equipment and storage medium |
| CN111753674A (en) * | 2020-06-05 | 2020-10-09 | 广东海洋大学 | Fatigue driving detection and identification method based on deep learning |
| CN111860254A (en) * | 2020-07-10 | 2020-10-30 | 东莞正扬电子机械有限公司 | Driver abnormal behavior detection method and device, storage medium and equipment |
| CN112052815A (en) * | 2020-09-14 | 2020-12-08 | 北京易华录信息技术股份有限公司 | Behavior detection method and device and electronic equipment |
| CN112188088A (en) * | 2020-09-21 | 2021-01-05 | 厦门大学 | Underwater self-photographing system |
| CN112183482A (en) * | 2020-10-29 | 2021-01-05 | 苏州市职业大学 | Dangerous driving behavior recognition method, device and system and readable storage medium |
| CN112183356A (en) * | 2020-09-28 | 2021-01-05 | 广州市几米物联科技有限公司 | Driving behavior detection method and device and readable storage medium |
| CN112241647A (en) * | 2019-07-16 | 2021-01-19 | 青岛点之云智能科技有限公司 | Dangerous driving behavior early warning device and method based on depth camera |
| CN112380976A (en) * | 2020-11-12 | 2021-02-19 | 华东师范大学 | Gesture recognition system and method based on neural network visual touch sensor fusion |
| CN112380977A (en) * | 2020-11-12 | 2021-02-19 | 深兰人工智能芯片研究院（江苏）有限公司 | Smoking behavior detection method and device |
| CN112464810A (en) * | 2020-11-25 | 2021-03-09 | 创新奇智(合肥)科技有限公司 | Smoking behavior detection method and device based on attention map |
| CN113033374A (en) * | 2021-03-22 | 2021-06-25 | 开放智能机器（上海）有限公司 | Artificial intelligence dangerous behavior identification method and device, electronic equipment and storage medium |
-
2021
- 2021-07-14 CN CN202110795174.1A patent/CN113591615A/en active Pending
Patent Citations (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105260703A (en) * | 2015-09-15 | 2016-01-20 | 西安邦威电子科技有限公司 | Detection method suitable for smoking behavior of driver under multiple postures |
| CN106709420A (en) * | 2016-11-21 | 2017-05-24 | 厦门瑞为信息技术有限公司 | Method for monitoring driving behaviors of driver of commercial vehicle |
| CN106682603A (en) * | 2016-12-19 | 2017-05-17 | 陕西科技大学 | Real time driver fatigue warning system based on multi-source information fusion |
| CN107977607A (en) * | 2017-11-20 | 2018-05-01 | 安徽大学 | A kind of fatigue driving monitoring method based on machine vision |
| CN111325058A (en) * | 2018-12-14 | 2020-06-23 | 长沙智能驾驶研究院有限公司 | Driving behavior detection method, device and system and storage medium |
| CN109872160A (en) * | 2019-02-01 | 2019-06-11 | 广州逗号智能科技有限公司 | Electric paying method and device |
| CN112241647A (en) * | 2019-07-16 | 2021-01-19 | 青岛点之云智能科技有限公司 | Dangerous driving behavior early warning device and method based on depth camera |
| CN110705357A (en) * | 2019-09-02 | 2020-01-17 | 深圳中兴网信科技有限公司 | Face recognition method and face recognition device |
| CN110728234A (en) * | 2019-10-12 | 2020-01-24 | 爱驰汽车有限公司 | Driver face recognition method, system, device and medium |
| CN210983433U (en) * | 2019-10-23 | 2020-07-10 | 德瑞姆创新科技(深圳)有限公司 | Driver smoking detection system based on computer vision technology |
| CN110738186A (en) * | 2019-10-23 | 2020-01-31 | 德瑞姆创新科技(深圳)有限公司 | driver smoking detection method and system based on computer vision technology |
| CN110837815A (en) * | 2019-11-15 | 2020-02-25 | 济宁学院 | Driver state monitoring method based on convolutional neural network |
| CN111160288A (en) * | 2019-12-31 | 2020-05-15 | 北京奇艺世纪科技有限公司 | Gesture key point detection method and device, computer equipment and storage medium |
| CN111178341A (en) * | 2020-04-10 | 2020-05-19 | 支付宝(杭州)信息技术有限公司 | Living body detection method, device and equipment |
| CN111488841A (en) * | 2020-04-15 | 2020-08-04 | 上海眼控科技股份有限公司 | Smoking behavior detection method and device, computer equipment and storage medium |
| CN111753674A (en) * | 2020-06-05 | 2020-10-09 | 广东海洋大学 | Fatigue driving detection and identification method based on deep learning |
| CN111860254A (en) * | 2020-07-10 | 2020-10-30 | 东莞正扬电子机械有限公司 | Driver abnormal behavior detection method and device, storage medium and equipment |
| CN112052815A (en) * | 2020-09-14 | 2020-12-08 | 北京易华录信息技术股份有限公司 | Behavior detection method and device and electronic equipment |
| CN112188088A (en) * | 2020-09-21 | 2021-01-05 | 厦门大学 | Underwater self-photographing system |
| CN112183356A (en) * | 2020-09-28 | 2021-01-05 | 广州市几米物联科技有限公司 | Driving behavior detection method and device and readable storage medium |
| CN112183482A (en) * | 2020-10-29 | 2021-01-05 | 苏州市职业大学 | Dangerous driving behavior recognition method, device and system and readable storage medium |
| CN112380976A (en) * | 2020-11-12 | 2021-02-19 | 华东师范大学 | Gesture recognition system and method based on neural network visual touch sensor fusion |
| CN112380977A (en) * | 2020-11-12 | 2021-02-19 | 深兰人工智能芯片研究院（江苏）有限公司 | Smoking behavior detection method and device |
| CN112464810A (en) * | 2020-11-25 | 2021-03-09 | 创新奇智(合肥)科技有限公司 | Smoking behavior detection method and device based on attention map |
| CN113033374A (en) * | 2021-03-22 | 2021-06-25 | 开放智能机器（上海）有限公司 | Artificial intelligence dangerous behavior identification method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104299009B (en) | License plate character recognition method based on multi-feature fusion | |
| CN111563417B (en) | Pyramid structure convolutional neural network-based facial expression recognition method | |
| CN105488453B (en) | A kind of driver based on image procossing does not fasten the safety belt detection recognition method | |
| CN110119676A (en) | A kind of Driver Fatigue Detection neural network based | |
| CN109460704B (en) | Fatigue detection method and system based on deep learning and computer equipment | |
| CN108710865A (en) | A kind of driver's anomaly detection method based on neural network | |
| CN111401250A (en) | Chinese lip language identification method and device based on hybrid convolutional neural network | |
| CN112183238B (en) | Remote education attention detection method and system | |
| CN104504856A (en) | Fatigue driving detection method based on Kinect and face recognition | |
| CN106991421A (en) | A kind of ID card information extraction system | |
| CN113076861B (en) | Bird fine-granularity identification method based on second-order features | |
| CN112906481A (en) | Method for realizing forest fire detection based on unmanned aerial vehicle | |
| WO2019140641A1 (en) | Information processing method and system, cloud processing device and computer program product | |
| Samantaray et al. | Optical character recognition (ocr) based vehicle's license plate recognition system using python and opencv | |
| CN115240210A (en) | System and method for auxiliary exercise of handwritten Chinese characters | |
| CN113076860B (en) | Bird detection system under field scene | |
| CN109977862A (en) | A kind of recognition methods of parking stall limiter | |
| CN104463238A (en) | License plate recognition method and system | |
| CN113591615A (en) | Multi-model-based driver smoking detection method | |
| CN112069898A (en) | Method and device for recognizing human face group attribute based on transfer learning | |
| JP2893948B2 (en) | License plate recognition device | |
| Navada et al. | Design of Mobile Application for Assisting Color Blind People to Identify Information on Sign Boards. | |
| Rahman et al. | Real time road sign recognition system using artificial neural networks for Bengali textual information box | |
| CN113743226A (en) | Daytime headlamp language recognition and early warning method and system | |
| CN106920398A (en) | A kind of intelligent vehicle license plate recognition system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |