CN114937302A - Smoking identification method, device and equipment and computer readable storage medium - Google Patents
Smoking identification method, device and equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN114937302A CN114937302A CN202210602059.2A CN202210602059A CN114937302A CN 114937302 A CN114937302 A CN 114937302A CN 202210602059 A CN202210602059 A CN 202210602059A CN 114937302 A CN114937302 A CN 114937302A
- Authority
- CN
- China
- Prior art keywords
- human body
- target
- body image
- head
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 75
- 230000000391 smoking effect Effects 0.000 title claims abstract description 48
- 235000019504 cigarettes Nutrition 0.000 claims abstract description 29
- 238000001514 detection method Methods 0.000 claims description 40
- 230000003321 amplification Effects 0.000 claims description 20
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 20
- 239000000779 smoke Substances 0.000 claims description 18
- 238000004422 calculation algorithm Methods 0.000 claims description 14
- 238000007781 pre-processing Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 11
- 238000010586 diagram Methods 0.000 claims description 10
- 230000001960 triggered effect Effects 0.000 claims description 2
- 238000012545 processing Methods 0.000 abstract description 7
- 230000006399 behavior Effects 0.000 abstract description 6
- 238000004364 calculation method Methods 0.000 abstract description 5
- 238000012549 training Methods 0.000 description 23
- 230000008569 process Effects 0.000 description 14
- 238000013527 convolutional neural network Methods 0.000 description 6
- 230000000694 effects Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 5
- 230000004913 activation Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B31/00—Predictive alarm systems characterised by extrapolation or other computation using updated historic data
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Emergency Management (AREA)
- Business, Economics & Management (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a smoking identification method, a device, equipment and a computer readable storage medium, belongs to the field of image processing, and is used for identifying smoking behavior. The problem that the calculation power demand is big and the inefficiency brought to the whole image is directly carried out cigarette discernment is considered, and cigarette must be located human head region when considering smoking, therefore this application can be after determining the human image in the picture of waiting to detect, enlarge the head region in the human image, then carry out cigarette discernment and report to the police in enlarged head region, reduced the calculation power demand on the one hand, on the other hand has promoted recognition efficiency.
Description
Technical Field
The invention relates to the field of image processing, in particular to a smoking identification method, and also relates to a smoking identification device, equipment and a computer readable storage medium.
Background
The danger caused by smoking in places such as gas stations and automobile cabs is very high, and the explosion of the gas stations and automobile accidents caused by smoking also occur frequently, so a method for identifying smoking behaviors is urgently needed to solve the problem.
Therefore, how to provide a solution to the above technical problem is a problem that needs to be solved by those skilled in the art.
Disclosure of Invention
The invention aims to provide a smoking identification method, which can amplify a head area in a human body image after the human body image in a picture to be detected is determined, and then identify cigarettes in the amplified head area and give an alarm, so that the calculation force requirement is reduced, and the identification efficiency is improved; another object of the present invention is to provide a smoking identification device, a device and a computer readable storage medium, which can amplify a head region in a human body image after determining the human body image in a picture to be detected, and then perform cigarette identification and alarm in the amplified head region, thereby reducing computational demands and improving identification efficiency.
In order to solve the technical problem, the invention provides a smoking identification method, which comprises the following steps:
acquiring a picture to be detected through a camera;
determining a human body image in the picture to be detected through a target detection model;
amplifying the head region in the human body image through a target amplification network;
judging whether cigarettes exist in the enlarged head area;
if the alarm exists, the alarm is controlled to give an alarm.
Preferably, the amplifying the head region in the human body image through the target amplifying network specifically includes:
determining a head region in the human body image;
pre-processing the head region;
expanding the number of channels of the preprocessed header region through a first convolution network;
amplifying the head region passing through the first convolutional network by a preset algorithm;
expanding the number of channels of the enlarged head region through a second convolution network;
and reducing the number of channels of the head area to the original number through a third convolution network so as to compare the original number with the real image to verify the prediction result.
Preferably, the determining, by the target detection model, the human body image in the picture to be detected specifically includes:
preprocessing the picture to be detected;
acquiring a feature map of the preprocessed picture to be detected;
determining a target recommendation area corresponding to a preset target object type from the characteristic diagram;
cutting the target recommendation area on the feature map and unifying the size;
and determining a human body image from each target recommendation area.
Preferably, the determining of the human body image from each of the target recommendation regions specifically includes:
determining the object type of each target recommendation area;
judging whether a human body exists in the object type;
if not, executing the step of acquiring the picture to be detected through the camera;
and if so, executing the step of preprocessing the human body image.
Preferably, after the human body image in the picture to be detected is determined by the target detection model and before the head region in the human body image is amplified by the target amplification network, the method for identifying smoking further includes:
judging whether a head exists in the human body image or not;
if not, executing the step of acquiring the picture to be detected through the camera;
if yes, the step of amplifying the head area in the human body image through the target amplifying network is executed.
Preferably, after determining whether the head exists in the human body image and before the head region in the human body image is enlarged by the target enlargement network, the method for identifying smoking further includes:
if the head exists, judging whether a shelter exists in the head area;
if the sheltering object exists, the step of acquiring the picture to be detected through the camera is executed;
and if no occlusion exists, executing the step of amplifying the head area in the human body image through the target amplifying network.
Preferably, before the head region in the human body image is enlarged through the target enlarging network, the method for identifying smoking further comprises:
judging whether the head region in the human body image is matched with the head image in the tracking library or not;
if the pictures are matched, the step of acquiring the pictures to be detected through the camera is executed;
and if the head regions do not coincide with each other, adding the current head region into the tracking library and executing the step of amplifying the head region in the human body image through a target amplification network.
In order to solve the above technical problem, the present invention further provides a smoking identification device, including:
the acquisition module is used for acquiring the picture to be detected through the camera;
the target detection module is used for determining the human body image in the picture to be detected through a target detection model;
the target amplification module is used for amplifying the head area in the human body image through a target amplification network;
the judging module is used for judging whether cigarettes exist in the amplified head area or not, and if yes, the alarming module is triggered;
and the alarm module is used for controlling an alarm to give an alarm.
In order to solve the above technical problem, the present invention further provides a smoking identification device, including:
a memory for storing a computer program;
a processor for implementing the steps of the method of smoke recognition as described above when executing the computer program.
In order to solve the above technical problem, the present invention further provides a computer-readable storage medium, having a computer program stored thereon, where the computer program, when executed by a processor, implements the steps of the smoke recognition method as described above.
The invention provides a smoking identification method, which considers the problems of high computational power demand and low efficiency caused by directly identifying cigarettes for a complete image and considers that the cigarettes are inevitably positioned in a human head area during smoking, so that the method can amplify the head area in a human image after the human image in a picture to be detected is determined, and then identifies the cigarettes in the amplified head area and gives an alarm, thereby reducing the computational power demand and improving the identification efficiency.
The invention also provides a smoking identification device, equipment and a computer readable storage medium, which have the same beneficial effects as the smoking identification method.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed in the prior art and the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic flow chart of a smoking identification method according to the present invention;
FIG. 2 is a flow chart of a training process for a target amplification network according to the present invention;
FIG. 3 is a flow chart of a training process of a target detection network according to the present invention;
fig. 4 is a schematic structural diagram of a smoke recognition device provided in the present invention;
fig. 5 is a schematic structural diagram of a smoking identification device provided by the present invention.
Detailed Description
The core of the invention is to provide a smoking identification method, the application can amplify the head area in the human body image after determining the human body image in the picture to be detected, and then carry out cigarette identification and alarm in the amplified head area, so that the computational power requirement is reduced on one hand, and the identification efficiency is improved on the other hand; another core of the invention is to provide a smoking recognition device, a device and a computer readable storage medium, which can amplify a head region in a human body image after determining the human body image in a picture to be detected, and then perform cigarette recognition and alarm in the amplified head region, thereby reducing computational power requirements and improving recognition efficiency.
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a schematic flow chart of a smoke recognition method provided in the present invention, the smoke recognition method includes:
s101: acquiring a picture to be detected through a camera;
specifically, in the prior art, an object detection algorithm based on deep learning has been researched more importantly in the last years, and with the development of artificial intelligence and the development of some object detection algorithms based on a deep Convolutional Neural network (Convolutional Neural network) network, some algorithms are distinguished in the field of computer vision, and at present, the identification of smoking behavior can be generally performed through human body posture detection or a direct object detection algorithm, for example, a classical object detection network SDD (Single Shot multi box Detector), an RCNN (Region-Convolutional Neural network), a fast RCNN (fast Region Convolutional Neural network), a YOLO (You Only Look Once) series, and the like. The detection method based on deep learning has a good effect, and the detection performance of the detection method based on deep convolutional neural network is better and better along with the improvement of the performance of the deep convolutional neural network architecture; however, the traditional character detection method based on deep learning has huge network models, high computational power and poor effect on small targets, while most of the existing gas station detection devices are edge terminal devices, and the edge terminals are limited by the computational power and are difficult to be directly deployed on the edge terminal devices. In view of the technical problems in the background art, the prior art generally performs cigarette identification on a complete image directly, so that the computational power requirement is high and the efficiency is low, and in view of the fact that cigarettes are inevitably located in a human head region during smoking, the application aims to directly amplify the head region in the human image and perform cigarette identification after determining the human image in the picture to be detected, thereby being beneficial to reducing the image identification area, reducing the computational power requirement and improving the efficiency.
Specifically, in view of the above object, in the embodiment of the present invention, a to-be-detected picture needs to be acquired through a camera first, where a video stream of the camera may be acquired, and then pictures are acquired from the video stream, and each frame of picture may be sequentially identified.
The application scenarios in the embodiment of the present invention may be various, for example, the application scenarios may be a gas station or an automobile cab, and the embodiment of the present invention is not limited herein.
S102: determining a human body image in the picture to be detected through a target detection model;
specifically, considering that the category of the object in the picture to be detected is uncertain, and the smoking behavior to be focused on in the embodiment of the present invention is also executed by the human body, the human body image in the picture to be detected can be determined by the target detection model in the embodiment of the present invention, and is used as the data base of the subsequent step, so as to identify the smoking behavior.
The target detection model may be of various types, such as fast RCNN, and the like, and the embodiment of the present invention is not limited herein.
S103: amplifying a head region in the human body image through a target amplifying network;
specifically, considering that the smoking behavior is generated by the mouth, and the mouth is located on the head of the human body, in order to reduce the range of image recognition, the embodiment of the present invention separately extracts the head region in the human body image, and further, in order to more clearly and accurately recognize the cigarettes in the head region, the embodiment of the present invention may further enlarge the head region, and then use the enlarged head region as the data base of the subsequent step.
The target amplifying network may be of various types, and the embodiment of the present invention is not limited herein.
S104: judging whether cigarettes exist in the enlarged head area;
specifically, considering that once the cigarette is found in the head region, the human body corresponding to the head region is smoking at a high probability, the embodiment of the present invention may perform the cigarette identification on the enlarged head region, and trigger the subsequent control action according to the determination result.
S105: if the alarm exists, the alarm is controlled to give an alarm.
Specifically, after cigarettes exist in the head area, the fact that the human body is smoking can be directly judged, and the judgment accuracy is high, so that in the situation, people who are smoking can be warned conveniently, and the alarm can be controlled to give an alarm.
The alarm can selectively give an alarm to the smoking personnel and/or the management personnel according to different setting positions and setting forms of the alarm, so that the safety problem caused by smoking can be better prevented.
The invention provides a smoking identification method, which considers the problems of high computational power demand and low efficiency caused by directly identifying cigarettes for a complete image and considers that the cigarettes are inevitably positioned in a human head area during smoking, so that the method can amplify the head area in a human image after the human image in a picture to be detected is determined, and then identifies the cigarettes in the amplified head area and gives an alarm, thereby reducing the computational power demand and improving the identification efficiency.
For better explaining the embodiment of the present invention, please refer to fig. 2, fig. 2 is a training flowchart of a target amplification network provided by the present invention, and on the basis of the above embodiment:
as a preferred embodiment, the enlarging the head region in the human body image by the target enlarging network specifically comprises:
determining a head region in the human body image;
preprocessing a head region;
expanding the number of channels of the preprocessed header region through a first convolution network;
amplifying the head region passing through the first convolution network through a preset algorithm;
expanding the number of channels of the amplified header region by a second convolutional network;
and reducing the number of channels of the head area to the original number through a third convolution network so as to compare the original number with the real image to verify the prediction result.
Specifically, the determining the head region in the human body image may specifically be: the coordinates of the head extracted from the human body image are mapped to the original picture to be detected, the selected target is expanded in the picture to be detected, the expanded area can be set to be a square area by the center point of the head, and the square area is set to be 120-120, so that the probability that the cigarette is contained in the head area is favorably improved.
Specifically, in order to facilitate processing of the image of the head region, the head region may be preprocessed, and the preprocessing process may include removing noise in the picture, and the like.
Specifically, for feature separation, the number of channels in the preprocessed header region may be expanded through the first convolution network, for example, 3 channels may be expanded to 64 channels, and the embodiment of the present invention is not limited herein.
After passing through the first convolutional network, the head region passing through the first convolutional network may be amplified by a preset algorithm, where the amplification size may also be set autonomously, for example, an image of 120mm × 120mm may be amplified to 360mm × 360mm, and the like, an embodiment of the present invention is not limited herein, and cigarette identification may be performed after amplification, and on this basis, in order to improve the display quality of the image of the head region, and thus better compare the image of the head region with a "real image taken by another camera with higher resolution" in the present invention, the embodiment of the present invention may also expand the number of channels of the amplified head region by the second convolutional network, and through this step, the number of channels after interpolation may be expanded, so as to further fuse point values on a feature map obtained by upsampling, and then for convenience of comparison, the number of channels of the header region may be reduced to the original number by a third convolutional network.
It should be noted that the second convolutional network may include a first sub convolutional layer and a second sub convolutional layer, and the two sub convolutional layers may respectively perform channel number expansion once, where two sub convolutional layers are used, it is considered that if the number of sub convolutional layers of the second convolutional network is too small, the influence on the final image amplification effect is too small, and if the number of sub convolutional layers is too large, the model parameters may be increased and the model operation speed may be slowed down to be fast, so in order to balance the model operation speed and the image amplification effect, in the embodiment of the present invention, the number of sub convolutional layers of the second convolutional network is determined to be two.
Specifically, the amplifying the head region passing through the first convolutional network by using a preset algorithm may specifically be:
amplifying the head region by a bicubic interpolation method;
specifically, the head region may be enlarged by a bicubic interpolation method for multiple times, for example, the bicubic interpolation method may be performed three times, so as to achieve the target enlargement effect.
Of course, other types of preset algorithms may be used to enlarge the head region besides the cubic bicubic interpolation method, and the embodiment of the present invention is not limited herein.
Specifically, the main steps of the training process of the target amplification network in fig. 2 are as follows:
STEP 1: setting the batch size of training, and extracting pictures;
STEP 2: preprocessing the picture, taking out noise points in the picture, and scaling the size of the picture to a specific size;
preferably, the specific size is set to 120 × 120, in the process of scaling the size of the picture, scaling is performed without changing the length-width ratio of the picture, the maximum side is scaled to 120, and the small side is subjected to gray filling;
STEP 4: the processed picture enters a first convolution layer;
preferably, the first convolution layer expands the picture channel to 64 dimensions;
STEP 5: performing Bicubic interpolation (Bicubic interpolation) on the pictures;
preferably, the picture is scaled up to 180 x 180;
STEP 6: performing Bicubic interpolation (Bicubic interpolation) on the pictures;
preferably, the picture size is enlarged to 240 x 240;
STEP 7: performing Bicubic interpolation (Bicubic interpolation) on the pictures;
preferably, the picture size is enlarged to 360 x 360;
STEP 8: the picture enters a second convolution layer, the convolution kernel adopts 3 × 3 convolution kernels, the activation function adopts Relu, and the picture belongs to a picture channel (1 × 360 × 128);
STEP 9: the picture enters a second convolution layer, the convolution kernel adopts 3 × 3 convolution kernels, the activation function adopts Relu, and the picture belongs to a picture channel (1 × 360 × 256);
STEP 10: the picture enters a second convolution layer, the convolution kernel adopts 3 × 3 convolution kernels, the activation function adopts Lienar, and the picture channel belongs to (1 × 360 × 3);
STEP 11: and obtaining an amplified target picture.
For better explaining the embodiment of the present invention, please refer to fig. 3, and fig. 3 is a training flowchart of a target detection network provided by the present invention, and as a preferred embodiment, the determining, by a target detection model, a human body image in a to-be-detected picture specifically includes:
preprocessing a picture to be detected;
acquiring a feature map of the preprocessed picture to be detected;
determining a target recommendation area corresponding to a preset target object type from the characteristic diagram;
cutting and unifying the size of the target recommendation area on the feature map;
and determining a human body image from each target recommendation area.
Specifically, in order to facilitate subsequent processing of the human body image, in the embodiment of the present invention, a to-be-detected picture may be first preprocessed, where the preprocessing includes, but is not limited to: the method and the device for processing the image have the advantages that noise in the image to be detected is removed, the size of the image is scaled to a specific size and the like, the scaling action can reduce the data processing amount of the subsequent steps, and accordingly the calculation speed is improved, the specific size can be 640 x 640 and the like.
Specifically, in order to facilitate detection and identification of a target in subsequent steps, in the embodiment of the present invention, the feature map of the preprocessed picture to be detected may be obtained, and the "obtaining the feature map of the preprocessed picture to be detected" may specifically be:
inputting the preprocessed picture to be detected into a ResNet50(Residual Neural Network 50, 50-layer Residual error Network) Network;
inputting the picture to be detected output by the ResNet50 Network into an FPN (Feature Pyramid) Network, and extracting a Feature map.
Specifically, the characteristic diagram is extracted through a ResNet50 network and an FPN network, and the method has the advantages of being high in speed and accuracy.
Of course, besides using the above two networks, the feature map may be extracted in other types of manners, and the embodiment of the present invention is not limited herein.
Specifically, after the feature map is obtained, a target recommendation area corresponding to a preset target object type can be determined from the feature map, that is, a useful object image is found from the feature map, so that the data processing amount is reduced.
The preset target object type may be of various types, for example, a human body, and the like, and the embodiment of the present invention is not limited herein.
Specifically, after the target recommendation areas exist, the target recommendation areas on the feature map can be cut out, then the sizes of the cut target recommendation areas are unified, classification is facilitated, and finally the human body images can be determined from the target recommendation areas.
Specifically, the step of cutting the target recommendation area on the feature map may specifically be: simultaneously inputting the obtained feature map and the target recommendation Region into an ROI Align (Region Of Interest) network to obtain a feature map with a required size, which specifically comprises the following steps:
(1) equally dividing the bbox area according to the size of the output requirement, wherein the vertexes are likely to fall onto the real pixel points after equally dividing;
(2) taking fixed 4 points in each block, weighting the values of 4 real pixel points nearest to each point (bilinear interpolation) aiming at each point, and obtaining the value of the point;
(3) the output of 2x2 is finally obtained by calculating 4 new values in a block, taking the maximum value max among these new values as the output value of the block.
Of course, the target recommendation area may also be cut in other manners, and the embodiment of the present invention is not limited herein.
Specifically, in fig. 3, the main steps of the training process of the target detection network provided by the embodiment of the present invention include:
STEP 1: setting the batch size of training, and extracting pictures;
STEP 2: preprocessing the picture, taking out noise points in the picture, and scaling the size of the picture to a specific size;
preferably, the specific size is set to 640 x 640, in the process of scaling the size of the picture, scaling is performed without changing the length-width ratio of the picture, the maximum side is scaled to 640, and the small side is subjected to gray filling;
STEP 3: performing feature enhancement on training data;
the characteristic enhancement adopts Mosaic and MixUp to carry out data enhancement of pictures so as to enlarge the number of the pictures of the training data set, and the method for enhancing the data has the advantages of high speed and good effect.
Of course, in addition to performing data enhancement by using the Mosaic algorithm and the MixUp algorithm, the training data set may be extended by using other methods.
STEP 4: the processed picture enters a ResNet50 network;
STEP 5: the picture enters an FPN network, and a characteristic diagram is extracted;
STEP 6: carrying out RPN on the extracted feature map to obtain a target recommendation area of the picture;
STEP 7: simultaneously inputting the obtained feature map and the target recommendation area into an ROI Align network to obtain a feature map with a required size;
STEP 7.1: equally dividing the bbox area according to the size of the output requirement, wherein the vertexes are likely to fall onto the real pixel points after equally dividing:
STEP 7.2: taking fixed 4 points in each block, weighting the values of 4 real pixel points nearest to each point (bilinear interpolation) aiming at each point, and obtaining the value of the point;
STEP 7.3: 4 new values are calculated in a block, max is taken out of the new values, and the output value of the block is finally 2x 2;
STEP 8: obtaining a target detection area through a Head layer by the feature diagram with the required size;
STEP 9: the target detection network respectively obtains target frames, classification and score conditions through a full connection layer;
in the embodiment of the invention, the pictures in the daytime can be used for pre-training, so that the parameters of the model can be quickly drawn close to the final target value, then the pictures at night are trained, and finally the pictures in the daytime and at night are subjected to mixed training, so that the model is further converged, and the model training efficiency and the model accuracy are improved.
Of course, in addition to the training mode of the training data set, other modes of application may be performed on the training data set to form another training mode, and the embodiment of the present invention is not limited herein.
As a preferred embodiment, the determining of the human body image from each target recommendation area specifically includes:
determining the object type of each target recommendation area;
judging whether a human body exists in the object type;
if not, executing the step of acquiring the picture to be detected through the camera;
and if so, executing the step of preprocessing the human body image.
Specifically, a feature map with a required size can be processed through a Head layer of a target detection network to obtain a target detection area, then a target frame, classification and score condition of each target object are obtained through a full connection layer, so that the object type of each target recommendation area can be determined, then in order to further reduce the data detection amount, whether a human body exists in the determined object type can be judged, the subsequent steps are executed only when the human body exists, and otherwise, the next picture to be detected can be identified.
As a preferred embodiment, after the human body image in the picture to be detected is determined by the target detection model, and before the head region in the human body image is enlarged by the target enlargement network, the smoking recognition method further includes:
judging whether a head exists in the human body image;
if not, executing the step of acquiring the picture to be detected through the camera;
if the head region exists, the step of amplifying the head region in the human body image through the target amplifying network is executed.
Specifically, considering that a head does not necessarily exist in a human body image due to the reason of an angle and the like, and the smoke extraction recognition is based on the expansion of the head, in order to further reduce the calculation amount, the embodiment of the invention can judge whether the head exists in the human body image, and only when the head exists in the human body image, the subsequent steps can be executed, otherwise, the recognition of the next picture to be detected can be executed.
As a preferred embodiment, after determining whether the head exists in the human body image, before magnifying the head region in the human body image by the target magnifying network, the method for identifying smoking further comprises:
if the head exists, judging whether a shelter exists in the head area;
if the shielding object exists, the step of acquiring the picture to be detected through the camera is executed;
if the obstruction does not exist, the step of amplifying the head area in the human body image through the target amplification network is executed.
Specifically, considering that cigarette identification is difficult to perform even if a head exists in a picture but the head is blocked by an object, the embodiment of the present invention may determine whether a blocking object exists in the head region, and only if the head does not have the blocking object, the subsequent steps may be performed, otherwise, identification of the next picture to be detected may be performed.
As a preferred embodiment, before the head region in the human body image is enlarged by the target enlargement network, the smoke recognition method further includes:
judging whether the head area in the human body image is matched with the head image in the tracking library or not;
if the images are matched, the step of acquiring the image to be detected through the camera is executed;
and if the head regions do not coincide with each other, adding the current head region into a tracking library and executing a step of amplifying the head region in the human body image through a target amplification network.
Specifically, considering that frequent alarming is not needed for the same smoker, and the frequent alarming can generate interference such as noise, the embodiment of the invention can judge whether the head area in the human body image is consistent with the head image in the tracking library, if so, alarming is not needed, and the step of obtaining the picture to be detected through the camera is executed, and if not, the current head area can be added into the tracking library and the subsequent steps are executed, so that frequent alarming can not be carried out on the same smoker, and the user experience is improved.
In the process of target tracking detection, KCF (Kernel Correlation Filter) can be selected for target tracking detection in the embodiment of the present invention, and the present invention has advantages in rate and accuracy.
Of course, the target tracking algorithm may be of various types other than KCF, and the embodiment of the present invention is not limited herein.
In summary, the smoke extraction identification method based on the elimination in the embodiment of the present invention has the following advantages:
(1) according to the method, Mosaic and MixUp are adopted to perform data enhancement in the training process, so that the generalization capability of the model is greatly improved;
(2) according to the method, a distributed training mode is adopted for model training, the images in the day are firstly adopted for pre-training, and finally mixed training of the images in the day and at night is carried out, so that the convergence rate of the model is increased;
(3) the network model set by the method is small in framework and few in model parameters, and occupies less equipment memory and storage;
(4) the method adopts a target amplification network to amplify the super-resolution of a specific target (head), so that the subsequent classification network can conveniently process the image;
(5) a KCF target tracking algorithm is set, so that each target only alarms once, and repeated alarming is avoided;
(6) according to the method and the device, the shielded target in the anti-static chain area is identified, so that the influence caused by shielding of the target is avoided, and the actual detection precision of the model is improved;
(7) the method and the device adopt the grading inspection of the large target and the small target, if no human body exists, the following process is not executed, and the real-time performance of detection is greatly improved.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a smoke recognition device provided in the present invention, the smoke recognition device includes:
an obtaining module 41, configured to obtain a picture to be detected through a camera;
the target detection module 42 is used for determining a human body image in the picture to be detected through the target detection model;
a target amplifying module 43, configured to amplify the head region in the human body image through a target amplifying network;
a judging module 44, configured to judge whether a cigarette exists in the amplified head region, and if yes, trigger an alarm module 45;
and the alarm module 45 is used for controlling an alarm to give an alarm.
For the description of the smoking identification device provided in the embodiment of the present invention, reference is made to the foregoing embodiment of the smoking identification method, and the embodiment of the present invention is not described herein again.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a smoke recognition device provided in the present invention, the smoke recognition device includes:
a memory 51 for storing a computer program;
a processor 52 for implementing the steps of the smoke recognition method as in the previous embodiments when executing the computer program.
For the description of the smoking identification device provided in the embodiment of the present invention, reference is made to the foregoing embodiment of the smoking identification method, and the embodiment of the present invention is not described herein again.
The invention also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, carries out the steps of the method of smoke recognition as in the preceding embodiments.
For the introduction of the computer-readable storage medium provided by the embodiment of the present invention, reference is made to the foregoing embodiment of the smoke identification method, and details of the embodiment of the present invention are not repeated herein.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description. It should also be noted that, in this specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method of identifying a smoking event, comprising:
acquiring a picture to be detected through a camera;
determining a human body image in the picture to be detected through a target detection model;
amplifying the head region in the human body image through a target amplification network;
judging whether cigarettes exist in the enlarged head area;
if the alarm exists, the alarm is controlled to give an alarm.
2. The method according to claim 1, wherein the amplifying the head region in the human body image through the target amplifying network specifically comprises:
determining a head region in the human body image;
pre-processing the head region;
expanding the number of channels of the preprocessed header region through a first convolution network;
amplifying the head region passing through the first convolutional network by a preset algorithm;
expanding the number of channels of the enlarged head region through a second convolution network;
and reducing the number of channels of the head area to the original number through a third convolution network so as to compare the original number with the real image to verify the prediction result.
3. The smoking identification method according to claim 2, wherein the determination of the human body image in the picture to be detected by the target detection model specifically comprises:
preprocessing the picture to be detected;
acquiring a feature map of the preprocessed picture to be detected;
determining a target recommendation area corresponding to a preset target object type from the characteristic diagram;
cutting and unifying the size of the target recommendation area on the feature map;
and determining a human body image from each target recommendation area.
4. The smoking identification method according to claim 3, wherein the determining of the human body image from each of the target recommendation areas specifically includes:
determining the object type of each target recommendation area;
judging whether a human body exists in the object type;
if not, executing the step of acquiring the picture to be detected through the camera;
and if so, executing the step of preprocessing the human body image.
5. The method according to claim 4, wherein after the human body image in the picture to be detected is determined by the target detection model and before the head region in the human body image is enlarged by the target enlargement network, the method further comprises:
judging whether a head exists in the human body image or not;
if not, executing the step of acquiring the picture to be detected through the camera;
if yes, the step of amplifying the head area in the human body image through the target amplification network is executed.
6. The method of claim 5, wherein after determining whether the head is present in the human body image and before the head region in the human body image is enlarged by the target enlarging network, the method further comprises:
if the head exists, judging whether a shelter exists in the head area;
if the shielding object exists, the step of acquiring the picture to be detected through the camera is executed;
and if no occlusion exists, executing the step of amplifying the head area in the human body image through the target amplifying network.
7. The method according to any one of claims 1 to 6, wherein before the head region in the human body image is enlarged by the target enlarging network, the method further comprises:
judging whether the head area in the human body image is matched with the head image in the tracking library or not;
if the images are matched, the step of acquiring the images to be detected through the camera is executed;
and if the head regions do not coincide with each other, adding the current head region into the tracking library and executing the step of amplifying the head region in the human body image through a target amplification network.
8. A smoke recognition device, comprising:
the acquisition module is used for acquiring the picture to be detected through the camera;
the target detection module is used for determining the human body image in the picture to be detected through a target detection model;
the target amplification module is used for amplifying the head area in the human body image through a target amplification network;
the judging module is used for judging whether cigarettes exist in the amplified head area or not, and if yes, the alarming module is triggered;
and the alarm module is used for controlling an alarm to give an alarm.
9. A smoke evacuation identification device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the method of smoke recognition according to any one of claims 1 to 7 when executing said computer program.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the method of identifying a smoking according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210602059.2A CN114937302A (en) | 2022-05-30 | 2022-05-30 | Smoking identification method, device and equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210602059.2A CN114937302A (en) | 2022-05-30 | 2022-05-30 | Smoking identification method, device and equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114937302A true CN114937302A (en) | 2022-08-23 |
Family
ID=82867547
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210602059.2A Withdrawn CN114937302A (en) | 2022-05-30 | 2022-05-30 | Smoking identification method, device and equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114937302A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115205767A (en) * | 2022-09-16 | 2022-10-18 | 浪潮通信信息系统有限公司 | Smoking behavior detection method, system and device |
-
2022
- 2022-05-30 CN CN202210602059.2A patent/CN114937302A/en not_active Withdrawn
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115205767A (en) * | 2022-09-16 | 2022-10-18 | 浪潮通信信息系统有限公司 | Smoking behavior detection method, system and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210248378A1 (en) | Spatiotemporal action detection method | |
| CN112528960B (en) | Smoking behavior detection method based on human body posture estimation and image classification | |
| CN112052797A (en) | MaskRCNN-based video fire identification method and system | |
| CN111079739B (en) | Multi-scale attention feature detection method | |
| CN111524145A (en) | Intelligent picture clipping method and system, computer equipment and storage medium | |
| CN112115775A (en) | Smoking behavior detection method based on computer vision in monitoring scene | |
| CN112818871B (en) | Target detection method of full fusion neural network based on half-packet convolution | |
| CN112257643A (en) | Smoking behavior and calling behavior identification method based on video streaming | |
| CN113469050A (en) | Flame detection method based on image subdivision classification | |
| CN111582074A (en) | Monitoring video leaf occlusion detection method based on scene depth information perception | |
| CN111860316A (en) | Driving behavior recognition method and device and storage medium | |
| CN110942456B (en) | Tamper image detection method, device, equipment and storage medium | |
| CN113326754A (en) | Smoking behavior detection method and system based on convolutional neural network and related equipment | |
| CN114937302A (en) | Smoking identification method, device and equipment and computer readable storage medium | |
| CN115761627A (en) | Fire smoke flame image identification method | |
| CN112991159B (en) | Face illumination quality evaluation method, system, server and computer readable medium | |
| CN114399734A (en) | Forest fire early warning method based on visual information | |
| CN114359733A (en) | Vision-based smoke fire detection method and system | |
| CN112766046B (en) | Target detection method and related device | |
| CN112435256A (en) | CNV active focus detection method and device based on image and electronic equipment | |
| CN112784642A (en) | Vehicle detection method and device | |
| CN113065379A (en) | Image detection method and device fusing image quality and electronic equipment | |
| CN113688810B (en) | Target capturing method and system of edge device and related device | |
| CN111832439B (en) | Multi-face rapid identification method and processing terminal | |
| CN113128340B (en) | Personnel intrusion detection method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220823 |
|
| WW01 | Invention patent application withdrawn after publication |