CN112132090A - Smoke and fire automatic detection and early warning method based on YOLOV3 - Google Patents

Abstract

The present invention provides a method for automatic detection and early warning of fireworks based on YOLOV3, comprising: S1, constructing a sample set of a training detection model; S2, constructing a deep learning target detection network architecture based on YOLOV3; S3, configuring training parameters, and training a detection model; S4, obtain the image information to be detected; from the monitoring equipment at the scene to be detected, obtain the image frames of the live video picture, and use the image preprocessing method to process the frame-by-frame images; S5, detect smoke and flame targets; process in step S4 The passed video image frames are sent to the pre-trained detection model in step S3 to detect the target, and the detection result is output; S6, the detection result is post-processed; S7, the multi-frame image detection results are continuously analyzed to confirm that the target is valid and output Call the police. The YOLOV3-based automatic pyrotechnic detection and early warning method of the present invention can realize second-level detection and alarm, greatly shorten the fire warning time, notify timely rescue in time, and effectively prevent the spread of fire.

Description

A fireworks automatic detection and early warning method based on YOLOV3

technical field

The invention belongs to the technical field of video surveillance, and in particular relates to a fireworks automatic detection and early warning method based on YOLOV3.

Background technique

Fireworks (smoke and flame) detection refers to the identification and positioning of fireworks in surveillance video images, which is of great significance in the field of security monitoring.

Fire is one of the very common and extremely harmful disasters, which often causes huge loss of resources and property and may cause casualties. Fire prevention and control and early warning have become the top priority. Timely early warning can quickly notify duty personnel and assist firefighters to deal with fire crises in a timely manner, so as to prevent and avoid the sudden and spread of fire accidents as soon as possible, and minimize losses. Among them, accurate and rapid detection of smoke and flames has become the top priority.

Traditional detection methods are mainly sensors for physical sampling of temperature, transparency, smoke, etc., but sensors are mainly suitable for short-distance sensing, which are easily limited by venues, and are difficult to guarantee ease of use, detection accuracy, and reliability. It is difficult to judge the specific situation of the scene in time if the personnel do not visit the site. The video-based detection method can realize remote viewing through the transmission of long-distance pictures, so it has been rapidly developed and applied. The commonly used video detection method generally extracts the motion area of the video image and discriminates according to the characteristics of the RGB or HSV channel components of the flame area. This method has some effects on the discrimination of flames, but it cannot be used for smoke. It is also used to predict the image in blocks, use sliding windows of different sizes to extract regions, and then send them to various CNNs (convolutional neural networks) for classification to determine whether it is flame or smoke, but this method is inefficient and accurate. It is not enough, and the location of the fireworks is not accurate enough, because the sliding window is pre-selected and the size and shape are fixed, while the shape of the smoke and flames is not fixed.

SUMMARY OF THE INVENTION

In view of this, in order to overcome above-mentioned defect, the present invention aims to propose a kind of pyrotechnic automatic detection and early warning method based on YOLOV3,

In order to achieve the above object, the technical scheme of the present invention is achieved in this way:

A fireworks automatic detection and early warning method based on YOLOV3, comprising:

S1. Construct a sample set for training the detection model; first collect original materials, process the acquired original materials with image data enhancement technology, and obtain a training sample data set; then use a sample labeling tool to mark the detection target in the picture of the training sample data set. Set the target frame and set the category to which it belongs, save the sample image and the generated annotation file as the sample set respectively; use the clustering algorithm to cluster the target frame marked in all the training set samples;

S2. Build a deep learning target detection network architecture based on YOLOV3; use a cropped Darknet-53 convolutional neural network as a backbone to extract features from the input image, and input the feature map generated by the neural network into the detection model of YOLOV3;

S3. Configure the training parameters and train the detection model;

S4, acquiring the image information to be detected; from the monitoring equipment on the site to be detected, acquiring image frames of the on-site video picture, and using an image preprocessing method to process the frame-by-frame images;

S5, detecting smoke and flame targets; sending the video image frames processed in step S4 into the pre-trained detection model in step S3 for target detection, and outputting the detection results;

S6, post-processing the detection result; the processing method includes determining whether it is an effective target according to the confidence level of each target detected in step S5, and if the confidence level is lower than the threshold, there may be a false detection, and no processing is performed; Determine whether the target is in the specified area according to the coordinates of the target, if not, exclude it; for the overlapping targets existing in the detection results, set an IOU parameter to judge the degree of overlap, and remove the ones with a large overlap and a low confidence value in the box, only Retaining a target frame with the highest and higher degree of overlap may detect the same target; according to the coordinates of the detection result, determine the size of the target, and remove the detection results that do not conform to the actual scene size range;

S7. Continuously analyze the detection results of multi-frame images, confirm that the target is valid and output an alarm; analyze the detection results of continuous multi-frame images, determine whether it is a valid target, output an alarm signal in time, and record relevant information.

Further, in the step S1, the original materials include but are not limited to smoke and flame videos and pictures;

The detection target is smoke or flame;

The target frame is a circumscribed rectangular frame.

Further, the concrete method of described step S3 is as follows:

Set the hyperparameters of the training detection model, set the initial learning rate to 0.001, set each training batch to 64, and set the total number of iteration rounds of training samples to 140 rounds; the model training is based on the BP principle, using the SGD algorithm to optimize the network weight parameters, Perform iterative training to reduce the loss value of the network to a lower value. After the training is completed, obtain a model for detecting smoke and flames;

Further, in the step S5, the detection result output by the model includes the category to which the target belongs, the coordinates of the bounding rectangle of the target, and the corresponding confidence level;

Further, the calculation method of the loss value is as follows:

The loss of training network is divided into confidence loss L _conf (O, C), category loss L _cla (o, c) and localization loss L _loc (l, g), the total loss L (O, o, C, c, l, g) is the weighted sum of the three. Use the information b(x, y, w, h, C, c ₁ , c ₂ ) of the predicted frame output by the network and the real value g(x, y, w, h) to calculate the loss to obtain the final loss, where (x , y, w, h) represent the abscissa and ordinate of the midpoint of the bounding rectangle of the target and the width and height of the rectangle respectively, C represents the probability that the location of the prediction frame is a target, c ₁ , c ₂ represent the category to which the target belongs The probability.

Calculate the total loss value:

L(O,o,C,c,l,g)=λ ₁ L _conf (O,C)+λ ₂ L _cla (o,c)+λ ₃ L _loc (l,g)

Among them, λ ₁ , λ ₂ and λ ₃ are the weighted weights of confidence loss, category loss and localization loss, which are 0.3, 0.2, and 0.5, respectively;

Confidence loss calculation:

where O _i indicates whether the current position has a target, which is the true value, 1 if there is a target, 0 otherwise, and C _i is the probability that the current position predicted by the model is a target;

Class loss calculation:

c _ij =Sigmoid(c _ij )

Among them, o _ij indicates whether there is the jth category at the position of the _i -th prediction frame, which belongs to the real value. If it exists, it is 1; probability of a class;

Positioning loss calculation:

Where (g ^x , g ^y , g ^w , g ^h ) is the manually labeled target frame information g(x, y, w, h), the subscript i represents the ith frame, which belongs to the true value, (b ^x , b ^y , b ^w , b ^h ) represent the coordinate information of the detection frame predicted by the network, corresponding to (g ^x , g ^y , g ^w , g ^h ), and (c ^x , c ^y , p ^w , p ^h ) represent the pre- Set the information of anchors (anchor points), where (c ^x , c ^y ) represents the position of the anchors center point on the feature map, and (p ^w , p ^h ) represents the width and height of the preset anchors, which are separate from the annotation information and prediction information. Correspondingly; Anchors is the result of statistics and clustering of the width and height of the target frame marked in the training data through the clustering algorithm during YOLOV3 training.

Compared with the prior art, the YOLOV3-based pyrotechnic automatic detection and early warning method of the present invention has the following advantages:

(1) The video picture of the automatic detection and early warning method for fireworks based on YOLOV3 according to the present invention is intuitive and easy to use; the camera is used to monitor the designated area, and the real-time status of the scene can be remotely viewed anytime, anywhere, and the camera is installed outside the monitoring area. It can predict the trend of fire spread from a higher perspective and remotely command disaster relief tasks. Realize non-contact detection, not limited by space and environmental conditions, suitable for a variety of scenarios (deep forests, fields, factories, warehouses, supermarkets, residences, etc.), can achieve second-level detection and alarm, greatly shorten the fire warning time , timely notification and timely rescue, and effectively prevent the spread of fire.

(2) The automatic detection and early warning method for fireworks based on YOLOV3 according to the present invention is intelligently detected, which saves costs; realizes 24-hour uninterrupted real-time intelligent analysis, automatically detects and recognizes smoke and flames, automatically alarms, and no longer relies on manual inspections way to save labor costs and improve work efficiency. And does not rely on any other sensor equipment, only one camera on site can achieve a wide range of coverage compatible with large and small scenes, far and near distances, and various angles, saving equipment costs.

(3) The YOLOV3-based pyrotechnic automatic detection and early warning method of the present invention has deep learning and accurate detection; the algorithm of the present invention combines the most cutting-edge deep learning technology, is based on the Darknet-53 deep neural network, and adopts the deep learning target based on YOLOV3 The detection algorithm has the characteristics of high detection rate, fast detection speed, good real-time performance and stable performance, which can meet the needs of practical applications.

Description of drawings

The accompanying drawings constituting a part of the present invention are used to provide further understanding of the present invention, and the exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the attached image:

FIG. 1 is a flowchart of an automatic detection and early warning method according to an embodiment of the present invention.

Detailed ways

It should be noted that the embodiments of the present invention and the features of the embodiments may be combined with each other under the condition of no conflict.

In the description of the present invention, it should be understood that the terms "center", "portrait", "horizontal", "top", "bottom", "front", "rear", "left", "right", " The orientation or positional relationship indicated by vertical, horizontal, top, bottom, inner, outer, etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and The description is simplified rather than indicating or implying that the device or element referred to must have a particular orientation, be constructed and operate in a particular orientation, and therefore should not be construed as limiting the invention. In addition, the terms "first", "second", etc. are used for descriptive purposes only, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature defined as "first", "second", etc., may expressly or implicitly include one or more of that feature. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

In the description of the present invention, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense, unless otherwise expressly specified and limited, for example, it may be a fixed connection or a detachable connection Connection, or integral connection; can be mechanical connection, can also be electrical connection; can be directly connected, can also be indirectly connected through an intermediate medium, can be internal communication between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific situations.

The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

The algorithm of the invention is suitable for automatic detection and early warning of smoke and flames in various scenarios. The purpose is to realize 24-hour work under unattended conditions, automatically analyze possible abnormal smoke and open flames in the monitoring area, and at the same time, it can also be viewed remotely. The real-time on-site picture is convenient for judging the on-site situation according to the intuitive picture, and commanding and dispatching rescue.

The specific implementation method is as follows:

1. Train the detection model. Through the collected raw materials, the sample images for training are obtained after processing, and the deep learning neural network is trained based on the Darknet framework to obtain a model for detecting smoke and flames, which is mainly divided into three steps.

The first is to construct a sample set for training. The collection of samples can be used to download videos and pictures from the Internet using crawler tools, or download public fireworks data sets. Manually label the sample data, mark the bounding rectangle of smoke and flame (detection target) in the training sample set picture and set the category (Smoke and Fire), and then use a variety of image data enhancement techniques to enhance the image and label. process, and finally save the sample image and the generated annotation file as a sample set. The target frames marked in all the training set samples are clustered by the clustering algorithm, and 9 anchors are obtained. Each anchor is actually a clustering of the width and height of the target frame, which is used to return the target size during training;

Then build a deep learning target detection network architecture based on YOLOV3. The cropped Darknet-53 convolutional neural network is used as the backbone to extract features from the input image, and the high-dimensional feature map generated by the network extraction is used as the yolo layer of YOLOV3 to predict and calculate the loss value. Cutting the Darknet-53 network is to reduce the number of output channels of all layers of the network by half, reducing the amount of calculation and processing speed;

Finally, configure the training parameters and train the detection model. Set the benchmark learning rate of the network to 0.001, send the samples to the network in batches for training, set each training batch to 64, set the total number of iterations of the training samples to 140, and set the number of training rounds every certain sample. (such as 40), multiply the learning rate by 0.1, and perform three operations in total, and reduce the learning rate during the training process, which can make the model training quickly converge and stabilize. Model training is based on the principle of BP (back-propagation), the loss (loss) value calculated by the prediction result output by the back-propagation network and the actual labeling result, and the SGD (stochastic gradient descent) algorithm is used to update the network weight. Through continuous iterative optimization training, The loss value of the network gradually decreases and becomes stable. After the training is complete, get a model for detecting smoke and flames;

2. Detect the picture and output the detection result. The detection model generated by the previous training is used to detect and analyze the video frame of the actual monitoring scene, and obtain the confidence and position information of the detection target in the picture, which is mainly divided into three steps.

The first is equipment selection and installation. The algorithm of the invention is suitable for various scenarios, and the camera device can be installed at a high point of the scene to be monitored, and the monitoring area can be tilted downwards. Installed in a high and unobstructed position, the area that can be monitored is larger. If you need multi-angle and long-distance monitoring, you can use a variable magnification and rotating dome camera to achieve 360° rotation and high magnification zoom. You only need to set the preset position in advance, and you can automatically cruise and monitor all the coverage areas;

Then set the detection area. Set the detection area in the video screen, and draw a polygonal area in clockwise or counterclockwise order as the designated area. Only the target within the specified area is detected as a valid target;

Finally, the model is used to detect the picture information in real time and output the results. The acquired video frame image is simply preprocessed, and then the detection model is used for detection and analysis to obtain the detection result of the current frame. If there is a valid target in the picture, the model will output the coordinates of the bounding rectangle of the target, the category to which the target belongs, and the corresponding confidence;

3. Processing and analysis of test results. The detection results output by the model are only all detected target information in the screen, and cannot all be output as valid targets. Therefore, it is necessary to make reasonable judgments according to the confidence, state, location and other information of the target, according to the preset confidence threshold and detection area. , and then output the final result, which is mainly divided into two steps.

Post-processing the test results. It mainly includes setting a confidence threshold in advance, and judging whether it is an effective target according to the confidence of each target detected in (v). The confidence level indicates how likely the detection result is to be a valid target. The higher the confidence level, the higher the probability of being a valid target, and the value is 1. On the contrary, if the confidence level is low, it may be a false detection, and no further testing is required. Other processing; according to the corresponding coordinates of each target to determine whether the target is in the designated area, mainly to determine the center point of the target, if the center point is in the designated area, it is determined to be in the area, if not, it is excluded; for the detection results If there are overlapping detection frames in the frame, set an IOU (intersection and union ratio) parameter to judge the degree of overlap. If there are multiple frames with a large degree of overlap, remove the frame with a high degree of overlap and a low confidence value, and only keep the one with the highest degree of overlap. , the target frame with high overlap may detect the same target, and only one valid result with the highest confidence can be output; according to the coordinates of the target frame of the detection result, the size of the target in the screen is judged, and some The result of possible abnormal size does not conform to the actual scene size range. Finally output a valid target that satisfies all conditions;

Then, the continuous video frame images are comprehensively analyzed to determine the final counting result. Analyze the detection results of continuous multi-frame images to determine whether it is an effective target. Through the comparison of the target positions, it is determined that the targets in the same position are detected in consecutive multiple frames of images, and the target area has the characteristics of becoming larger, then it is confirmed as a valid result, and an alarm signal is sent to the security personnel in time, and relevant records are recorded. information.

Among them, the calculation method of the loss value is shown in the following formula. The loss of training network is divided into confidence loss L _conf (O, C), category loss L _cla (o, c) and localization loss L _loc (l, g), the total loss L (O, o, C, c, l, g) is the weighted sum of the three. Use the information b(x, y, w, h, C, c ₁ , c ₂ ) of the predicted frame output by the network and the real value g(x, y, w, h) to calculate the loss to obtain the final loss, where (x , y, w, h) represent the abscissa and ordinate of the midpoint of the bounding rectangle of the target and the width and height of the rectangle respectively, C represents the probability that the location of the prediction frame is a target, c ₁ , c ₂ represent the category to which the target belongs The probability.

Calculate the total loss value:

L(O,o,C,c,l,g)=λ ₁ L _conf (O,C)+λ ₂ L _cla (o,c)+λ ₃ L _loc (l,g)

Among them, λ ₁ , λ ₂ and λ ₃ are the weighted weights of confidence loss, category loss and localization loss, respectively, which are 0.3, 0.2, and 0.5.

Confidence loss calculation:

where O _i indicates whether the current position has a target, which is the true value, 1 if there is a target, 0 otherwise, and C _i is the probability that the current position predicted by the model is a target;

Class loss calculation:

c _ij =Sigmoid(c _ij )

Among them, o _ij indicates whether there is the jth category at the position of the _i -th prediction frame, which belongs to the real value. If it exists, it is 1; probability of a class;

Positioning loss calculation:

Where (g ^x , g ^y , g ^w , g ^h ) is the manually labeled target frame information g(x, y, w, h), the subscript i represents the ith frame, which belongs to the true value, (b ^x , b ^y , b ^w , b ^h ) represent the coordinate information of the detection frame predicted by the network, corresponding to (g ^x , g ^y , g ^w , g ^h ), and (c ^x , c ^y , p ^w , p ^h ) represent the pre- Set the information of anchors (anchor points), where (c ^x , c ^y ) represents the position of the anchors center point on the feature map, and (p ^w , p ^h ) represents the width and height of the preset anchors, which are separate from the annotation information and prediction information. Corresponding. Anchors is the result of statistics and clustering of the width and height of the target frame marked in the training data through the clustering algorithm during YOLOV3 training.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the scope of the present invention. within the scope of protection.

Claims (5)

1. a pyrotechnic automatic detection and early warning method based on YOLOV3, is characterized in that, comprises: S1. Construct a sample set for training the detection model; first collect original materials, process the acquired original materials with image data enhancement technology, and obtain a training sample data set; then use a sample labeling tool to mark the detection target in the picture of the training sample data set. Set the target frame and set the category to which it belongs, save the sample image and the generated annotation file as the sample set respectively; use the clustering algorithm to cluster the target frame marked in all the training set samples; S2. Build a deep learning target detection network architecture based on YOLOV3; use a cropped Darknet-53 convolutional neural network as a backbone to extract features from the input image, and input the feature map generated by the neural network into the detection model of YOLOV3; S3. Configure the training parameters and train the detection model; S4, acquiring the image information to be detected; from the monitoring equipment on the site to be detected, acquiring image frames of the on-site video picture, and using an image preprocessing method to process the frame-by-frame images; S5, detecting smoke and flame targets; sending the video image frames processed in step S4 into the pre-trained detection model in step S3 for target detection, and outputting the detection results; S6, post-processing the detection result; the processing method includes determining whether it is an effective target according to the confidence level of each target detected in step S5, and if the confidence level is lower than the threshold, there may be a false detection, and no processing is performed; Determine whether the target is in the specified area according to the coordinates of the target, if not, exclude it; for the overlapping targets existing in the detection results, set an IOU parameter to judge the degree of overlap, and remove the ones with a large overlap and a low confidence value in the box, only Retaining a target frame with the highest and higher degree of overlap may detect the same target; according to the coordinates of the detection result, determine the size of the target, and remove the detection results that do not conform to the actual scene size range; S7. Continuously analyze the detection results of multi-frame images, confirm that the target is valid and output an alarm; analyze the detection results of continuous multi-frame images, determine whether it is a valid target, output an alarm signal in time, and record relevant information. 2. The fireworks automatic detection and early warning method based on YOLOV3 according to claim 1, is characterized in that: in described step S1, original material includes but not limited to smoke and flame video, picture; The detection target is smoke or flame; The target frame is a circumscribed rectangular frame. 3. the pyrotechnic automatic detection and early warning method based on YOLOV3 according to claim 1, is characterized in that, the concrete method of described step S3 is as follows: Set the hyperparameters of the training detection model, set the initial learning rate to 0.001, set each training batch to 64, and set the total number of iteration rounds of training samples to 140 rounds; the model training is based on the BP principle, using the SGD algorithm to optimize the network weight parameters, Iterative training is performed to reduce the loss value of the network to a lower value. After the training is completed, the model for detecting smoke and fire is obtained. 4. the pyrotechnic automatic detection and early warning method based on YOLOV3 according to claim 1, is characterized in that: in described step S5, the detection result of model output comprises the category to which target belongs, the coordinates of the bounding rectangle of target and corresponding confidence Spend. 5. the pyrotechnic automatic detection and early warning method based on YOLOV3 according to claim 3, is characterized in that: the calculation method of loss value is as follows: The loss of training network is divided into confidence loss L _conf (O, C), category loss L _cla (o, c) and localization loss L _loc (l, g), the total loss L (O, o, C, c, l, g) is the weighted sum of the three. Use the information b(x, y, w, h, C, c ₁ , c ₂ ) of the predicted frame output by the network and the real value g(x, y, w, h) to calculate the loss to obtain the final loss, where (x , y, w, h) represent the abscissa and ordinate of the midpoint of the bounding rectangle of the target and the width and height of the rectangle respectively, C represents the probability that the location of the prediction frame is a target, c ₁ , c ₂ represent the category to which the target belongs The probability. Calculate the total loss value: L(O,o,C,c,l,g)=λ ₁ L _conf (O,C)+λ ₂ L _cla (o,c)+λ ₃ L _loc (l,g) Among them, λ ₁ , λ ₂ and λ ₃ are the weighted weights of confidence loss, category loss and localization loss, which are 0.3, 0.2, and 0.5, respectively; Confidence loss calculation:

where O _i indicates whether the current position has a target, which is the true value, 1 if there is a target, 0 otherwise, and C _i is the probability that the current position predicted by the model is a target; Class loss calculation:

c _ij =Sigmoid(c _ij ) Among them, o _ij indicates whether there is the jth category at the position of the _i -th prediction frame, which belongs to the real value. If it exists, it is 1; probability of a class; Positioning loss calculation:

Where (g ^x , g ^y , g ^w , g ^h ) is the manually labeled target frame information g(x, y, w, h), the subscript i represents the ith frame, which belongs to the true value, (b ^x , b ^y , b ^w , b ^h ) represent the coordinate information of the detection frame predicted by the network, corresponding to (g ^x , g ^y , g ^w , g ^h ), and (c ^x , c ^y , p ^w , p ^h ) represent the pre- Set the information of the anchors, where (c ^x , c ^y ) represents the position of the center point of the anchors on the feature map, and (p ^w , p ^h ) represents the width and height of the preset anchors, corresponding to the annotation information and prediction information respectively; Anchors It is the result of statistics and clustering of the width and height of the target frame marked in the training data through the clustering algorithm during YOLOV3 training.

Images