CN110969205A - Forest smoke and fire detection method based on target detection, storage medium and equipment - Google Patents
Forest smoke and fire detection method based on target detection, storage medium and equipment Download PDFInfo
- Publication number
- CN110969205A CN110969205A CN201911200889.7A CN201911200889A CN110969205A CN 110969205 A CN110969205 A CN 110969205A CN 201911200889 A CN201911200889 A CN 201911200889A CN 110969205 A CN110969205 A CN 110969205A
- Authority
- CN
- China
- Prior art keywords
- fire
- forest
- smoke
- network
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 239000000779 smoke Substances 0.000 title claims abstract description 93
- 238000001514 detection method Methods 0.000 title claims abstract description 92
- 238000003062 neural network model Methods 0.000 claims abstract description 39
- 238000004364 calculation method Methods 0.000 claims abstract description 14
- 238000012549 training Methods 0.000 claims description 33
- 238000000034 method Methods 0.000 claims description 28
- 238000004891 communication Methods 0.000 claims description 17
- 238000004422 calculation algorithm Methods 0.000 claims description 11
- 239000000284 extract Substances 0.000 claims description 8
- 238000012360 testing method Methods 0.000 claims description 7
- 238000011176 pooling Methods 0.000 claims description 6
- 238000011410 subtraction method Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 2
- 238000002372 labelling Methods 0.000 claims description 2
- 238000012544 monitoring process Methods 0.000 abstract description 7
- 230000008569 process Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 239000000463 material Substances 0.000 description 3
- 241000282414 Homo sapiens Species 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B17/00—Fire alarms; Alarms responsive to explosion
- G08B17/005—Fire alarms; Alarms responsive to explosion for forest fires, e.g. detecting fires spread over a large or outdoors area
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B17/00—Fire alarms; Alarms responsive to explosion
- G08B17/12—Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions
- G08B17/125—Actuation by presence of radiation or particles, e.g. of infrared radiation or of ions by using a video camera to detect fire or smoke
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Emergency Management (AREA)
- Business, Economics & Management (AREA)
- Biodiversity & Conservation Biology (AREA)
- Multimedia (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a forest smoke and fire detection method based on target detection, a storage medium and equipment, and belongs to the field of computer vision target detection and forest smoke and fire video monitoring. Firstly, acquiring a forest smoke and fire image; then, a neural network model comprising a multistage network is adopted to perform forward calculation on the forest firework image to obtain a firework area and confidence thereof; and comparing the confidence with a confidence threshold, and when the confidence is greater than the confidence threshold, determining that the fire is generated, and when the confidence is less than or equal to the confidence threshold, determining that the fire is not generated. According to the invention, the network model with high precision is obtained by using the small sample, the influence of the environment is small, the detection accuracy is high, the false alarm is not easy to occur, and the boundary frame of the fire and smoke area can be obtained, so that the position of the fire can be judged, and the fire fighter can rapidly arrive at the fire area to extinguish the fire. The storage medium and the device can be directly deployed and used for forest smoke and fire detection.
Description
Technical Field
The invention belongs to the field of computer vision target detection and forest smoke and fire video monitoring, and particularly relates to a forest smoke and fire detection method, a storage medium and equipment based on target detection.
Background
Forest fires are common natural disasters, and casualties and property losses caused by forest fires in the world are huge every year, for example, in recent years, the forest fires in California, Indonesia, Amazon and the like, not only a great amount of casualties and property losses are caused, but also the natural environment is seriously influenced. In the fire fighting process, a large amount of manpower and material resources are input, the forest fire is still difficult to be effectively extinguished, and the forest fire poses serious threats to the survival of human beings and animals. China is a multi-forest country, the number of casualties caused by forest fires is recorded in tens of thousands every year, the property loss is hundreds of millions, and the amount of manpower and material resources invested for forest fire prevention and suppression every year is very large.
For forest fires, early discovery can effectively reduce the difficulty in putting out, reduce the loss of forestry resources and effectively reduce casualties. The forest protection personnel find forest fires and are difficult to effectively manage the forest in large area, so that the forest fires can be remotely monitored through the video monitoring system along with the development of computer technology and video technology, the human input is effectively reduced, and the monitoring efficiency is improved. The traditional method is to detect forest fires only through a monitoring system, and still needs personnel to watch video images in real time to determine the occurrence situation of the fire, but with the development of machine vision and deep learning, the forest fire situation can be automatically judged by analyzing and processing the forest images through designing a forest fire and smoke detection algorithm, so that the people are relieved from repeated and monotonous work.
When forest fire and smoke detection is carried out, a commonly used method at present is a picture classification method, namely, whether fire occurs or not is judged by classifying pictures. However, the image classification method has a small visual field, only local information can be acquired, the information of a position to be detected cannot be combined with global information, and false alarm is easily caused if the local illumination condition is special or the image classification method is located at a boundary with large contrast between sky and forest, so that the method is easily influenced by environmental conditions and has low accuracy when the image classification method is used for forest smoke and fire detection. In addition, when the detection model is trained, sample data is concentrated, the marked smoke part can only occupy a small part of the sample image, a large part of the sample image is a background, and when a picture classification method is adopted, a large number of samples are needed to judge forest smoke and fire, so that the requirements of the model building and training processes on computer hardware are high. And when the image classification method is adopted, the specific position of fire and smoke can not be accurately detected, so that when a forest fire breaks out, a rescue worker cannot reach the designated position quickly, and the forest fire is rescued.
In conclusion, the existing forest smoke and fire detection method is easily influenced by the external environment, the detection accuracy is low, and the occurrence position of forest smoke and fire cannot be accurately detected, so that the forest fire suppression is influenced.
Disclosure of Invention
The technical problem is as follows: the invention provides a forest smoke and fire detection method based on target detection, aiming at obtaining a model for accurately judging forest smoke and fire by training with a small amount of samples, accurately judging the forest smoke and fire conditions without being influenced by the external environment and judging the occurrence position of forest fire, so that firefighters can quickly put out the forest fire. Meanwhile, the invention also provides a storage medium and equipment comprising the storage medium, which are used for engineering deployment and forest smoke and fire detection.
The technical scheme is as follows: the invention relates to a forest smoke and fire detection method based on target detection, which comprises the steps of firstly, obtaining a forest smoke and fire image;
adopting a neural network model comprising a multistage network to perform forward calculation on the forest smoke and fire image to obtain a smoke and fire area and confidence thereof;
and comparing the confidence with a confidence threshold, and when the confidence is greater than the confidence threshold, determining that the fire is generated, and when the confidence is less than or equal to the confidence threshold, determining that the fire is not generated.
Further, building and training the multi-stage neural network model comprises the following steps:
s1: establishing a sample data set of a forest firework image, carrying out boundary box labeling on a firework area in the sample image in a coordinate mode, and converting the labeled boundary box into quantitative information;
s2: constructing a neural network model comprising a multistage network, wherein the neural network model can extract image characteristic information, extract time characteristics of interframe information, exchange the interframe information and judge the occurrence position of forest smoke and fire by using the characteristic information;
s3: training a neural network model by adopting an open source data set training and the smoke and fire sample data set, determining network model parameters, and storing a weight file.
Further, the neural network model comprises three levels, wherein a first level network is a basic network and is used for extracting image characteristic information; the second-level network is an interframe information exchange network and is used for extracting the time characteristics of interframe information to exchange the interframe information; and the third-level network is a target detection and bounding box regression network and is used for judging the occurrence position of forest smoke and fire.
Further, the basic network adopts a VGG, Mobilene, Resnet, Eficientnet, S-Enet or Densenet network model structure; the interframe information communication network adopts an LSTM or GRU network model structure and comprises a plurality of LSTM layers or GRU layers; the target detection and bounding box regression network adopts a fast RCNN, SSD, CenterNet or YOLOv3 network model structure.
Further, training the network model comprises the following steps:
s3.1: pre-training a basic network by adopting an Imagenet data set, and training weight parameters of the basic network;
s3.2: removing a full connection layer of a basic network, connecting the interframe information communication network, connecting an average pooling layer, a full connection layer and a Softmax classifier, and training by adopting a forest smoke and fire image sample data set to obtain weight parameters of the interframe information communication network;
s3.3: and (3) removing the average pooling layer, the full connection layer and the Softmax classifier in the step (3.2), then connecting the target detection and the bounding box regression network, training again by adopting a forest smoke and fire image sample data set to obtain weight parameters of the whole neural network model, selecting a model with the optimal performance according to the test precision, and storing a weight file.
Further, the basic network adopts a Mobilenetv2 network model in a Mobilenet network.
Further, the inter-frame information communication network adopts an LSTM network, and comprises 2 LSTM layers and 1 convolutional layer.
Further, the target detection and bounding box regression network employs YOLOv 3.
Further, before the neural network model is adopted for forward calculation, a dynamic region detection algorithm is adopted for dynamically detecting forest smoke and fire images, the suspected smoke and fire region is used as a region to be detected, and a continuous multi-frame image sequence of the region to be detected is obtained.
Further, the dynamic region detection algorithm is an interframe method or a background subtraction method.
Further, in step S1, each forest firework sample includes several consecutive frames of images.
The storage medium of the present invention stores computer program instructions for performing the forest fire and smoke detection method, the instructions when executed enabling the medium to:
acquiring a forest smoke and fire image;
performing forward calculation on the image sequence by adopting the neural network model to obtain a firework region and a confidence coefficient thereof;
comparing the confidence coefficient with a confidence coefficient threshold value to judge whether a fire disaster occurs;
when dynamic detection is carried out, the medium can adopt a dynamic region detection algorithm to carry out dynamic detection on forest smoke and fire images, a suspected smoke and fire region is used as a region to be detected, and a continuous multi-frame image sequence of the region to be detected is obtained.
The device of the invention comprises one or more storage media for forest smoke and fire detection.
Has the advantages that: compared with the prior art, the invention has the following advantages:
the invention discloses a forest smoke and fire detection method based on target detection, which is used for detecting and judging forest smoke and fire based on a target detection method. The target detection method can find out local smoke points based on information of a larger area, has more complete background information, and has better robustness on a boundary between local illumination change and large contrast, so that the target detection method is relatively less influenced by the external environment, and the accuracy of smoke and fire detection is effectively improved. In the invention, a neural network model comprising a multi-level network is established, and a preferred neural network model comprises a three-level network. The built neural network model firstly extracts image characteristic information by using a basic network, then extracts time characteristics of interframe information by using an interframe information communication network to exchange interframe information, and finally uses a target detection and bounding box regression network to judge the occurrence position of forest smoke and fire. When the network model is adopted, the position of firework occurrence can be determined based on the boundary frame of the part to be detected, most of the marked part of the image is the part to be detected, the marked background part is few, the visual field is larger, the related information is more, and a model with high detection precision can be obtained by applying a small amount of samples, so that the requirement on computer hardware is effectively reduced, and the accuracy of firework in deep forest is effectively improved. In addition, the method can obtain the boundary frame of the smoke and fire area, and further judge the position of the forest fire, so that fire fighters can quickly arrive at the fire area to extinguish the fire, thereby reducing the investment of manpower and material resources and reducing the loss of personnel and property. When the equipment with the method is adopted to detect forest fire, the forest fire situation can be judged quickly and accurately, so that the forest fire can be found as early as possible, and the forest fire can be suppressed as early as possible.
Drawings
FIG. 1 is a flow chart of an implementation of a forest smoke and fire detection method based on target detection according to the present invention;
FIG. 2 is a schematic diagram of a three-level neural network architecture employed in the method of the present invention;
fig. 3 is a structural diagram of a three-level neural network constructed in the embodiment of the present invention.
Detailed Description
The invention is further described with reference to the following examples and the accompanying drawings.
Referring to fig. 1, a detailed description is given of a specific implementation process of the forest smoke and fire detection method based on target detection, the method includes that a forest smoke and fire video monitoring system or a forest smoke and fire image is removed, then a neural network model is adopted to perform forward calculation on the forest smoke and fire image to obtain a forest smoke and fire area and confidence coefficient thereof, and then forest fire situations are judged according to the confidence coefficient, and the method mainly comprises the steps of building and training the neural network model. According to the dotted frame part in fig. 1, the process of building and training the neural network model is as follows:
s1: establishing a forest firework image sample data set
Based on a forest smoke and fire monitoring system, forest smoke and fire image samples are collected through a manual collection method or an automatic collection method, and a forest smoke and fire pattern sample set is established. In order to enable context learning, the image samples need to be in the form of sequential images, that is, each sample includes multiple frames of images, for which a single sample uses N consecutive frames of images to form an image sequence, and meanwhile, when collecting the image samples, if continuous sampling is performed, in order to avoid an excessive degree of similarity between frames, one sample may be collected at intervals of several frames, specifically, the number of the intervals is selected according to circumstances.
The firework area is marked in a coordinate mode in the marking of the sample, and the marked boundary box is converted into quantitative information, wherein the quantitative information is mainly coordinate values of the lower left corner and the upper right corner of the boundary box, because the data can be used in model training, the image is marked by a marking tool usually, the boundary box information can be converted into quantitative information by a common marking tool, and the boundary box is a rectangular box and can frame the firework area.
In order to enable the forest firework sample data set to have enough representativeness, factors such as the landform, the distance, the illumination, the shooting angle of a camera and the like of a forest scene are comprehensively considered when the forest firework sample is collected, and the comprehensive coverage of the forest scene is achieved. After the forest firework sample data set is established, the forest firework sample data set is divided into a test set and a training set.
S2: constructing a neural network model including a multi-level network
The built neural network model can extract image characteristic information, extract time characteristics of interframe information, exchange the interframe information and judge the occurrence position of forest smoke and fire by using the characteristic information. In one embodiment of the invention, the neural network model comprises three levels, wherein a first level network is a basic network and is used for extracting image characteristic information; the second-level network is an interframe information exchange network and is used for extracting the time characteristics of interframe information to exchange the interframe information; the third-level network is a target detection and bounding box regression network and is used for judging the occurrence positions of forest smoke and fire, and a schematic diagram of the network structure is shown in FIG. 2.
When building a basic network, adopting a VGG, Mobile, Resnet, Eficientnet, S-Enet or Densenet network model structure, wherein the interframe information communication network adopts an LSTM or GRU network model structure and comprises a plurality of LSTM layers or GRU layers; the target detection and bounding box regression network adopts a fast RCNN, SSD, CenterNet or YOLOv3 network model structure.
For example, in an embodiment of the present invention, as shown in fig. 3, the basic network adopts a Mobilenetv2 network model of the Mobilenet network series, and in an actual network, a full connection layer needs to be removed to connect a next-level network; the interframe information communication network adopts an LSTM network model, and specifically comprises 2 LSTM layers and 1 convolutional layer; the target detection and bounding box regression network uses the YOLOv3 network model. The structure of the entire neural network model is therefore: the Mobilenetv2 removes the full connection layer, then connects 1 LSTM layer, 1 convolutional layer, 1 LSTM layer in sequence, and connects the YOLOv3 network. Since each image sample includes N frames, when calculations are performed inside the network after N successive images are input to the network model, the N images are processed in parallel, and there is information exchange between each frame. In the figure, t is the sequence number of the currently processed frame, and t-n is the sequence number of the frame n before the current frame, for example, t is 10, n is 2, then t is the sequence number of the current frame, i.e., the 10 th frame, and t-n is the sequence number of the 8 th frame.
The constructed neural network model inputs image samples of a sequence with the size of NxWxH, and parameters and confidence degrees of the boundary box of the output firework area, wherein N represents the frame number of the image samples, W represents the width of a single-frame image, and H represents the height of the single-frame image. N is determined when the sample is collected, and when N is determined, the size of the image can be represented by W × H, i.e., the size of a single frame image.
Of course, in other embodiments of the present invention, the built neural network model may take other forms, for example, the basic network adopts Mobilenet, the inter-frame communication network adopts LSTM, and the target detection and bounding box regression network adopts SSD; the basic network adopts the resource, the interframe communication network adopts the LSTM, and the target detection and bounding box regression network adopts the YOLOv 3; the basic network adopts Mobilene, the interframe communication network adopts GRU, and the target detection and bounding box regression network adopts YOLOv 3. But is not limited to the listed combinations and combinations according to the network model that can be employed at each stage can also be performed.
S3: training neural network model
After the neural network model is built, the model needs to be trained to determine the optimal weight parameters of the network model. Because the neural network model of the invention comprises multiple stages, when network training is carried out, a mode of carrying out training step by step is selected, and the method is described by combining with a three-stage network model in an embodiment, and specifically comprises the following steps:
s3.1: pre-training a basic network by adopting an Imagenet data set, and training weight parameters of the basic network;
s3.2: removing a full connection layer of a basic network, connecting the interframe information communication network, connecting an average pooling layer, a full connection layer and a Softmax classifier, and training by adopting a forest smoke and fire image sample data set to obtain weight parameters of the interframe information communication network;
s3.3: and (3) removing the average pooling layer, the full connection layer and the Softmax classifier in the step (3.2), then connecting the target detection and the bounding box regression network, training again by adopting a forest smoke and fire image sample data set to obtain weight parameters of the whole neural network model, selecting a model with the optimal performance according to the test precision, and storing a weight file.
It should be noted that, when step S3.2 is performed, since the weight parameter of the base network is basically determined, the weight parameter of the second-level network, i.e., the inter-frame information communication network, is mainly trained in step S3.2, but the parameter of the base network is also subjected to fine tuning. Similarly, in step S3.3, the parameters of the first two levels of networks are basically determined, and the third level of networks, that is, the weight parameters of the target detection and bounding box regression networks, are mainly trained, but the weight parameters of the first two levels of networks are also subjected to fine tuning. When model training is carried out, the model weight parameters are subjected to iterative optimization, and the learning rate, the iteration times and other super parameters can be adjusted according to the model precision.
In order to reduce the calculation amount, when the method is adopted, after the forest firework image is obtained, the area to be detected which is possibly a firework area can be obtained firstly. In practical application, most of the time, the image is in a static state, and a dynamic area detection method can be used as a pre-algorithm, such as an interframe method or a background subtraction method, so that a suspected smoke and fire area in a forest smoke and fire image is obtained as a to-be-detected area, wherein a motion area with large change is used as the suspected smoke and fire area, so that the calculation amount of the whole algorithm is reduced, and the real-time performance of the algorithm is enhanced. Then, the center of the area is kept unchanged, the area to be detected is expanded according to the aspect ratio of 1:1, after a continuous multi-frame image sequence of the area to be detected is obtained, a trained neural network model is used for carrying out forward calculation on the firework image, the firework area in the suspected smoke area and the confidence coefficient of the firework area can be obtained, and firework detection is completed. Typically, a confidence threshold is set, typically 0.5, and when the output confidence value is greater than the set confidence threshold, a forest fire is deemed to have occurred, whereas when the output confidence value is less than or equal to the set confidence threshold, a forest fire is deemed to have not occurred.
According to the steps, a sample data set containing 2400 sample images is constructed, the sequence length N of each sample image is 10, the size of a single frame image is 224 multiplied by 224, and a training set and a test set are divided by using a three-fold cross validation method 2:1 in proportion, namely the sample data set is divided into 3 parts, 2 parts are used as the training set, and 1 part is used as the test set. And then, finishing model training by adopting the constructed network model as shown in the figure 3, and storing a weight file. Then inputting the forest smoke and fire image into a model, determining a region to be detected by adopting a background subtraction method, then keeping the center of the region unchanged, expanding the region to be detected by using an aspect ratio of 1:1, obtaining a continuous multi-frame image sequence of the region to be detected, and then executing forward calculation by using a network model to finish smoke and fire detection. Through tests, the precision of the mAP (mean Average precision) in the example can reach 85.7%, the higher precision is achieved, and the boundary frame of the firework area is obtained, so that the occurrence position of the fire is judged.
The method of the invention is written into executable program instructions through a computer, weight files of the trained neural network model are stored in a storage medium, and when the instructions are executed, the following operations can be completed:
acquiring a forest smoke and fire image;
reading a weight file of a trained neural network model, calling the neural network model, and performing forward calculation on the image sequence to obtain a forest smoke and fire region and confidence;
comparing the confidence coefficient with a confidence coefficient threshold value to judge whether a fire disaster occurs;
when dynamic detection is carried out, a dynamic region detection algorithm can be adopted to carry out dynamic detection on forest smoke and fire images, the suspected smoke and fire region is used as a region to be detected, and a continuous multi-frame image sequence of the region to be detected is obtained.
Meanwhile, the invention also provides equipment for forest smoke and fire detection, wherein the equipment comprises one or more storage media, and in actual engineering, the equipment is directly deployed to complete forest smoke and fire detection.
The above examples are only preferred embodiments of the present invention, it should be noted that: it will be apparent to those skilled in the art that various modifications and equivalents can be made without departing from the spirit of the invention, and it is intended that all such modifications and equivalents fall within the scope of the invention as defined in the claims.
Claims (13)
1. A forest smoke and fire detection method based on target detection is characterized by comprising the steps of obtaining a forest smoke and fire image;
adopting a neural network model comprising a multistage network to perform forward calculation on the forest smoke and fire image to obtain a smoke and fire area and confidence thereof;
and comparing the confidence with a confidence threshold, and when the confidence is greater than the confidence threshold, determining that the fire is generated, and when the confidence is less than or equal to the confidence threshold, determining that the fire is not generated.
2. A forest smoke and fire detection method based on target detection as claimed in claim 1, wherein building and training the multistage neural network model comprises the following steps:
s1: establishing a sample data set of a forest firework image, carrying out boundary box labeling on a firework area in the sample image in a coordinate mode, and converting the labeled boundary box into quantitative information;
s2: constructing a neural network model comprising a multistage network, wherein the neural network model can extract image characteristic information, extract time characteristics of interframe information, exchange the interframe information and judge the occurrence position of forest smoke and fire by using the characteristic information;
s3: training a neural network model by adopting an open source data set training and the smoke and fire sample data set, determining network model parameters, and storing a weight file.
3. A forest smoke and fire detection method based on target detection as claimed in claim 2, wherein the neural network model comprises three levels, a first level network is a basic network for extracting image characteristic information; the second-level network is an interframe information exchange network and is used for extracting the time characteristics of interframe information to exchange the interframe information; and the third-level network is a target detection and bounding box regression network and is used for judging the occurrence position of forest smoke and fire.
4. A forest smoke and fire detection method based on target detection as claimed in claim 3, wherein the base network adopts a VGG, Mobilene, Resnet, Eficientnet, S-Enet or Densenet network model structure; the interframe information communication network adopts an LSTM or GRU network model structure and comprises a plurality of LSTM layers or GRU layers; the target detection and bounding box regression network adopts a fast RCNN, SSD, CenterNet or YOLOv3 network model structure.
5. A forest smoke and fire detection method based on target detection as claimed in claim 4, wherein the training of the network model comprises the following steps:
s3.1: pre-training a basic network by adopting an Imagenet data set, and training weight parameters of the basic network;
s3.2: removing a full connection layer of a basic network, connecting the interframe information communication network, connecting an average pooling layer, a full connection layer and a Softmax classifier, and training by adopting a forest smoke and fire image sample data set to obtain weight parameters of the interframe information communication network;
s3.3: and (3) removing the average pooling layer, the full connection layer and the Softmax classifier in the step (3.2), then connecting the target detection and the bounding box regression network, training again by adopting a forest smoke and fire image sample data set to obtain weight parameters of the whole neural network model, selecting a model with the optimal performance according to the test precision, and storing a weight file.
6. A forest fire and smoke detection method based on target detection as claimed in claim 4, wherein the base network adopts a Mobilenetv2 network model in a Mobilenet network.
7. A forest smoke and fire detection method based on target detection as claimed in claim 4, wherein the inter-frame information communication network adopts an LSTM network, and comprises 2 LSTM layers and 1 convolutional layer.
8. A forest smoke and fire detection method based on target detection as claimed in claim 4, wherein the target detection and bounding box regression network adopts YOLOv 3.
9. The forest smoke and fire detection method based on target detection as claimed in claim 1, wherein before the forward calculation is performed by using the neural network model, a dynamic area detection algorithm is used for dynamically detecting forest smoke and fire images, a suspected smoke and fire area is used as an area to be detected, and a continuous multi-frame image sequence of the area to be detected is obtained.
10. A forest fire and smoke detection method based on object detection as claimed in claim 9 wherein said dynamic area detection algorithm is interframe method or background subtraction method.
11. A method as claimed in claim 2, wherein each forest smoke and fire sample in step S1 comprises a number of consecutive frames of images.
12. A storage medium, characterized by storing computer program instructions for carrying out the forest fire and smoke detection method of any one of claims 1 to 11, the instructions when executed enabling the medium to:
acquiring a forest smoke and fire image;
performing forward calculation on the image sequence by adopting the neural network model to obtain a firework region and a confidence coefficient thereof;
comparing the confidence coefficient with a confidence coefficient threshold value to judge whether a fire disaster occurs;
when dynamic detection is carried out, the medium can adopt a dynamic region detection algorithm to carry out dynamic detection on forest smoke and fire images, a suspected smoke and fire region is used as a region to be detected, and a continuous multi-frame image sequence of the region to be detected is obtained.
13. A device comprising one or more storage media as claimed in claim 12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911200889.7A CN110969205A (en) | 2019-11-29 | 2019-11-29 | Forest smoke and fire detection method based on target detection, storage medium and equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911200889.7A CN110969205A (en) | 2019-11-29 | 2019-11-29 | Forest smoke and fire detection method based on target detection, storage medium and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110969205A true CN110969205A (en) | 2020-04-07 |
Family
ID=70032118
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911200889.7A Pending CN110969205A (en) | 2019-11-29 | 2019-11-29 | Forest smoke and fire detection method based on target detection, storage medium and equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110969205A (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111598843A (en) * | 2020-04-24 | 2020-08-28 | 国电南瑞科技股份有限公司 | Power transformer respirator target defect detection method based on deep learning |
| CN111914818A (en) * | 2020-09-21 | 2020-11-10 | 北京林业大学 | Forest fire smoke root node detection method based on multi-frame discrete confidence |
| CN112115941A (en) * | 2020-09-11 | 2020-12-22 | 北京锐安科技有限公司 | Fire detection method, device, equipment and storage medium |
| CN112132090A (en) * | 2020-09-28 | 2020-12-25 | 天地伟业技术有限公司 | Smoke and fire automatic detection and early warning method based on YOLOV3 |
| CN112149583A (en) * | 2020-09-27 | 2020-12-29 | 山东产研鲲云人工智能研究院有限公司 | Smoke detection method, terminal device and storage medium |
| CN112309068A (en) * | 2020-10-29 | 2021-02-02 | 电子科技大学中山学院 | Forest fire early warning method based on deep learning |
| CN112766206A (en) * | 2021-01-28 | 2021-05-07 | 深圳市捷顺科技实业股份有限公司 | High-order video vehicle detection method and device, electronic equipment and storage medium |
| CN112801148A (en) * | 2021-01-14 | 2021-05-14 | 西安电子科技大学 | Fire recognition and positioning system and method based on deep learning |
| CN112861737A (en) * | 2021-02-11 | 2021-05-28 | 西北工业大学 | Forest fire smoke detection method based on image dark channel and YoLov3 |
| CN112906481A (en) * | 2021-01-23 | 2021-06-04 | 招商新智科技有限公司 | Method for realizing forest fire detection based on unmanned aerial vehicle |
| CN113515989A (en) * | 2020-07-20 | 2021-10-19 | 阿里巴巴集团控股有限公司 | Moving object, smoke and fire detection method, device and storage medium |
| CN113822368A (en) * | 2021-09-29 | 2021-12-21 | 成都信息工程大学 | Anchor-free incremental target detection method |
| CN114078218A (en) * | 2021-11-24 | 2022-02-22 | 南京林业大学 | Self-adaptive fusion forest smoke and fire identification data augmentation method |
| CN116091935A (en) * | 2023-04-07 | 2023-05-09 | 四川三思德科技有限公司 | Forest fire and smoke alarm agriculture operation interference resistance processing method, device and medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180096457A1 (en) * | 2016-09-08 | 2018-04-05 | Carnegie Mellon University | Methods and Software For Detecting Objects in Images Using a Multiscale Fast Region-Based Convolutional Neural Network |
| CN110378265A (en) * | 2019-07-08 | 2019-10-25 | 创新奇智(成都)科技有限公司 | A kind of incipient fire detection method, computer-readable medium and system |
| CN110443969A (en) * | 2018-05-03 | 2019-11-12 | 中移(苏州)软件技术有限公司 | A kind of fire point detecting method, device, electronic equipment and storage medium |
| CN110490043A (en) * | 2019-06-10 | 2019-11-22 | 东南大学 | A kind of forest rocket detection method based on region division and feature extraction |
-
2019
- 2019-11-29 CN CN201911200889.7A patent/CN110969205A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180096457A1 (en) * | 2016-09-08 | 2018-04-05 | Carnegie Mellon University | Methods and Software For Detecting Objects in Images Using a Multiscale Fast Region-Based Convolutional Neural Network |
| CN110443969A (en) * | 2018-05-03 | 2019-11-12 | 中移(苏州)软件技术有限公司 | A kind of fire point detecting method, device, electronic equipment and storage medium |
| CN110490043A (en) * | 2019-06-10 | 2019-11-22 | 东南大学 | A kind of forest rocket detection method based on region division and feature extraction |
| CN110378265A (en) * | 2019-07-08 | 2019-10-25 | 创新奇智(成都)科技有限公司 | A kind of incipient fire detection method, computer-readable medium and system |
Non-Patent Citations (2)
| Title |
|---|
| MOHAMMADREZA ZOLFAGHARI等: "《ECO: Efficient Convolutional Network for Online Video Understanding》", pages 1 - 24 * |
| 黄心汉: "《微装配机器人》", 国防工业出版社, pages: 112 - 117 * |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111598843A (en) * | 2020-04-24 | 2020-08-28 | 国电南瑞科技股份有限公司 | Power transformer respirator target defect detection method based on deep learning |
| CN111598843B (en) * | 2020-04-24 | 2022-11-11 | 国电南瑞科技股份有限公司 | Power transformer respirator target defect detection method based on deep learning |
| CN113515989A (en) * | 2020-07-20 | 2021-10-19 | 阿里巴巴集团控股有限公司 | Moving object, smoke and fire detection method, device and storage medium |
| CN112115941A (en) * | 2020-09-11 | 2020-12-22 | 北京锐安科技有限公司 | Fire detection method, device, equipment and storage medium |
| CN112115941B (en) * | 2020-09-11 | 2023-12-05 | 北京锐安科技有限公司 | Fire detection method, device, equipment and storage medium |
| CN111914818A (en) * | 2020-09-21 | 2020-11-10 | 北京林业大学 | Forest fire smoke root node detection method based on multi-frame discrete confidence |
| CN111914818B (en) * | 2020-09-21 | 2024-05-24 | 北京林业大学 | Method for detecting forest fire smoke root nodes based on multi-frame discrete confidence |
| CN112149583A (en) * | 2020-09-27 | 2020-12-29 | 山东产研鲲云人工智能研究院有限公司 | Smoke detection method, terminal device and storage medium |
| CN112132090A (en) * | 2020-09-28 | 2020-12-25 | 天地伟业技术有限公司 | Smoke and fire automatic detection and early warning method based on YOLOV3 |
| CN112309068A (en) * | 2020-10-29 | 2021-02-02 | 电子科技大学中山学院 | Forest fire early warning method based on deep learning |
| CN112801148A (en) * | 2021-01-14 | 2021-05-14 | 西安电子科技大学 | Fire recognition and positioning system and method based on deep learning |
| CN112906481A (en) * | 2021-01-23 | 2021-06-04 | 招商新智科技有限公司 | Method for realizing forest fire detection based on unmanned aerial vehicle |
| CN112766206A (en) * | 2021-01-28 | 2021-05-07 | 深圳市捷顺科技实业股份有限公司 | High-order video vehicle detection method and device, electronic equipment and storage medium |
| CN112766206B (en) * | 2021-01-28 | 2024-05-28 | 深圳市捷顺科技实业股份有限公司 | High-order video vehicle detection method and device, electronic equipment and storage medium |
| CN112861737A (en) * | 2021-02-11 | 2021-05-28 | 西北工业大学 | Forest fire smoke detection method based on image dark channel and YoLov3 |
| CN113822368A (en) * | 2021-09-29 | 2021-12-21 | 成都信息工程大学 | Anchor-free incremental target detection method |
| CN114078218A (en) * | 2021-11-24 | 2022-02-22 | 南京林业大学 | Self-adaptive fusion forest smoke and fire identification data augmentation method |
| CN114078218B (en) * | 2021-11-24 | 2024-03-29 | 南京林业大学 | Adaptive fusion forest smoke and fire identification data augmentation method |
| CN116091935A (en) * | 2023-04-07 | 2023-05-09 | 四川三思德科技有限公司 | Forest fire and smoke alarm agriculture operation interference resistance processing method, device and medium |
| CN116091935B (en) * | 2023-04-07 | 2023-08-01 | 四川三思德科技有限公司 | Forest fire and smoke alarm agriculture operation interference resistance processing method, device and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110969205A (en) | Forest smoke and fire detection method based on target detection, storage medium and equipment | |
| CN107609470B (en) | Method for detecting early smoke of field fire by video | |
| CN110688925B (en) | Cascade target identification method and system based on deep learning | |
| CN111062429A (en) | Chef cap and mask wearing detection method based on deep learning | |
| CN111626188B (en) | Indoor uncontrollable open fire monitoring method and system | |
| KR102149832B1 (en) | Automated Violence Detecting System based on Deep Learning | |
| CN113850242B (en) | Storage abnormal target detection method and system based on deep learning algorithm | |
| CN111814638A (en) | Security scene flame detection method based on deep learning | |
| Wang et al. | Early forest fire region segmentation based on deep learning | |
| CN111079572A (en) | Forest smoke and fire detection method based on video understanding, storage medium and equipment | |
| CN105139429A (en) | Fire detecting method based on flame salient picture and spatial pyramid histogram | |
| CN112257643A (en) | Smoking behavior and calling behavior identification method based on video streaming | |
| Hongyu et al. | An improved multi-scale fire detection method based on convolutional neural network | |
| CN111062950A (en) | Method, storage medium and equipment for multi-class forest scene image segmentation | |
| CN115761627A (en) | Fire smoke flame image identification method | |
| CN111898419A (en) | Partition landslide detection system and method based on cascade deep convolutional neural network | |
| CN115719463A (en) | Smoke and fire detection method based on super-resolution reconstruction and adaptive extrusion excitation | |
| CN112836608B (en) | Forest fire source estimation model training method, estimation method and system | |
| Agrawal et al. | Classification of natural disaster using satellite & drone images with CNN using transfer learning | |
| CN114399734A (en) | Forest fire early warning method based on visual information | |
| CN114821486B (en) | Personnel identification method in power operation scene | |
| CN111126230A (en) | Smoke concentration quantitative evaluation method and electronic equipment applying same | |
| CN115311601A (en) | Fire detection analysis method based on video analysis technology | |
| Zhan et al. | A High-precision Forest Fire Smoke Detection Approach Based on DRGNet to Remote Sensing Through Uavs | |
| CN115546682A (en) | Dynamic smoke detection method based on video |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |