CN112906463A - Image-based fire detection method, device, equipment and storage medium - Google Patents
Image-based fire detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112906463A CN112906463A CN202110051354.9A CN202110051354A CN112906463A CN 112906463 A CN112906463 A CN 112906463A CN 202110051354 A CN202110051354 A CN 202110051354A CN 112906463 A CN112906463 A CN 112906463A
- Authority
- CN
- China
- Prior art keywords
- image
- fire
- smoke
- enhanced
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 146
- 239000000779 smoke Substances 0.000 claims abstract description 101
- 238000012544 monitoring process Methods 0.000 claims abstract description 60
- 238000012549 training Methods 0.000 claims abstract description 32
- 238000002372 labelling Methods 0.000 claims abstract description 16
- 238000010586 diagram Methods 0.000 claims description 42
- 238000000605 extraction Methods 0.000 claims description 22
- 230000006870 function Effects 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 18
- 238000010606 normalization Methods 0.000 claims description 13
- 238000006243 chemical reaction Methods 0.000 claims description 10
- 238000009795 derivation Methods 0.000 claims description 6
- 238000009432 framing Methods 0.000 claims description 5
- 238000000034 method Methods 0.000 abstract description 17
- 230000001965 increasing effect Effects 0.000 description 8
- 238000004364 calculation method Methods 0.000 description 6
- 238000011176 pooling Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 239000003242 anti bacterial agent Substances 0.000 description 2
- 229940088710 antibiotic agent Drugs 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 235000019504 cigarettes Nutrition 0.000 description 1
- 238000002485 combustion reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000002087 whitening effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/60—Rotation of whole images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/90—Dynamic range modification of images or parts thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30232—Surveillance
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Health & Medical Sciences (AREA)
- Fire-Detection Mechanisms (AREA)
Abstract
The invention relates to the technical field of video monitoring, and discloses a fire detection method, device, equipment and storage medium based on images. The method comprises the following steps: collecting a plurality of sample images containing smoke and fire, and labeling smoke and fire areas in the sample images to obtain labeled images; generating an initial data set according to the marked images, and performing data enhancement on the initial data set to obtain an enhanced data set, wherein the enhanced data set comprises a plurality of enhanced marked images; inputting the enhanced annotation image into a preset YOLOv3 model for training to obtain a smoke and fire detection model; acquiring a monitoring video to be detected, inputting the monitoring video into a smoke and fire detection model for frame-by-frame detection, and outputting a detection result; and if the detection result is that smoke and/or fire exist in the current video frame, transmitting the video frame with the smoke and/or fire information to the monitoring terminal for fire early warning. The invention improves the accuracy of fire detection by detecting the monitoring video in real time.
Description
Technical Field
The invention relates to the technical field of video monitoring, in particular to a fire detection method, a fire detection device, fire detection equipment and a storage medium based on images.
Background
The parcel quantity is huge every day in express delivery trade, and equipment is numerous, the condition that personnel are irregular appears, leads to the emergence of conflagration to be extremely dangerous thing, and the loss that brings is inestimable. The effect of real-time monitoring video is required to be fully utilized for effective control, so that the problems are early discovered and timely solved. With the rapid development of computer technology, the computer technology is utilized to detect the fire, so that the influence of the environment can be overcome, the fire condition can be responded quickly and timely, the real-time condition of the fire scene can be clearly provided, and the rescue personnel can conveniently handle the fire condition
The traditional fire detection technology generally detects fire based on the characteristics of color, size and the like of fire. However, such methods based on the given flame characteristics have poor interference resistance and poor generalization capability, and the occurrence scene, combustion form, form of smoke generated therewith, and the like of the flame have diversity and are easily affected by the environment, so that the false alarm rate of the detection algorithm is high in different scenes.
Disclosure of Invention
The invention mainly aims to solve the technical problem of high error rate of the existing fire detection.
The invention provides a fire detection method based on an image, which comprises the following steps:
collecting a plurality of sample images containing smoke and fire, and labeling smoke and fire areas in the sample images to obtain labeled images;
generating an initial data set according to the annotation images, and performing data enhancement on the initial data set to obtain an enhanced data set, wherein the enhanced data set comprises a plurality of enhanced annotation images;
inputting the enhanced annotation image into a preset YOLOv3 model for training to obtain a smoke and fire detection model, wherein the YOLOv3 model comprises: darknet-53 network, Batch nonilization layer, LeakyReLU layer and convolutional layer;
acquiring a monitoring video to be detected, inputting the monitoring video into the smoke and fire light detection model for frame-by-frame detection, and outputting a detection result;
and if the detection result indicates that smoke and/or fire exist in the current video frame, transmitting the video frame marked with the smoke and/or fire information to a monitoring terminal for fire early warning.
Optionally, in a first implementation manner of the first aspect of the present invention, the acquiring a plurality of sample images including smoke and flare, and labeling the smoke and flare areas in the sample images to obtain a labeled image includes:
collecting a plurality of sample images containing smoke and fire;
and calling a preset Labellmg tool, selecting smoke and fire regions in the sample image to obtain a sample region image, and carrying out region information annotation on the sample region image to obtain an annotated image.
Optionally, in a second implementation manner of the first aspect of the present invention, the generating an initial data set according to the annotation image, and performing data enhancement on the initial data set to obtain an enhanced data set includes:
converting the marked image into a JPG format to obtain a JPG image;
storing the JPG image into a preset folder to generate an initial data set;
reading a plurality of JPG images in the initial data set, and sequentially turning, zooming and adjusting the color gamut of each image to obtain a plurality of standard annotation images;
splicing the standard annotation images based on a preset splicing rule to obtain a plurality of enhanced annotation images;
and storing the enhanced annotation images in a VOC format to obtain an enhanced data set.
Optionally, in a third implementation manner of the first aspect of the present invention, the inputting the enhanced annotation image into a preset YOLOv3 model for training to obtain a smoke and fire detection model includes:
inputting the enhanced annotation image into the Darknet-53 network for feature extraction to obtain a first feature map;
inputting the first feature map into the Batch nonillization layer for normalization processing to obtain a normalized first feature map;
inputting the normalized first characteristic diagram into the LeakyReLU layer for nonlinear conversion to obtain a second characteristic diagram;
inputting the second characteristic diagram into the convolutional layer for pixel point prediction to obtain a prediction result corresponding to the second characteristic diagram;
calling a preset loss function, and calculating a prediction result corresponding to the second feature map and a loss value of the enhanced labeled image;
and adjusting parameters of the YOLOv3 model according to the loss value until the YOLOv3 model converges to obtain a smoke and fire detection model.
Optionally, in a fourth implementation manner of the first aspect of the present invention, the invoking a preset loss function, and calculating the prediction result corresponding to the second feature map and the loss value of the enhanced labeled image includes:
calling a preset Focal local function, and calculating a prediction result corresponding to the second feature map and the backward gradient of the enhanced labeled image through the Focal local function to obtain a backward gradient value;
and carrying out derivation operation on the backward gradient value to obtain a prediction result corresponding to the second characteristic diagram and a loss value of the enhanced labeled image.
Optionally, in a fifth implementation manner of the first aspect of the present invention, the acquiring a surveillance video to be detected, and inputting the surveillance video into the smoke and fire detection model for frame-by-frame detection, and outputting a detection result includes:
acquiring a monitoring video to be detected;
inputting the monitoring video into the Darknet-53 network for feature extraction frame by frame to obtain a third feature map;
inputting the third feature map into the Batch nonillization layer for normalization processing to obtain a normalized third feature map;
inputting the normalized third feature map into the LeakyReLU layer for nonlinear conversion to obtain a fourth feature map;
and inputting the fourth characteristic diagram into the convolutional layer for pixel point prediction to obtain a detection result corresponding to the fourth characteristic diagram.
Optionally, in a sixth implementation manner of the first aspect of the present invention, the inputting the monitoring video into the Darknet-53 network to perform feature extraction frame by frame, and obtaining a third feature map includes:
framing the monitoring video based on a preset FFmpeg frame to obtain a plurality of video frames;
sequentially denoising, contrast enhancement, brightness and saturation adjustment are carried out on each video frame to obtain a plurality of standard video frames;
and sequentially inputting the standard video frames into the Darknet-53 network for feature extraction to obtain third feature maps corresponding to the standard video frames.
A second aspect of the present invention provides an image-based fire detection apparatus, comprising:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a plurality of sample images containing smoke and fire, and labeling smoke and fire areas in the sample images to obtain labeled images;
the data enhancement module is used for generating an initial data set according to the annotation images and carrying out data enhancement on the initial data set to obtain an enhanced data set, wherein the enhanced data set comprises a plurality of enhanced annotation images;
a model training module, configured to input the enhanced annotation image into a preset YOLOv3 model for training, so as to obtain a smoke and fire detection model, where the YOLOv3 model includes: darknet-53 network, Batch nonilization layer, LeakyReLU layer and convolutional layer;
the detection module is used for acquiring a monitoring video to be detected, inputting the monitoring video into the smoke and fire light detection model for frame-by-frame detection and outputting a detection result;
and the fire early warning module is used for transmitting the video frame marked with the smoke and/or fire information to the monitoring terminal for fire early warning if the detection result indicates that smoke and/or fire exists in the current video frame.
Optionally, in a first implementation manner of the second aspect of the present invention, the acquisition module is specifically configured to:
collecting a plurality of sample images containing smoke and fire;
and calling a preset Labellmg tool, selecting smoke and fire regions in the sample image to obtain a sample region image, and carrying out region information annotation on the sample region image to obtain an annotated image.
Optionally, in a second implementation manner of the second aspect of the present invention, the data enhancement module is specifically configured to:
converting the marked image into a JPG format to obtain a JPG image;
storing the JPG image into a preset folder to generate an initial data set;
reading a plurality of JPG images in the initial data set, and sequentially turning, zooming and adjusting the color gamut of each image to obtain a plurality of standard annotation images;
splicing the standard annotation images based on a preset splicing rule to obtain a plurality of enhanced annotation images;
and storing the enhanced annotation images in a VOC format to obtain an enhanced data set.
Optionally, in a third implementation manner of the second aspect of the present invention, the model training module includes:
the network training unit is used for inputting the enhanced annotation image into the Darknet-53 network for feature extraction to obtain a first feature map; inputting the first feature map into the Batch nonillization layer for normalization processing to obtain a normalized first feature map; inputting the normalized first characteristic diagram into the LeakyReLU layer for nonlinear conversion to obtain a second characteristic diagram; inputting the second characteristic diagram into the convolutional layer for pixel point prediction to obtain a prediction result corresponding to the second characteristic diagram;
the loss calculation unit is used for calling a preset loss function and calculating a prediction result corresponding to the second feature map and a loss value of the enhanced annotation image;
and the parameter adjusting and optimizing unit is used for adjusting the parameters of the YOLOv3 model according to the loss value until the YOLOv3 model converges to obtain a smoke and fire detection model.
Optionally, in a fourth implementation manner of the second aspect of the present invention, the loss calculating unit is specifically configured to:
calling a preset Focal local function, and calculating a prediction result corresponding to the second feature map and the backward gradient of the enhanced labeled image through the Focal local function to obtain a backward gradient value;
and carrying out derivation operation on the backward gradient value to obtain a prediction result corresponding to the second characteristic diagram and a loss value of the enhanced labeled image.
Optionally, in a fifth implementation manner of the second aspect of the present invention, the detecting module includes:
the acquisition unit is used for acquiring a monitoring video to be detected;
the feature extraction unit is used for inputting the monitoring video into the Darknet-53 network to carry out feature extraction frame by frame to obtain a third feature map;
the feature processing unit is used for inputting the third feature map into the Batch nonillization layer for normalization processing to obtain a normalized third feature map; inputting the normalized third feature map into the LeakyReLU layer for nonlinear conversion to obtain a fourth feature map; and inputting the fourth characteristic diagram into the convolutional layer for pixel point prediction to obtain a detection result corresponding to the fourth characteristic diagram.
Optionally, in a sixth implementation manner of the second aspect of the present invention, the feature extraction unit is specifically configured to:
framing the monitoring video based on a preset FFmpeg frame to obtain a plurality of video frames;
sequentially denoising, contrast enhancement, brightness and saturation adjustment are carried out on each video frame to obtain a plurality of standard video frames;
and sequentially inputting the standard video frames into the Darknet-53 network for feature extraction to obtain third feature maps corresponding to the standard video frames.
A third aspect of the present invention provides an image-based fire detection apparatus, comprising: a memory and at least one processor, the memory having instructions stored therein; the at least one processor invokes the instructions in the memory to cause the image-based fire detection apparatus to perform the image-based fire detection method described above.
A fourth aspect of the present invention provides a computer-readable storage medium having stored therein instructions, which, when run on a computer, cause the computer to perform the image-based fire detection method described above.
In the technical scheme provided by the invention, a model for detecting smoke and fire is trained by adopting a YOLOv3 model, the data set is subjected to data enhancement, the detection background is enriched, the data of a plurality of images can be simultaneously calculated during calculation, when a real-time monitoring video is detected, the monitoring video is input into a smoke and fire detection model for frame-by-frame detection, the detection result is output, and when the detection result indicates that smoke and/or fire exist in the current video frame, the video frame marked with smoke and/or fire information is transmitted to a monitoring terminal for fire early warning. The invention improves the accuracy of fire detection by detecting the monitoring video in real time.
Drawings
FIG. 1 is a schematic diagram of a first embodiment of a method for image-based fire detection in accordance with an embodiment of the present invention;
FIG. 2 is a schematic diagram of a second embodiment of the image-based fire detection method in an embodiment of the present invention;
FIG. 3 is a schematic diagram of a third embodiment of a fire detection method based on images according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of an embodiment of an image-based fire detection apparatus according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of an embodiment of an image-based fire detection apparatus according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides a fire detection method, a fire detection device, fire detection equipment and a storage medium based on an image. The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Furthermore, the terms "comprises," "comprising," or "having," and any variations thereof, are intended to cover non-exclusive inclusions, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
For ease of understanding, a detailed flow of an embodiment of the present invention is described below, and referring to fig. 1, a first embodiment of a fire detection method based on an image in an embodiment of the present invention includes:
101. collecting a plurality of sample images containing smoke and fire, and labeling smoke and fire areas in the sample images to obtain labeled images;
it is to be understood that the executing subject of the present invention may be an image-based fire detection device, and may also be a terminal or a server, which is not limited herein. The embodiment of the present invention is described by taking a server as an execution subject.
In this embodiment, a plurality of smoke and fire images are collected, for example: the image of the smoke of the cigarette is that the smoke and the fire light are large and small. In this example, a Labellmg tool is used to label the sample image.
Optionally, in an embodiment, the acquiring a plurality of sample images including smoke and fire, and labeling smoke and fire regions in the sample images to obtain a labeled image includes:
collecting a plurality of sample images containing smoke and fire;
and calling a preset Labellmg tool, selecting smoke and fire regions in the sample image to obtain a sample region image, and carrying out region information annotation on the sample region image to obtain an annotated image.
In this embodiment, the Labellmg tool first executes an open file command, then selects the smoke and flare regions in the sample image, inputs the name of the category in the create rechbox, and finally generates the xml file under the path of the saved file, where the name of the xml file is consistent with the name of the labeled picture, and when the image is labeled, the name of the category should be a lower case letter, for example: cars use car.
102. Generating an initial data set according to the annotation images, and performing data enhancement on the initial data set to obtain an enhanced data set, wherein the enhanced data set comprises a plurality of enhanced annotation images;
in this embodiment, a folder is newly created to store the whole data set, for example: VOC2007, then like VOC2007, the following folders are newly created inside the folder: antotions, images, JPEGimages, SegmentationClass, and SegmentationObjects. Putting the sample images into JPEGImages, then renaming the images into a form of '000005. jpg' of VOC2007, storing the annotation files into antibiotics, obtaining an XML file by each image and annotation, arranging one sample image in a JPEGImages folder corresponding to one same name XML file in the antibiotics, and arranging the sample images in a consistent name. And newly creating a folder named as Main in ImageSets, and generating four txt files in the Main folder, namely: txt is the test set; txt is the training set; txt is the verification set; txt is the training and validation set, where train is 50% of the entire data set and test is 50% of the entire data set; train is 50% of trainval, which is 50% of trainval. the content in the txt file is the name of the sample image (without suffix). For example: 000002. according to the generated xml file, making a train entry.txt in the initial data set; txt; txt; txt.
103. Inputting the enhanced annotation image into a preset YOLOv3 model for training to obtain a smoke and fire detection model, wherein the YOLOv3 model comprises: darknet-53 network, Batch nonilization layer, LeakyReLU layer and convolutional layer;
in this embodiment, the YOLOv3 model is a model combining classification and detection, and is used for object classification and object detection, and the initial purpose of establishing the model is to improve the training speed of the model on the premise of ensuring the accuracy. The Yolov3 model consists of a Darknet-53 network, a Batch nonilization layer, a LeakyReLU layer, and a convolutional layer connected together. Inputting the enhanced annotation image into the Darknet-53 network, and extracting a corresponding target feature map; and then, after each convolution, deepening, thinning and correcting the characteristics of the target characteristic diagram. And the Batch nonilization layer is used for carrying out Batch normalization processing on the target feature map generated from each convolution layer in the Darknet-53 network, and carrying out normalization processing on the data of the feature map so as to improve the convergence of the model. The LeakyReLU layer connects the last pooling layer and the last convolution layer of the Darknet-53 network, superposes the corresponding pooled or convolved feature maps, further expands the resolution of the features and refines the fine feature granularity. The convolution layer is used for predicting the position of a feature frame in a target feature map, and the output data format of the convolution layer is the prior frame number x (5+ classification number), in the YOLOv3 model, the prior frame number is 5, the classification number is 20-bit characters, and 5 in brackets represents the central two-dimensional coordinate, the width and height dimension and the confidence coefficient of a feature map boundary frame, wherein the confidence coefficient of the feature map boundary frame is represented by an IOU (Intersection over unit).
104. Acquiring a monitoring video to be detected, inputting the monitoring video into the smoke and fire light detection model for frame-by-frame detection, and outputting a detection result;
in this embodiment, the surveillance video to be detected is mainly a surveillance video of an express storage warehouse or a warehouse surveillance video in a transportation process. And if the fire exists, transmitting the video frame marked with the smoke and/or fire information to a monitoring terminal for fire early warning.
Optionally, in an embodiment, the obtaining a surveillance video to be detected, and inputting the surveillance video into the smoke and fire detection model for frame-by-frame detection, and outputting a detection result includes:
acquiring a monitoring video to be detected;
inputting the monitoring video into the Darknet-53 network for feature extraction frame by frame to obtain a third feature map;
inputting the third feature map into the Batch nonillization layer for normalization processing to obtain a normalized third feature map;
inputting the normalized third feature map into the LeakyReLU layer for nonlinear conversion to obtain a fourth feature map;
and inputting the fourth characteristic diagram into the convolutional layer for pixel point prediction to obtain a detection result corresponding to the fourth characteristic diagram.
In this embodiment, the enhanced labeled image of the Darknet-53 network is provided with a prior frame, a third feature map of the enhanced labeled image is extracted according to the prior frame, the number of features of the third feature map is increased after each convolution, the number of channels is increased after each pooling, and the fine granularity of the channels is refined, so that the feature depth of the first feature map is increased. After each convolution, calculating the offset of the prior frame by comparing the central coordinates and the width and height sizes of the prior frame and the feature picture frame, correspondingly adjusting the central coordinates and the area size of the prior frame, and gradually optimizing the target detection accuracy of the feature picture.
Optionally, in an embodiment, the inputting the monitoring video into the Darknet-53 network to perform feature extraction frame by frame, and obtaining a third feature map includes:
framing the monitoring video based on a preset FFmpeg frame to obtain a plurality of video frames;
sequentially denoising, contrast enhancement, brightness and saturation adjustment are carried out on each video frame to obtain a plurality of standard video frames;
and sequentially inputting the standard video frames into the Darknet-53 network for feature extraction to obtain third feature maps corresponding to the standard video frames.
In this embodiment, the FFmpeg frame includes encoder and decoder, and the FFmpeg frame carries out video coding and decoding to the surveillance video through encoder and decoder to the realization divides the frame to the surveillance video, and the video frame noise point that obtains is more, consequently need get rid of the noise point to the video frame, promptly falls and makes an uproar, carries out contrast enhancement again, and the video frame that luminance and saturation adjustment obtained is more clear, and the follow-up discernment when examining that makes is higher.
105. And if the detection result indicates that smoke and/or fire exist in the current video frame, transmitting the video frame marked with the smoke and/or fire information to a monitoring terminal for fire early warning.
In this embodiment, when the monitoring terminal receives the video frame marked with the smoke and/or fire information, the monitoring terminal sends out warning information and prompts that there is a danger of smoke and fire.
In the embodiment of the invention, a model for detecting smoke and fire is trained by adopting a YOLOv3 model, the data set is subjected to data enhancement, the detection background is enriched, the data of a plurality of images can be simultaneously calculated during calculation, when a real-time monitoring video is detected, the monitoring video is input into a smoke and fire detection model to be detected frame by frame, the detection result is output, and when the detection result is that smoke and/or fire exists in the current video frame, the video frame marked with smoke and/or fire information is transmitted to a monitoring terminal to carry out fire early warning. The invention improves the accuracy of fire detection by detecting the monitoring video in real time.
Referring to fig. 2, a second embodiment of the fire detection method based on images according to the embodiment of the present invention includes:
201. collecting a plurality of sample images containing smoke and fire, and labeling smoke and fire areas in the sample images to obtain labeled images;
202. converting the marked image into a JPG format to obtain a JPG image;
203. storing the JPG image into a preset folder to generate an initial data set;
204. reading a plurality of JPG images in the initial data set, and sequentially turning, zooming and adjusting the color gamut of each image to obtain a plurality of standard annotation images;
in this embodiment, the flipping is to transform the original image pixels in the position space, the flipping is to perform mirror image operation on the original image, and the flipping mainly includes horizontal mirror image flipping, vertical mirror image flipping, and origin mirror image flipping, and selects a corresponding flipping operation in combination with a data format, for example: the data set is car image data, and the training set test set is all pictures taken normally, and only horizontal mirroring operation is used at the moment. The image can be zoomed outwards or inwards, when zoomed outwards, the final image size is larger than the original image size, and in order to keep the size of the original image, the image with the same size as the original image is cut out from the zoomed image in combination with cutting out. Another method is inward scaling, which reduces the image size to a preset size. Scaling also brings some problems, for example, the aspect ratio difference between the scaled image size and the original image size is large, the image frame loss phenomenon occurs, the final result is influenced to a certain extent in the experiment, scaling needs to be performed in equal proportion, and edge filling is performed on the insufficient place. The JPG image usually performs some color gamut filling operations to expand a data set, wherein the color gamut filling is mainly used for enhancing the color of the image, and the brightness, the saturation and the contrast of the image are mainly adjusted.
205. Splicing the standard annotation images based on a preset splicing rule to obtain a plurality of enhanced annotation images;
206. storing each enhanced annotation image in a VOC format to obtain an enhanced data set;
in the embodiment, mosaics data enhancement is adopted, four images are spliced, each image is provided with a corresponding labeling frame, a new image is obtained after the four images are spliced, the labeling frame corresponding to the image is obtained at the same time, and then the new image is input into a neural network to learn, namely, the four images are transmitted into the neural network for learning at one time, so that the backgrounds of detected objects are enriched, and the data of the four images can be calculated at one time during calculation.
207. Inputting the enhanced annotation image into a preset YOLOv3 model for training to obtain a smoke and fire detection model, wherein the YOLOv3 model comprises: darknet-53 network, Batch nonilization layer, LeakyReLU layer and convolutional layer;
208. acquiring a monitoring video to be detected, inputting the monitoring video into the smoke and fire light detection model for frame-by-frame detection, and outputting a detection result;
209. and if the detection result indicates that smoke and/or fire exist in the current video frame, transmitting the video frame marked with the smoke and/or fire information to a monitoring terminal for fire early warning.
In the embodiment of the invention, the sample image with the labeling information is subjected to data enhancement, and the obtained enhanced labeling image is trained, so that the model can process a plurality of data at one time, and the overall detection and identification process is simplified; reducing the calculation amount of the model; thereby increasing the efficiency of identifying the smoke and fire images.
Referring to fig. 3, a third embodiment of the fire detection method based on images according to the embodiment of the present invention includes:
301. collecting a plurality of sample images containing smoke and fire, and labeling smoke and fire areas in the sample images to obtain labeled images;
302. generating an initial data set according to the annotation images, and performing data enhancement on the initial data set to obtain an enhanced data set, wherein the enhanced data set comprises a plurality of enhanced annotation images;
303. inputting the enhanced annotation image into the Darknet-53 network for feature extraction to obtain a first feature map;
in this embodiment, the size of the last first feature map obtained by the Darknet-53 network convolution is 26 × 26, where the first feature map obtained by the last convolution layer and the first feature map obtained by the last pooling layer are subjected to global feature fusion, the size of the first feature map is superimposed from 26 × 26 to 13 × 13, and the number of channels is increased.
304. Inputting the first feature map into the Batch nonillization layer for normalization processing to obtain a normalized first feature map;
in this embodiment, a Batch nonillization layer is followed after the Darknet-53 network, the first feature map obtained by training is normalized, the generalization capability of the network is increased, the adaptation degree of the training data to the training network is increased, and then the next convolution layer or pooling layer is input, so that the convergence of the model can be improved, the dependence on regularization is reduced, overfitting can be prevented, and the detection efficiency of the model is improved. Specifically, whitening preprocessing is performed on training data in the first feature map, so that correlation among features of the training data is eliminated, and then change reconstruction is performed on the training data to recover feature distribution of the training data.
305. Inputting the normalized first characteristic diagram into the LeakyReLU layer for nonlinear conversion to obtain a second characteristic diagram;
306. inputting the second characteristic diagram into the convolutional layer for pixel point prediction to obtain a prediction result corresponding to the second characteristic diagram;
in this embodiment, when a prior frame is constructed, a target feature in each prior frame in an enhanced labeled image is labeled, and a detection result is obtained by predicting a category of the target feature by comparing labeled information with data information corresponding to the target feature, where the data format of each feature category is four coordinate values, one confidence and 20 category values in the prior frame, and the four coordinate values are a central two-dimensional coordinate of the prior frame and a width and height size of an area, respectively.
307. Calling a preset loss function, and calculating a prediction result corresponding to the second feature map and a loss value of the enhanced labeled image;
optionally, in an embodiment, the calling a preset loss function, and calculating the prediction result corresponding to the second feature map and the loss value of the enhanced labeled image includes:
calling a preset Focal local function, and calculating a prediction result corresponding to the second feature map and the backward gradient of the enhanced labeled image through the Focal local function to obtain a backward gradient value;
and carrying out derivation operation on the backward gradient value to obtain a prediction result corresponding to the second characteristic diagram and a loss value of the enhanced labeled image.
In this embodiment, the Focal length mainly solves the problem of serious imbalance of the positive and negative sample ratios in target detection. The loss function reduces the weight of a large number of simple negative samples in training, a large number of negative samples are filtered through scoring and nms screening in a candidate frame stage, then the proportion of the positive samples and the negative samples is fixed in a classification regression stage, the prediction result corresponding to the second feature map and the backward gradient of the enhanced labeled image are calculated, after the backward gradient value is obtained, derivation operation is carried out on the backward gradient value, and the prediction result corresponding to the second feature map and the loss value of the enhanced labeled image are obtained.
308. Adjusting parameters of the YOLOv3 model according to the loss value until the YOLOv3 model converges to obtain a smoke and fire detection model;
309. acquiring a monitoring video to be detected, inputting the monitoring video into the smoke and fire light detection model for frame-by-frame detection, and outputting a detection result;
310. and if the detection result indicates that smoke and/or fire exist in the current video frame, transmitting the video frame marked with the smoke and/or fire information to a monitoring terminal for fire early warning.
In the embodiment of the invention, a training process of a smoke and fire detection model is described in detail, a sample image is labeled, a labeled image is subjected to data enhancement, a first feature image corresponding to a prior frame is extracted from the image subjected to data enhancement, the model convergence is accelerated after the first feature image is subjected to normalization processing, then the first feature images with different resolutions are subjected to associated superposition to obtain a second feature image with smaller fine granularity, so that the detection precision of the model is increased, finally, the detection result of the training sample image is predicted by combining labeling information and the second feature image, and the completion of the training of the detection model can be judged when the model converges.
With reference to fig. 4, the method for detecting fire based on images in the embodiment of the present invention is described above, and a device for detecting fire based on images in the embodiment of the present invention is described below, where an embodiment of the device for detecting fire based on images in the embodiment of the present invention includes:
the acquisition module 401 is configured to acquire a plurality of sample images including smoke and fire, and label smoke and fire areas in the sample images to obtain labeled images;
a data enhancement module 402, configured to generate an initial data set according to the annotation image, and perform data enhancement on the initial data set to obtain an enhanced data set, where the enhanced data set includes a plurality of enhanced annotation images;
a model training module 403, configured to input the enhanced annotation image into a preset YOLOv3 model for training, so as to obtain a smoke and fire detection model, where the YOLOv3 model includes: darknet-53 network, Batch nonilization layer, LeakyReLU layer and convolutional layer;
the detection module 404 is configured to acquire a surveillance video to be detected, input the surveillance video into the smoke and fire detection model for frame-by-frame detection, and output a detection result;
and a fire early warning module 405, configured to transmit the video frame marked with the smoke and/or fire information to a monitoring terminal to perform fire early warning if the detection result indicates that smoke and/or fire exists in the current video frame.
Optionally, in an embodiment, the acquisition module 401 is specifically configured to:
collecting a plurality of sample images containing smoke and fire;
and calling a preset Labellmg tool, selecting smoke and fire regions in the sample image to obtain a sample region image, and carrying out region information annotation on the sample region image to obtain an annotated image.
Optionally, in an embodiment, the data enhancement module 402 is specifically configured to:
converting the marked image into a JPG format to obtain a JPG image;
storing the JPG image into a preset folder to generate an initial data set;
reading a plurality of JPG images in the initial data set, and sequentially turning, zooming and adjusting the color gamut of each image to obtain a plurality of standard annotation images;
splicing the standard annotation images based on a preset splicing rule to obtain a plurality of enhanced annotation images;
and storing the enhanced annotation images in a VOC format to obtain an enhanced data set.
Optionally, in an embodiment, the model training module 403 includes:
the network training unit 4031 is used for inputting the enhanced annotation image into the Darknet-53 network for feature extraction to obtain a first feature map; inputting the first feature map into the Batch nonillization layer for normalization processing to obtain a normalized first feature map; inputting the normalized first characteristic diagram into the LeakyReLU layer for nonlinear conversion to obtain a second characteristic diagram; inputting the second characteristic diagram into the convolutional layer for pixel point prediction to obtain a prediction result corresponding to the second characteristic diagram;
a loss calculating unit 4032, configured to invoke a preset loss function, and calculate a prediction result corresponding to the second feature map and a loss value of the enhanced labeled image;
a parameter tuning unit 4033, configured to adjust a parameter of the YOLOv3 model according to the loss value until the YOLOv3 model converges, so as to obtain a smoke and fire detection model.
Optionally, in an embodiment, the loss calculating unit 4032 is specifically configured to:
calling a preset Focal local function, and calculating a prediction result corresponding to the second feature map and the backward gradient of the enhanced labeled image through the Focal local function to obtain a backward gradient value;
and carrying out derivation operation on the backward gradient value to obtain a prediction result corresponding to the second characteristic diagram and a loss value of the enhanced labeled image.
Optionally, in an embodiment, the detecting module 404 includes:
an obtaining unit 4041, configured to obtain a monitored video to be detected;
a feature extraction unit 4042, configured to input the monitoring video into the Darknet-53 network, and perform feature extraction frame by frame to obtain a third feature map;
the feature processing unit 4043 is configured to input the third feature map into the Batch nonillization layer for normalization processing, so as to obtain a normalized third feature map; inputting the normalized third feature map into the LeakyReLU layer for nonlinear conversion to obtain a fourth feature map; and inputting the fourth characteristic diagram into the convolutional layer for pixel point prediction to obtain a detection result corresponding to the fourth characteristic diagram.
Optionally, in an embodiment, the feature extraction unit 4042 is specifically configured to:
framing the monitoring video based on a preset FFmpeg frame to obtain a plurality of video frames;
sequentially denoising, contrast enhancement, brightness and saturation adjustment are carried out on each video frame to obtain a plurality of standard video frames;
and sequentially inputting the standard video frames into the Darknet-53 network for feature extraction to obtain third feature maps corresponding to the standard video frames.
In the embodiment of the invention, a model for detecting smoke and fire is trained by adopting a YOLOv3 model, the data set is subjected to data enhancement, the detection background is enriched, the data of a plurality of images can be simultaneously calculated during calculation, when a real-time monitoring video is detected, the monitoring video is input into a smoke and fire detection model to be detected frame by frame, the detection result is output, and when the detection result is that smoke and/or fire exists in the current video frame, the video frame marked with smoke and/or fire information is transmitted to a monitoring terminal to carry out fire early warning. The invention improves the accuracy of fire detection by detecting the monitoring video in real time.
Fig. 4 above describes the image-based fire detection apparatus in the embodiment of the present invention in detail from the perspective of the modular functional entity, and the image-based fire detection apparatus in the embodiment of the present invention is described in detail from the perspective of hardware processing.
Fig. 5 is a schematic structural diagram of an image-based fire detection apparatus 500 according to an embodiment of the present invention, which may have a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 510 (e.g., one or more processors) and a memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) storing applications 533 or data 532. Memory 520 and storage media 530 may be, among other things, transient or persistent storage. The program stored on the storage medium 530 may include one or more modules (not shown), each of which may include a series of instruction operations for the image-based fire detection apparatus 500. Still further, the processor 510 may be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the image-based fire detection apparatus 500.
The image-based fire detection apparatus 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input-output interfaces 560, and/or one or more operating systems 531, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, and the like. Those skilled in the art will appreciate that the configuration of the image-based fire detection apparatus shown in FIG. 5 does not constitute a limitation of the image-based fire detection apparatus and may include more or fewer components than shown, or some components may be combined, or a different arrangement of components.
The invention also provides an image-based fire detection device, which comprises a memory and a processor, wherein the memory stores computer readable instructions, and the computer readable instructions, when executed by the processor, cause the processor to execute the steps of the image-based fire detection method in the above embodiments.
The present invention also provides a computer readable storage medium, which may be a non-volatile computer readable storage medium, which may also be a volatile computer readable storage medium, having stored therein instructions, which, when run on a computer, cause the computer to perform the steps of the image based fire detection method.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. An image-based fire detection method, comprising:
collecting a plurality of sample images containing smoke and fire, and labeling smoke and fire areas in the sample images to obtain labeled images;
generating an initial data set according to the annotation images, and performing data enhancement on the initial data set to obtain an enhanced data set, wherein the enhanced data set comprises a plurality of enhanced annotation images;
inputting the enhanced annotation image into a preset YOLOv3 model for training to obtain a smoke and fire detection model, wherein the YOLOv3 model comprises: darknet-53 network, Batch nonilization layer, LeakyReLU layer and convolutional layer;
acquiring a monitoring video to be detected, inputting the monitoring video into the smoke and fire light detection model for frame-by-frame detection, and outputting a detection result;
and if the detection result indicates that smoke and/or fire exist in the current video frame, transmitting the video frame marked with the smoke and/or fire information to a monitoring terminal for fire early warning.
2. The image-based fire detection method of claim 1, wherein the acquiring a plurality of sample images containing smoke and fire, and labeling smoke and fire regions in the sample images to obtain labeled images comprises:
collecting a plurality of sample images containing smoke and fire;
and calling a preset Labellmg tool, selecting smoke and fire regions in the sample image to obtain a sample region image, and carrying out region information annotation on the sample region image to obtain an annotated image.
3. The image-based fire detection method of claim 1, wherein the generating an initial data set from the annotated image and performing data enhancement on the initial data set to obtain an enhanced data set comprises:
converting the marked image into a JPG format to obtain a JPG image;
storing the JPG image into a preset folder to generate an initial data set;
reading a plurality of JPG images in the initial data set, and sequentially turning, zooming and adjusting the color gamut of each image to obtain a plurality of standard annotation images;
splicing the standard annotation images based on a preset splicing rule to obtain a plurality of enhanced annotation images;
and storing the enhanced annotation images in a VOC format to obtain an enhanced data set.
4. The image-based fire detection method according to any one of claims 1-3, wherein the inputting of the enhanced annotation image into a preset YOLOv3 model for training to obtain a smoke and fire detection model comprises:
inputting the enhanced annotation image into the Darknet-53 network for feature extraction to obtain a first feature map;
inputting the first feature map into the Batch nonillization layer for normalization processing to obtain a normalized first feature map;
inputting the normalized first characteristic diagram into the LeakyReLU layer for nonlinear conversion to obtain a second characteristic diagram;
inputting the second characteristic diagram into the convolutional layer for pixel point prediction to obtain a prediction result corresponding to the second characteristic diagram;
calling a preset loss function, and calculating a prediction result corresponding to the second feature map and a loss value of the enhanced labeled image;
and adjusting parameters of the YOLOv3 model according to the loss value until the YOLOv3 model converges to obtain a smoke and fire detection model.
5. The image-based fire detection method of claim 4, wherein the calling a preset loss function and the calculating the prediction result corresponding to the second feature map and the loss value of the enhanced labeled image comprises:
calling a preset Focalloss function, and calculating a prediction result corresponding to the second feature map and the backward gradient of the enhanced labeled image through the Focalloss function to obtain a backward gradient value;
and carrying out derivation operation on the backward gradient value to obtain a prediction result corresponding to the second characteristic diagram and a loss value of the enhanced labeled image.
6. The image-based fire detection method according to claim 1, wherein the acquiring a surveillance video to be detected and inputting the surveillance video into the smoke and fire detection model for frame-by-frame detection, and the outputting the detection result comprises:
acquiring a monitoring video to be detected;
inputting the monitoring video into the Darknet-53 network for feature extraction frame by frame to obtain a third feature map;
inputting the third feature map into the Batch nonillization layer for normalization processing to obtain a normalized third feature map;
inputting the normalized third feature map into the LeakyReLU layer for nonlinear conversion to obtain a fourth feature map;
and inputting the fourth characteristic diagram into the convolutional layer for pixel point prediction to obtain a detection result corresponding to the fourth characteristic diagram.
7. The image-based fire detection method according to claim 6, wherein the inputting the surveillance video into the Darknet-53 network for feature extraction on a frame-by-frame basis to obtain a third feature map comprises:
framing the monitoring video based on a preset FFmpeg frame to obtain a plurality of video frames;
sequentially denoising, contrast enhancement, brightness and saturation adjustment are carried out on each video frame to obtain a plurality of standard video frames;
and sequentially inputting the standard video frames into the Darknet-53 network for feature extraction to obtain third feature maps corresponding to the standard video frames.
8. An image-based fire detection device, comprising:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a plurality of sample images containing smoke and fire, and labeling smoke and fire areas in the sample images to obtain labeled images;
the data enhancement module is used for generating an initial data set according to the annotation images and carrying out data enhancement on the initial data set to obtain an enhanced data set, wherein the enhanced data set comprises a plurality of enhanced annotation images;
a model training module, configured to input the enhanced annotation image into a preset YOLOv3 model for training, so as to obtain a smoke and fire detection model, where the YOLOv3 model includes: darknet-53 network, Batch nonilization layer, LeakyReLU layer and convolutional layer;
the detection module is used for acquiring a monitoring video to be detected, inputting the monitoring video into the smoke and fire light detection model for frame-by-frame detection and outputting a detection result;
and the fire early warning module is used for transmitting the video frame marked with the smoke and/or fire information to the monitoring terminal for fire early warning if the detection result indicates that smoke and/or fire exists in the current video frame.
9. An image-based fire detection apparatus, comprising: a memory and at least one processor, the memory having instructions stored therein;
the at least one processor invokes the instructions in the memory to cause the image-based fire detection apparatus to perform the image-based fire detection method of any one of claims 1-7.
10. A computer readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, implement the image-based fire detection method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110051354.9A CN112906463A (en) | 2021-01-15 | 2021-01-15 | Image-based fire detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110051354.9A CN112906463A (en) | 2021-01-15 | 2021-01-15 | Image-based fire detection method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112906463A true CN112906463A (en) | 2021-06-04 |
Family
ID=76113199
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110051354.9A Pending CN112906463A (en) | 2021-01-15 | 2021-01-15 | Image-based fire detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112906463A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113420664A (en) * | 2021-06-23 | 2021-09-21 | 国网电子商务有限公司 | Image-based potential safety hazard detection method, device, equipment and storage medium |
| CN113469057A (en) * | 2021-07-02 | 2021-10-01 | 中南大学 | Fire hole video self-adaptive detection method, device, equipment and medium |
| CN113610009A (en) * | 2021-08-10 | 2021-11-05 | 北京科技大学天津学院 | Flood disaster unmanned aerial vehicle image information extraction system |
| CN113688748A (en) * | 2021-08-27 | 2021-11-23 | 武汉大千信息技术有限公司 | Fire detection model and method |
| CN114140732A (en) * | 2021-12-09 | 2022-03-04 | 安徽大学 | Fire-fighting fire source detection method and device and storage medium under small sample condition |
| CN114389893A (en) * | 2022-01-22 | 2022-04-22 | 重庆长安汽车股份有限公司 | Live video processing vehicle real-name authentication system and method and automobile |
| CN114558267A (en) * | 2022-03-03 | 2022-05-31 | 上海应用技术大学 | Industrial scene fire prevention and control system |
| CN114943923A (en) * | 2022-06-17 | 2022-08-26 | 中国人民解放军陆军炮兵防空兵学院 | Method and system for recognizing explosion flare smoke of cannonball based on video of deep learning |
| CN114973110A (en) * | 2022-07-27 | 2022-08-30 | 四川九通智路科技有限公司 | On-line monitoring method and system for highway weather |
| CN116091491A (en) * | 2023-04-03 | 2023-05-09 | 中国科学技术大学先进技术研究院 | VOCs gas detection method, device, equipment and computer readable storage medium |
| CN116863252A (en) * | 2023-09-04 | 2023-10-10 | 四川泓宝润业工程技术有限公司 | Method, device, equipment and storage medium for detecting inflammable substances in live fire operation site |
| CN117893643A (en) * | 2024-03-18 | 2024-04-16 | 四川弘和数智集团有限公司 | Method, device, equipment and medium for generating gas leakage image of oil and gas station compressor |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018116966A1 (en) * | 2016-12-21 | 2018-06-28 | ホーチキ株式会社 | Fire monitoring system |
| WO2019101220A1 (en) * | 2017-12-11 | 2019-05-31 | 珠海大横琴科技发展有限公司 | Deep learning network and average drift-based automatic vessel tracking method and system |
| CN111091072A (en) * | 2019-11-29 | 2020-05-01 | 河海大学 | YOLOv 3-based flame and dense smoke detection method |
| KR102144974B1 (en) * | 2019-11-08 | 2020-08-14 | 주식회사 알체라 | Fire detection system |
| WO2020238902A1 (en) * | 2019-05-29 | 2020-12-03 | 腾讯科技（深圳）有限公司 | Image segmentation method, model training method, apparatuses, device and storage medium |
| CN112132090A (en) * | 2020-09-28 | 2020-12-25 | 天地伟业技术有限公司 | Smoke and fire automatic detection and early warning method based on YOLOV3 |
-
2021
- 2021-01-15 CN CN202110051354.9A patent/CN112906463A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018116966A1 (en) * | 2016-12-21 | 2018-06-28 | ホーチキ株式会社 | Fire monitoring system |
| WO2019101220A1 (en) * | 2017-12-11 | 2019-05-31 | 珠海大横琴科技发展有限公司 | Deep learning network and average drift-based automatic vessel tracking method and system |
| WO2020238902A1 (en) * | 2019-05-29 | 2020-12-03 | 腾讯科技（深圳）有限公司 | Image segmentation method, model training method, apparatuses, device and storage medium |
| KR102144974B1 (en) * | 2019-11-08 | 2020-08-14 | 주식회사 알체라 | Fire detection system |
| CN111091072A (en) * | 2019-11-29 | 2020-05-01 | 河海大学 | YOLOv 3-based flame and dense smoke detection method |
| CN112132090A (en) * | 2020-09-28 | 2020-12-25 | 天地伟业技术有限公司 | Smoke and fire automatic detection and early warning method based on YOLOV3 |
Non-Patent Citations (2)
| Title |
|---|
| ALEXEY BOCHKOVSKIY AND CHIEN-YAO WANG AND HONG-YUAN MARK LIAO: "YOLOv4: Optimal Speed and Accuracy of Object Detection", 《ARXIV COMPUTER SCIENCE》, pages 10934 * |
| 罗小权等: "改进YOLOV3的火灾检测方法", 《计算机工程与应用》, vol. 56, no. 17, pages 187 - 188 * |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113420664A (en) * | 2021-06-23 | 2021-09-21 | 国网电子商务有限公司 | Image-based potential safety hazard detection method, device, equipment and storage medium |
| CN113469057A (en) * | 2021-07-02 | 2021-10-01 | 中南大学 | Fire hole video self-adaptive detection method, device, equipment and medium |
| CN113610009A (en) * | 2021-08-10 | 2021-11-05 | 北京科技大学天津学院 | Flood disaster unmanned aerial vehicle image information extraction system |
| CN113688748B (en) * | 2021-08-27 | 2023-08-18 | 武汉大千信息技术有限公司 | Fire detection model and method |
| CN113688748A (en) * | 2021-08-27 | 2021-11-23 | 武汉大千信息技术有限公司 | Fire detection model and method |
| CN114140732A (en) * | 2021-12-09 | 2022-03-04 | 安徽大学 | Fire-fighting fire source detection method and device and storage medium under small sample condition |
| CN114389893A (en) * | 2022-01-22 | 2022-04-22 | 重庆长安汽车股份有限公司 | Live video processing vehicle real-name authentication system and method and automobile |
| CN114558267A (en) * | 2022-03-03 | 2022-05-31 | 上海应用技术大学 | Industrial scene fire prevention and control system |
| CN114943923A (en) * | 2022-06-17 | 2022-08-26 | 中国人民解放军陆军炮兵防空兵学院 | Method and system for recognizing explosion flare smoke of cannonball based on video of deep learning |
| CN114943923B (en) * | 2022-06-17 | 2022-12-23 | 中国人民解放军陆军炮兵防空兵学院 | Method and system for recognizing explosion flare smoke of cannonball based on video of deep learning |
| CN114973110A (en) * | 2022-07-27 | 2022-08-30 | 四川九通智路科技有限公司 | On-line monitoring method and system for highway weather |
| CN114973110B (en) * | 2022-07-27 | 2022-11-01 | 四川九通智路科技有限公司 | On-line monitoring method and system for highway weather |
| CN116091491A (en) * | 2023-04-03 | 2023-05-09 | 中国科学技术大学先进技术研究院 | VOCs gas detection method, device, equipment and computer readable storage medium |
| CN116863252A (en) * | 2023-09-04 | 2023-10-10 | 四川泓宝润业工程技术有限公司 | Method, device, equipment and storage medium for detecting inflammable substances in live fire operation site |
| CN116863252B (en) * | 2023-09-04 | 2023-11-21 | 四川泓宝润业工程技术有限公司 | Method, device, equipment and storage medium for detecting inflammable substances in live fire operation site |
| CN117893643A (en) * | 2024-03-18 | 2024-04-16 | 四川弘和数智集团有限公司 | Method, device, equipment and medium for generating gas leakage image of oil and gas station compressor |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112906463A (en) | Image-based fire detection method, device, equipment and storage medium | |
| CN110059694B (en) | Intelligent identification method for character data in complex scene of power industry | |
| CN111310645B (en) | Method, device, equipment and storage medium for warning overflow bin of goods accumulation | |
| CN103824053B (en) | The sex mask method and face gender detection method of a kind of facial image | |
| CN110084165B (en) | Intelligent identification and early warning method for abnormal events in open scene of power field based on edge calculation | |
| CN111681273B (en) | Image segmentation method and device, electronic equipment and readable storage medium | |
| CN111260666B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| CN111814753A (en) | Target detection method and device under foggy weather condition | |
| WO2021081562A2 (en) | Multi-head text recognition model for multi-lingual optical character recognition | |
| CN110059635B (en) | Skin defect detection method and device | |
| CN111524145A (en) | Intelligent picture clipping method and system, computer equipment and storage medium | |
| US20180046866A1 (en) | Method of Detecting a Moving Object by Reconstructive Image Processing | |
| CN111461101A (en) | Method, device and equipment for identifying work clothes mark and storage medium | |
| CN113516146A (en) | Data classification method, computer and readable storage medium | |
| CN111461260B (en) | Target detection method, device and equipment based on feature fusion and storage medium | |
| CN114782892A (en) | Illegal behavior target detection method, device, equipment and storage medium | |
| CN108229281B (en) | Neural network generation method, face detection device and electronic equipment | |
| CN115496971A (en) | Infrared target detection method and device, electronic equipment and storage medium | |
| JP2020017136A (en) | Object detection and recognition apparatus, method, and program | |
| CN113076889A (en) | Container lead seal identification method and device, electronic equipment and storage medium | |
| CN115861922B (en) | Sparse smoke detection method and device, computer equipment and storage medium | |
| CN115131826B (en) | Article detection and identification method, and network model training method and device | |
| CN107403192B (en) | Multi-classifier-based rapid target detection method and system | |
| CN115953744A (en) | Vehicle identification tracking method based on deep learning | |
| CN114283442A (en) | Intelligent identification method and device for secondary wiring diagram and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |