CN108537215A - A kind of flame detecting method based on image object detection - Google Patents
A kind of flame detecting method based on image object detection Download PDFInfo
- Publication number
- CN108537215A CN108537215A CN201810243014.4A CN201810243014A CN108537215A CN 108537215 A CN108537215 A CN 108537215A CN 201810243014 A CN201810243014 A CN 201810243014A CN 108537215 A CN108537215 A CN 108537215A
- Authority
- CN
- China
- Prior art keywords
- picture
- frame
- model
- flame
- standard size
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 83
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000012360 testing method Methods 0.000 claims abstract description 80
- 238000012549 training Methods 0.000 claims abstract description 53
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 18
- 238000012545 processing Methods 0.000 claims description 16
- 238000011176 pooling Methods 0.000 claims description 15
- 238000000605 extraction Methods 0.000 claims description 13
- 238000010606 normalization Methods 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000002372 labelling Methods 0.000 claims description 6
- 238000003062 neural network model Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 238000000638 solvent extraction Methods 0.000 claims description 4
- 230000014509 gene expression Effects 0.000 claims description 3
- 238000003064 k means clustering Methods 0.000 claims description 3
- 238000012544 monitoring process Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007635 classification algorithm Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000011897 real-time detection Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The present invention proposes a kind of flame detecting method detected based on image object, belongs to image procossing, fire detection and field of video monitoring.This method initially sets up the fire defector data set comprising flame picture and per pictures markup information, and is divided into training set and test set；Depth convolutional neural networks model is built, is iterated update using training the set pair analysis model, using test set to newer model counting loss function, if the loss function of "current" model no longer declines, model training finishes；Real-time video is shot, the model finished using training is detected each frame picture, if there are flame, model exports coordinate position of the flame in picture, and rectangle frame is used in combination to be marked.The present invention generates the candidate region of doubtful flame without hand-designed feature, directly depth convolutional neural networks can be used to carry out fire defector to whole pictures, the location information of flame is obtained, to which early warning fire occurs, minimizes the harm that fire is brought.
Description
Technical Field
The invention relates to the field of image processing, fire detection and video monitoring, in particular to a flame detection method based on image target detection.
Background
The fire disaster is a frequent disaster seriously harming the life and property safety of people, early warning of the occurrence of the fire disaster can avoid the spread of the fire disaster, and the harm brought by the fire disaster is reduced to the maximum extent. For flame detection tasks, especially for outdoor and large indoor spaces and other scenes, common physical sensors cannot be effectively completed, so that the flame detection method based on the image processing technology is an important research direction in the field of safety engineering at present.
The development of flame detection methods based on image processing techniques is mainly in two stages. The first stage is to judge whether flame exists in the image according to visual characteristics of flame such as color, shape, texture and the like and by combining a classification algorithm in pattern recognition. The methods for manually designing the characteristics have high detection speed, but have poor stability and generalization capability, false alarm is easy to occur when detecting objects with the color similar to that of flame, and false alarm is easy to occur when detecting flame with the color exceeding a set threshold value. To address the problems with such approaches, researchers have recently begun to utilize deep convolutional neural networks for flame detection. The deep convolutional neural network is the most widely studied and applied method in the field of computer vision in recent five years, and features are generally automatically extracted from an image by using a convolutional layer, a pooling layer, a Batch Normalization (BN) layer and the like, and classified by using a full connection layer to judge whether flames exist in the image. The method has the advantages that the tasks of image feature extraction and mode classification can be realized by utilizing the neural network without manually designing features. Compared with the traditional flame detection method, the technology based on the deep convolutional neural network has higher accuracy, recall rate and higher generalization capability. In recent years, with the development of hardware computing power of CPUs, GPUs and the like, the computing speed of the method can meet the real-time requirement of flame detection. The existing flame detection method based on the deep convolutional neural network has the problems that a candidate area of suspected flame still needs to be generated from an original image, and then the neural network is used for classifying the candidate area to judge whether flame exists or not.
At present, a flame detection method based on a deep convolutional neural network is provided, and the method comprises the steps of firstly extracting a suspected flame point in a color image, taking the suspected flame point as a foreground to obtain a binary image, obtaining a series of connected regions by using a connected region method, obtaining candidate regions through screening, and then carrying out classification judgment on whether flames exist in the candidate regions by using the convolutional neural network. The problem with this approach is that the generation of the suspected flame candidate area is still accomplished by a manual design feature. This process still requires the use of the color and dynamic characteristics of the flame, which results in poor adaptability of its detection method to environmental changes. To date, no method has emerged that can achieve image flame detection completely independent of manually designed features.
Disclosure of Invention
The invention aims to overcome the defects of the existing method and provides a flame detection method based on image target detection. According to the method, the candidate area of the suspected flame is generated without using manual design characteristics, the flame of the whole picture can be directly detected by using the deep convolutional neural network, and the position information of the flame is obtained, so that the occurrence of fire is early warned, and the harm caused by the fire is reduced to the maximum extent.
The invention provides a flame detection method based on image target detection, which is characterized by comprising the following steps of:
1) collecting N color pictures with flame pictures to establish a flame picture data set Fimage，N≥5000；
2) For the flame picture data set F established in the step 1)imageManually marking the flame area position coordinates of each picture by using a rectangular frame to obtain a real marking frame of the flame area position and 4 coordinate values of the real marking frame in each picture, wherein the 4 coordinate values are as follows:1, 2, N, whereinThe position coordinates of the top left corner vertex of the frame are really marked for the first picture,the position coordinates of the top point of the lower right corner of the frame are really marked for the first picture; storing the information of the real marking frame of each picture as an xml file, wherein the xml files corresponding to all the flame pictures form a flame marking data set Fannotation；FimageAnd FannotationForming a flame detection dataset F0；
3) Detecting the flame obtained in step 2) by using a data set F0Random partitioning into training sets FtrainAnd test set FtestWherein the training set FtrainOccupied flame detection dataset F0The ratio of a is more than or equal to 0.6 and less than or equal to 0.9;
4) constructing a deep convolution neural network model and training the model to obtain a trained flame detection model; the method comprises the following specific steps:
4-1) carrying out standardization and normalization processing on each picture of the training set and the test set;
standardizing each picture of the training set and the test set established in the step 1), wherein the standard width and the standard height are respectively W and H, W is more than or equal to 400 and less than or equal to 800, H is more than or equal to 400 and less than or equal to 800, and the channel number Ch is 3; taking the pixel value of the t channel and the coordinate (x, y) of any image after the normalization processing as I (t, x, y), wherein I (t, x, y) is more than or equal to 0 and less than or equal to 255, calculating the normalized pixel value I '(t, x, y) by using the formula (1), wherein I' (t, x, y) is more than or equal to 0 and less than or equal to 1:
4-2) constructing a deep convolutional neural network model, wherein the model comprises an input layer, a feature extraction part and an output layer which are sequentially connected; the characteristic extraction part is formed by combining convolution layers and pooling layers and comprises 3-6 pooling layers, wherein each pooling layer comprises 1-5 convolution layers in front of the pooling layer, and a batch standard layer is added behind each convolution layer;
4-3) training the model established in the step 4-2) to obtain a trained flame detection model; the method comprises the following specific steps:
4-3-1) sequentially inputting each picture of the training set subjected to the standardization and normalization processing in the step 4-1) into the model established in the step 4-2); inputting the first picture, outputting a feature map corresponding to the picture by the penultimate convolutional layer in the deep convolutional neural network, wherein the feature map is s multiplied by s units, each unit corresponds to different position areas of the input picture, then connecting the output of the penultimate convolutional layer to an output layer of the whole model through the last convolutional layer, outputting the output of the penultimate convolutional layer as k in the s multiplied by s different position areas of the input picture, wherein k is more than or equal to 5 and less than or equal to 9, and the class probability of each prediction candidate frameAnd coordinate offset relative to corresponding standard size box1≤i≤s2J is more than or equal to 1 and less than or equal to k, wherein i represents the ith position area, and j represents the jth prediction candidate frame of the position area; the width and height of the standard size frame relative to the whole input picture are represented by FimageThe width and height of the real mark frame of all pictures relative to the original pictures are obtained by clustering by a k-means method, so thatRespectively representing the width and the height of the jth standard size frame relative to the whole input picture; the center of each standard size frame is the center of the picture position area where the standard size frame is located; each standard size frame corresponds to a prediction candidate frame;
wherein,the probability that flame exists in the jth prediction candidate frame in the ith position area of the ith picture is represented, and the value range is 0-1;the coordinate offset of the jth prediction candidate frame center relative to the corresponding standard size frame center in the ith position area of the ith picture is represented, and the value range is 0-1 (delta wij，Δhij) Representing the width and height offset of a jth prediction candidate frame in an ith position area of the ith picture relative to the width and height of a corresponding standard size frame, wherein the value range is 0-1;
4-3-2) dividing s multiplied by k standard size frames of the input picture in the step 4-3-1) into positive and negative samples;
if the real labeled frame in each picture is G, and any standard size frame of the picture is T, the intersection ratio IoU of T and G is calculated as follows:
wherein area represents a region;
setting a threshold η of the positive and negative samples, wherein η is more than or equal to 0.5 and less than or equal to 0.7, and judging s multiplied by k standard size frames corresponding to each picture, wherein if IoU is more than or equal to η, flames exist in the standard size frames, the standard size frames are positive samples, and if IoU is less than η, the standard size frames do not exist, the standard size frames are negative samples;
4-3-3) calculating the error between the model output information and the real annotation information according to the loss function;
outputting the category probability of s multiplied by k prediction candidate frames corresponding to the first picture and the coordinate offset relative to the corresponding standard size frame according to the model obtained in the step 4-3-1), and calculating the picture prediction informationCorresponding real label informationThe error between; the loss function for the ith picture is defined as follows:
wherein,representing the real category corresponding to the jth prediction candidate frame in the ith position area of the ith picture, if the standard size frame corresponding to the prediction candidate frame is a positive sample, thenThe value is 1; if the standard size frame corresponding to the prediction candidate frame is negative sample, thenThe value is 0;indicating the coordinate offset of the center of the real labeling frame of the ith picture relative to the center of the jth standard size frame of the ith position area,the offset of the width and the height of the real marking frame of the ith picture relative to the width and the height of the jth standard size frame of the ith position area respectively is represented; relative offset of real marking frame of first pictureThe calculation expressions are respectively as follows:
wherein,respectively represent F corresponding to the first pictureimageThe width and height of the original picture;coordinate information of the jth standard size frame of the ith position area of the ith picture relative to the whole input picture,is the coordinate of the center of the ith position area relative to the whole input picture,obtained by k-means clustering; lambda [ alpha ]locA weight coefficient representing the position error loss relative to the class error loss, 0.01 ≦ λloc≤0.2；A Boolean value indicating whether the jth prediction candidate frame of the ith position area is a flame or not, if the standard size frame corresponding to the prediction candidate frame is a positive sample, the prediction candidate frame is divided into a plurality of regionsThe value is 1; if the standard size frame corresponding to the prediction candidate frame is negative sample, thenThe value is 0;a Boolean value indicating whether the jth prediction candidate frame of the ith position area is not the flame, if the standard size frame corresponding to the prediction candidate frame is a positive sample, thenThe value is 0; if the standard size frame corresponding to the prediction candidate frame is negative sample, thenThe value is 1; lambda [ alpha ]noobjλ is 0.01 ≦ for adjusting the weighting factor of the class loss ratio of the positive and negative samplesnoobj≤0.2；
4-3-4) repeating steps 4-3-1) to 4-3-3), and combining the training set FtrainInputting each picture into the Model established in the step 4-1), updating the Model through an error back propagation algorithm to obtain a current Model and recording the current Model as a Modelold；
4-3-5) Using test set FtestFor ModeloldTesting is carried out, and a test set FtestEach picture is input into a Model in sequenceoldCalculating F according to equation (3)testAll pictures in the ModeloldLoss of total Loss ofold；
4-3-6) repeat step 4-3-4), training set FtrainEach picture is input into a Model in sequenceoldObtaining a new current Model and marking as a Modelnew；
4-3-7) repeat step 4-3-5), using test set FtestFor ModelnewTesting was carried out to give FtestAll pictures in the ModelnewLoss of total Loss ofnew；
4-3-8) training stopping condition judgment:
if Lossnew≤LossoldContinuing to train the Model and setting the current ModelnewUpdate to a new ModeloldWill be the current LossnewUpdate to New LossoldAnd returning to the step 4-3-6);
if Lossnew≥LossoldWhen the Model is not trained, the Model stops training and outputs the current ModeloldTaking the flame detection model as a finally trained flame detection model, and entering the step 5);
5) carrying out flame detection on the real-time video by using the flame detection model obtained in the step 4);
5-1) shooting a real-time video, inputting each frame of picture of the real-time video into the flame detection model trained in the step 4) for detection, and outputting the class probability of s multiplied by k prediction candidate frames corresponding to the frame of picture and the coordinate offset relative to the corresponding standard size frame by the modelWhere the superscript o represents the flame detection model;
5-2) setting a flame judgment threshold value, wherein the value range is 0-1, and judging the category probability of a prediction candidate frame output by each frame of picture: if it isThen no flame is present in the prediction candidate box; if it isThe predicted candidate frame has a flame and the amount of coordinate offset of the predicted candidate frame with respect to the corresponding standard-sized frame is determined based on the flameAnd corresponding standard size frame coordinatesCalculating to obtain the vertex coordinates of the upper left corner and the lower right corner of the actually predicted flame area as
Wherein,respectively representing the original width and height of the frame picture;
5-3) if all the prediction candidate frames of the frame have no flame, returning to the step 5-1) again, and carrying out flame detection on the next frame; if any prediction candidate frame has flame, the model outputs the vertex coordinates of the actually predicted flame area in the upper left corner and the lower right corner of the picture, which are obtained by calculation in the step 5-2), and the coordinates are marked by using a rectangular frame, and then the step 5-1) is returned again to continue to detect the flame of the next frame of picture.
The invention has the characteristics and beneficial effects that:
the invention applies the target detection technology in the field of computer vision, is different from other flame detection methods, does not need to extract a candidate area of suspected flame, and converts the classification problem of flame detection into a regression problem. The invention does not depend on the static and dynamic characteristics of the flame needing manual design, can complete the whole detection process from the input of an original image to the output of flame position information through a deep convolution neural network in the whole process, and avoids the situations of false alarm and false alarm of the manually designed characteristics to a certain extent. In addition, the invention can adapt to image input with different sizes, can meet the requirement of real-time detection, has stronger generalization performance in different scenes, and can be applied to fire early warning tasks in various scenes such as indoor scenes, outdoor scenes and the like so as to avoid fire or reduce the harm brought by fire.
Drawings
FIG. 1 is an overall flow chart of the method of the present invention.
FIG. 2 is a diagram of a standard size box, a true label box, and a prediction candidate box according to an embodiment of the invention.
Fig. 3 is a schematic view of a process for labeling a flame picture according to an embodiment of the invention.
Fig. 4 is a schematic diagram of flame detection results in different scenarios according to an embodiment of the present invention.
Detailed Description
The invention provides a flame detection method based on image target detection, which is further described in detail below with reference to the accompanying drawings and specific embodiments.
The invention provides a flame detection method based on image target detection, the overall flow is shown as figure 1, and the method comprises the following steps:
1) collecting N (N is more than or equal to 5000) color pictures with flame pictures, and establishing a flame picture data set Fimage. The picture source has no special requirement, the pictures of the embodiment are all collected from the network, the pictures have no size requirement, and the pictures are required to have obvious flame pictures.
2) For the flame picture data set F established in the step 1)imageEach of which is shown inThe flame area position coordinates of the film are manually marked by a rectangular frame (the rectangular frame is the actual position of the flame area in the picture and is framed by the rectangular frame to obtain 4 coordinate values of the rectangular frame), and the real marking frame of the flame area position and the 4 coordinate values of the real marking frame in each picture are obtained as follows:1, 2, N, whereinThe position coordinates of the top left corner vertex of the frame are really marked for the first picture,the position coordinates of the top point of the lower right corner of the frame are really marked for the first picture; and storing the flame marking data set as an xml file in a format of a VOC007 data set, wherein each picture corresponds to one xml file, and the xml files corresponding to all the flame pictures form a flame marking data set Fannotation。FimageAnd FannotationForm a complete flame detection data set F0。
3) Detecting the flame obtained in step 2) by using a data set F0Random partitioning into training sets FtrainAnd test set Ftest(due to F)imagePictures and F inannotationThe xml files in (1) are in one-to-one correspondence, and the training set F is dividedtrainAnd test set FtestThe images obtained by the middle division and the xml files are in one-to-one correspondence), wherein the training set FtrainOccupied flame detection dataset F0A is more than or equal to 0.6 and less than or equal to 0.9, and a training set FtrainParameters to train the model, test set FtestTo test the generalization performance of the model.
4) Constructing a deep convolution neural network model and training the model to obtain a trained flame detection model; the method comprises the following specific steps:
4-1) carrying out standardization and normalization processing on each picture of the training set and the test set;
because the size of the collected picture in the step 1) is any size, and the input of the convolutional neural network needs to fix the size of the picture, the original picture needs to be scaled to a standard size, the standard width and height are respectively W and H, W is more than or equal to 400 and less than or equal to 800, H is more than or equal to 400 and less than or equal to 800, and Ch is 3 without changing the number of channels. In the training and testing process, the value ranges of all pixel values of the input picture sample are all between 0 and 255, and normalization processing is needed to make the pixel values between 0 and 1. Taking the pixel value of the t channel and the coordinate (x, y) of any normalized picture as I (t, x, y), wherein I (t, x, y) is more than or equal to 0 and less than or equal to 255, calculating the normalized pixel value I '(t, x, y) by using a formula (1), wherein I' (t, x, y) is more than or equal to 0 and less than or equal to 1:
4-2) constructing a deep convolutional neural network model, which comprises an input layer, a feature extraction part and an output layer which are sequentially connected; wherein, the normalized picture is input from the input layer and connected with the feature extraction part; the characteristic extraction part is formed by combining a convolution layer and a pooling layer, wherein the pooling layer comprises 3-6 pooling layers, 1-5 convolution layers are arranged in front of each pooling layer, and a batch standard layer is added behind each convolution layer and is used for normalizing each batch of input data in an output characteristic diagram of the convolution layer; and finally, the characteristic extraction part is connected with the output layer to output a prediction result.
4-3) training the model established in the step 4-2) to obtain a trained flame detection model; the method comprises the following specific steps:
4-3-1) randomly selecting a picture from the training set after the standardization and normalization processing in the step 4-1) and inputting the picture into the model established in the step 4-2); if the input is the first picture, the feature map corresponding to the picture is output by the penultimate convolutional layer in the deep convolutional neural network, and the feature map size corresponding to each picture is s × s units (the output feature map size s × s is determined by the width and height (W, H) of the input picture and the designed feature map sizeDetermined by the network structure), each unit corresponds to a different position region of the input picture, as shown in fig. 2, the large picture on the left side of fig. 2 is the input picture, the small picture on the right side is the feature map corresponding to the picture, when s is 3, the feature map includes 9 units, which correspond to 9 different position regions in the picture, and the uppermost-left corner unit of the feature map corresponds to the uppermost-left corner position region of the input picture. Then the output of the last convolution layer is connected to the output layer of the whole model through the last convolution layer, and the output is the class probability of k, k is more than or equal to 5 and less than or equal to 9 prediction candidate frames in different positions (s multiplied by s) of the input pictureAnd coordinate offset relative to corresponding standard size box1≤i≤s2J is more than or equal to 1 and less than or equal to k, wherein i represents the ith position area, and j represents the jth prediction candidate frame of the position area. As shown in fig. 2, the solid line frame in fig. 2 is a real labeled frame, only one frame in fig. 2 is shown, the dotted frame represents a prediction candidate frame, each position region has k equal to 5 prediction candidate frames, the dashed frame represents a standard size frame, each position region has k equal to 5 standard size frames, and the relative size of the standard size frame (i.e., the width and height of the standard size frame relative to the whole input picture, such that the standard size frame corresponds to the whole input picture, and the reference size frame indicates the relative size of the prediction candidate frameRespectively representing the width and height of the jth standard-size frame relative to the whole input picture) is represented by FimageThe width and height of the real marking frames of all the pictures relative to the original pictures (the original pictures are the pictures before standardization processing) are obtained by clustering through a k-means method, and the center of each standard size frame is the center of the picture position area where the standard size frame is located; and each standard size frame corresponds to one prediction candidate frame, and the coordinates of the prediction candidate frame can be calculated through the coordinate offset and the coordinates of the standard size frame.
WhereinThe probability of flame existing in the jth prediction candidate frame of the ith position area of the ith picture is represented, the value range is 0-1,closer to 1 indicates a greater likelihood of flame being present in the prediction candidate box;indicating the coordinate offset of the jth prediction candidate frame center relative to the corresponding standard size frame center in the ith position area of the ith picture, (delta wij，Δhij) And the width and the height of the jth prediction candidate frame in the ith position area of the ith picture are represented by the offset of the width and the height of the jth prediction candidate frame relative to the width and the height of the corresponding standard size frame, and the value ranges are 0-1.
In actual operation, the input of the model is a batch of batch, the image data with the size of batch being 8 or more and the size of batch being 64 or less, and the size of batch is batch × W × H × Ch.
4-3-2) dividing s multiplied by k standard size frames of the input picture in the step 4-3-1) into positive and negative samples;
and (3) if the real marked frame in each picture is G, and the frame with any standard size of the picture is T, the Intersection-over-Unit ratio of T and G is (IoU):
wherein area represents a region;
setting a threshold η of a positive sample and a negative sample, wherein η is more than or equal to 0.5 and less than or equal to 0.7, and judging s multiplied by k standard size frames corresponding to each picture, wherein if IoU is more than or equal to η, the standard size frames have flames, and are considered as positive samples, and if IoU is more than η, the standard size frames have no flames, and are considered as negative samples.
4-3-3) calculating the error between the model output information and the real annotation information according to the loss function;
outputting the category probability of s multiplied by k prediction candidate frames corresponding to the first picture and the coordinate offset relative to the corresponding standard size frame according to the model obtained in the step 4-3-1), and calculating the picture prediction informationCorresponding real label informationThe loss function of the ith picture is defined as follows:
wherein,indicating the real type corresponding to the jth prediction candidate frame in the ith position area of the ith picture (the same as the real type corresponding to the corresponding standard size frame), and if the standard size frame corresponding to the prediction candidate frame is a positive sample, determining that the current prediction candidate frame is a positive sampleThe value is 1; if the standard size frame corresponding to the prediction candidate frame is negative sample, thenThe value is 0;
indicating the coordinate offset of the center of the real labeling frame of the ith picture relative to the center of the jth standard size frame of the ith position area,the offset of the width and height of the actual marking frame of the ith picture relative to the width and height of the jth standard size frame of the ith position area (if the standard size frame is a positive sample, namelyThe coordinate information of the real label box makes sense). Relative offset of real label boxFrom FannotationThe marking information in the xml file is obtained by calculation, for example, the marking information of the flame area of the first picture isThe coordinate information of the jth standard size frame of the ith position area relative to the whole input picture is(Obtained by clustering with k-means,i.e. the coordinates of the center of the ith position area relative to the whole input picture), the relative offset of the real mark frame of the ith pictureThe calculation expressions are respectively as follows:
wherein,respectively represent F corresponding to the first pictureimageThe width and height of the original picture; lambda [ alpha ]locThe weight coefficient indicating the position error loss relative to the class error loss can be generally set to 0.01 ≦ λloc≤0.2；A boolean value (positive sample is 1, negative sample is 0) indicating whether the jth prediction candidate box of the ith position region is a flame;boolean values (positive sample 0, negative sample 1) indicating whether the ith cell, the jth prediction candidate box, is not a flame; since the number of candidate regions for negative samples is much larger than that for positive samples in the whole picture,the loss of terms will be much greater thanThus introducing a weighting factor lambdanoobjTo adjust the class loss ratio of the positive and negative samples, it can be set to 0.01 ≦ λnoobj≤0.2。
4-3-4) repeating steps 4-3-1) to 4-3-3), and combining the training set FtrainInputting each picture into the model established in the step 4-1), carrying out iterative training on the whole neural network model, updating the model through an error back propagation algorithm, and obtaining a current model and marking as Modelold；
4-3-5) Using test set FtestFor ModeloldTesting is carried out, and a test set FtestEach picture is input into a Model in sequenceoldCalculating F according to equation (3)testAll pictures in the ModeloldLoss of total Loss ofold；
4-3-6) repeat step 4-3-4), training set FtrainEach picture is input into a Model in sequenceoldObtaining a new current Model and marking as a Modelnew；
4-3-7) repeat step 4-3-5), using test set FtestFor ModelnewTesting was carried out to give FtestAll pictures in the ModelnewLoss of total Loss ofnew；
4-3-8) training stopping condition judgment:
if Lossnew≤LossoldModel, which shows that the test error is still decreasingnewIn test set FtestThe performance of the system is superior to that of a ModeloldContinuing to train the Model and setting the current ModelnewUpdate to a new ModeloldWill be the current LossnewUpdate to New LossoldAnd returning to the step 4-3-6);
if Lossnew≥LossoldWhen the test error begins to rise, the ModeloldIn test set FtestThe performance of the system is superior to that of a ModelnewIf so, the Model stops training and outputs the current ModeloldAnd 5) as a finally trained flame detection model, entering the step 5).
5) Carrying out flame detection on the real-time video by using the flame detection model obtained in the step 4);
5-1) shooting a real-time video, inputting each frame of picture of the real-time video into the flame detection model trained in the step 4) for detection, and outputting the class probability of s multiplied by k prediction candidate frames corresponding to the frame of picture and the corresponding standard by the modelCoordinate offset of size boxWhere the superscript o represents the flame detection model;
5-2) setting a flame judgment threshold value, wherein the value range is 0-1, and judging the category probability of a prediction candidate frame output by each frame of picture: if it isThen no flame is present in the prediction candidate box; if it isThe predicted candidate frame has a flame and the amount of coordinate offset of the predicted candidate frame with respect to the corresponding standard-sized frame is determined based on the flameAnd corresponding standard size frame coordinatesCalculating to obtain the vertex coordinates of the upper left corner and the lower right corner of the actually predicted flame area as
Wherein,respectively representing the original width and height of the frame picture;
5-3) if all the prediction candidate frames of the frame have no flame, returning to the step 5-1) again, and carrying out flame detection on the next frame; if any prediction candidate frame has flame, the model outputs the vertex coordinates of the actually predicted flame area in the upper left corner and the lower right corner in the picture, which are obtained by calculation in the step 5-2), and the coordinates are marked by using a rectangular frame, and then the step 5-1) is returned again), the flame detection of the next frame of picture is continued, and then the marked picture and the fire alarm signal can be sent to the control center.
The above method can be implemented by programming by those skilled in the art.
One embodiment of the present invention is as follows:
the flame detection method based on image target detection provided by the embodiment comprises the following steps:
1) collecting N (N is 5075) color pictures with flame pictures, and establishing a flame picture data set Fimage. The collected pictures are all gathered from the network, and all the pictures contain obvious flame pictures.
2) For the flame picture dataset F of step 1)imageManually marking the coordinates of the flame area position of each picture by using a rectangular frame, as shown in (a) and (b) of fig. 3, framing the flame area by using the rectangular frame, and obtaining a real marking frame of the flame area position and 4 coordinate values of the real marking frame in each picture, wherein the coordinate values are as follows:1, 2, N, whereinThe position coordinates of the top left corner vertex of the frame are really marked for the first picture,the position coordinates of the top point of the lower right corner of the frame are really marked for the first picture; and storing the labeling information as xml files in the format of a VOC007 data set, wherein each picture corresponds to one xml file, and the xml files corresponding to all flame pictures form a flame labeling data set Fannotation。FimageAnd FannotationForm a complete flame detection data set F0。
3) Detecting the flame data set F obtained in the step 2)0Random partitioning into training sets FtrainAnd test set Ftest(due to F)imagePictures and F inannotationThe xml files in (1) are in one-to-one correspondence, and the training set F is dividedtrainAnd test set FtestThe images obtained by the middle division and the xml files are in one-to-one correspondence), wherein the training set FtrainOccupied flame detection dataset F0In a proportion of 80% to FtrainIncluding 4060 pictures and corresponding label data, FtestIncluding 1015 pictures and corresponding labeled data, training set FtrainParameters to train the model, test set FtestTo test the generalization performance of the model.
4) Constructing a deep convolution neural network model and training the model to obtain a trained flame detection model; the method comprises the following specific steps:
4-1) carrying out standardization and normalization processing on each picture of the training set and the test set;
in this embodiment, the standard width and height are set to W and H (W416, H416), and the number Ch of channels of the color picture is 3. And (3) carrying out normalization processing on the pixel data of the zoomed picture according to a formula (1) to enable the pixel value to be between 0 and 1. The input of the model is picture data of one batch of batch 16, and the input size is batch × W × H × Ch.
4-2) constructing a deep convolutional neural network model, which comprises an input layer, a feature extraction part and an output layer: wherein, the normalized picture is input from the input layer and connected with the feature extraction part; the characteristic extraction part consists of 22 convolutional layers and 5 pooling layers, wherein each pooling layer comprises 1-5 convolutional layers in front, and a batch standard layer is added behind each convolutional layer for normalizing each batch of input data in an output characteristic diagram of the convolutional layer; and finally, the characteristic extraction part is connected with the output layer to output a prediction result.
4-3) training the model established in the step 4-2) to obtain a trained flame detection model; the method comprises the following specific steps:
4-3-1) randomly selecting a picture from the training set after the standardization and normalization processing in the step 4-1) and inputting the picture into the model established in the step 4-2); and if the input is the l picture, outputting a feature map corresponding to the picture by using the penultimate convolutional layer in the deep convolutional neural network, wherein the feature map corresponding to each picture is s × s units (s is 13), and each unit corresponds to a different position area of the input picture. The output of the last convolution layer is connected to the output layer of the whole model through the last convolution layer, and the output is the class probability of k-5 prediction candidate frames in different positions (s multiplied by s) of the input pictureAnd coordinate offset relative to corresponding standard size box1≤i≤s2J is more than or equal to 1 and less than or equal to k, wherein i represents the ith position area, and j represents the jth prediction candidate frame of the position area. By making a pair FimageK-means clustering is carried out on the real marking frames of all the pictures relative to the width and the height of the original pictures to obtain the relative width of 5 standard-size framesAnd heightComprises the following steps: (0.238,0.292),(0.754,0.819),(0.325,0.572),(0.094,0.117),(0.597,0.407).
4-3-2) dividing the s × s × k standard size frames of the input picture in the step 4-3-1) into positive and negative samples, setting the intersection ratio threshold η of the positive and negative samples to be 0.6, and judging the s × s × k standard size frames corresponding to each picture, wherein if IoU is larger than or equal to η, flames exist in the standard size frames, the standard size frames are considered to be positive samples, and if IoU is smaller than η, flames do not exist in the standard size frames, and the standard size frames are considered to be negative samples.
4-3-3) calculating the error between the model output information and the real annotation information according to the loss function;
outputting the category probability of s multiplied by k prediction candidate frames corresponding to the first picture and the coordinate offset relative to the corresponding standard size frame according to the model obtained in the step 4-3-1), and calculating the picture prediction informationCorresponding real label informationThe loss function of the first picture is defined in formula (3).
Wherein the weight coefficient of the position error loss relative to the class error loss is set as lambdalocThe weight coefficient for adjusting the class loss ratio of the positive and negative samples is set to λ 0.2noobj＝0.2。
4-3-4) repeating steps 4-3-1) to 4-3-3), and combining the training set FtrainInputting each picture into the Model established in the step 4-1), carrying out iterative training on the whole neural network Model, updating the Model through an error back propagation algorithm, and obtaining a current Model and recording the current Model as a Modelold；
4-3-5) Using test set FtestFor ModeloldTesting is carried out, and a test set FtestEach picture is input into a Model in sequenceoldCalculating F according to equation (3)testAll pictures in the ModeloldLoss of total Loss ofold；
4-3-6) repeat step 4-3-4), training set FtrainEach picture is input into a Model in sequenceoldObtaining a new current Model and marking as a Modelnew；
4-3-7) repeat step 4-3-5), using test set FtestFor ModelnewTesting was carried out to give FtestAll pictures in the ModelnewLoss of total Loss ofnew；
4-3-8) training stopping condition judgment:
if Lossnew≤LossoldModel, which shows that the test error is still decreasingnewIn test set FtestThe performance of the system is superior to that of a ModeloldContinuing to train the Model and setting the current ModelnewUpdate to a new ModeloldWill be the current LossnewUpdate to New LossoldAnd returning to the step 4-3-6);
if Lossnew≥LossoldWhen the test error begins to rise, the ModeloldIn test set FtestThe performance of the system is superior to that of a ModelnewIf so, the Model stops training and outputs the current ModeloldAnd 5) as a finally trained flame detection model, entering the step 5).
5) Carrying out flame detection on the real-time video by using the flame detection model obtained in the step 4);
5-1) shooting a real-time video, inputting each frame of picture of the real-time video into the flame detection model trained in the step 4) for detection, and outputting the class probability of s multiplied by k prediction candidate frames corresponding to the frame of picture and the coordinate offset relative to a standard size frame by the modelWherein the superscript o representsAnd (4) a flame detection model.
5-2) setting a flame judgment threshold value, wherein the value range is 0-1, and judging the category probability of a prediction candidate frame output by each frame of picture: if it isThen no flame is present in the prediction candidate box; if it isThe predicted candidate frame has a flame and the coordinate offset of the predicted candidate frame with respect to the corresponding standard-sized frame is usedAnd corresponding standard size frame coordinatesCalculating the vertex coordinates of the upper left corner and the lower right corner of the actually predicted flame area according to the formulas (8) to (11) to obtain
5-3) if all the prediction candidate frames of the frame have no flame, returning to the step 5-1) again, and carrying out flame detection on the next frame; if any prediction candidate frame has flame, the model outputs the vertex coordinates of the actually predicted flame area in the upper left corner and the lower right corner of the picture, which are calculated in the step 5-2), and the coordinates are marked by using a rectangular frame, as shown in (a) to (e) in fig. 4, and then the step 5-1 is returned again, the flame detection of the next frame of picture is continued, and then the marked picture and the fire alarm signal can be sent to the control center.
Claims (1)
1. A flame detection method based on image target detection is characterized by comprising the following steps:
1) collecting N color pictures with flame pictures to establish a flame picture data set Fimage，N≥5000；
2) For the flame picture data set F established in the step 1)imageManually marking the flame area position coordinates of each picture by using a rectangular frame to obtain a real marking frame of the flame area position and 4 coordinate values of the real marking frame in each picture, wherein the 4 coordinate values are as follows:whereinThe position coordinates of the top left corner vertex of the frame are really marked for the first picture,the position coordinates of the top point of the lower right corner of the frame are really marked for the first picture; storing the information of the real marking frame of each picture as an xml file, wherein the xml files corresponding to all the flame pictures form a flame marking data set Fannotation；FimageAnd FannotationForming a flame detection dataset F0；
3) Detecting the flame obtained in step 2) by using a data set F0Random partitioning into training sets FtrainAnd test set FtestWherein the training set FtrainOccupied flame detection dataset F0The ratio of a is more than or equal to 0.6 and less than or equal to 0.9;
4) constructing a deep convolution neural network model and training the model to obtain a trained flame detection model; the method comprises the following specific steps:
4-1) carrying out standardization and normalization processing on each picture of the training set and the test set;
standardizing each picture of the training set and the test set established in the step 1), wherein the standard width and the standard height are respectively W and H, W is more than or equal to 400 and less than or equal to 800, H is more than or equal to 400 and less than or equal to 800, and the channel number Ch is 3; taking the pixel value of the t channel and the coordinate (x, y) of any image after the normalization processing as I (t, x, y), wherein I (t, x, y) is more than or equal to 0 and less than or equal to 255, calculating the normalized pixel value I '(t, x, y) by using the formula (1), wherein I' (t, x, y) is more than or equal to 0 and less than or equal to 1:
4-2) constructing a deep convolutional neural network model, wherein the model comprises an input layer, a feature extraction part and an output layer which are sequentially connected; the characteristic extraction part is formed by combining convolution layers and pooling layers and comprises 3-6 pooling layers, wherein each pooling layer comprises 1-5 convolution layers in front of the pooling layer, and a batch standard layer is added behind each convolution layer;
4-3) training the model established in the step 4-2) to obtain a trained flame detection model; the method comprises the following specific steps:
4-3-1) sequentially inputting each picture of the training set subjected to the standardization and normalization processing in the step 4-1) into the model established in the step 4-2); inputting the first picture, outputting a feature map corresponding to the picture by the penultimate convolutional layer in the deep convolutional neural network, wherein the feature map is s multiplied by s units, each unit corresponds to different position areas of the input picture, then connecting the output of the penultimate convolutional layer to an output layer of the whole model through the last convolutional layer, outputting the output of the penultimate convolutional layer as k in the s multiplied by s different position areas of the input picture, wherein k is more than or equal to 5 and less than or equal to 9, and the class probability of each prediction candidate frameAnd coordinate offset relative to corresponding standard size box1≤i≤s2J is more than or equal to 1 and less than or equal to k, wherein i represents the ith position area, and j represents the jth prediction candidate frame of the position area; the width and height of the standard size frame relative to the whole input picture are represented by FimageThe width and height of the real mark frame of all pictures relative to the original pictures are obtained by clustering by a k-means method, so thatRespectively representing the width and the height of the jth standard size frame relative to the whole input picture; the center of each standard size frame is the center of the picture position area where the standard size frame is located; each standard size frame corresponds to a prediction candidate frame;
wherein,the probability that flame exists in the jth prediction candidate frame in the ith position area of the ith picture is represented, and the value range is 0-1;the coordinate offset of the jth prediction candidate frame center relative to the corresponding standard size frame center in the ith position area of the ith picture is represented, and the value range is 0-1 (delta wij，Δhij) Representing the width and height offset of a jth prediction candidate frame in an ith position area of the ith picture relative to the width and height of a corresponding standard size frame, wherein the value range is 0-1;
4-3-2) dividing s multiplied by k standard size frames of the input picture in the step 4-3-1) into positive and negative samples;
if the real labeled frame in each picture is G, and any standard size frame of the picture is T, the intersection ratio IoU of T and G is calculated as follows:
wherein area represents a region;
setting a threshold η of the positive and negative samples, wherein η is more than or equal to 0.5 and less than or equal to 0.7, and judging s multiplied by k standard size frames corresponding to each picture, wherein if IoU is more than or equal to η, flames exist in the standard size frames, the standard size frames are positive samples, and if IoU is less than η, the standard size frames do not exist, the standard size frames are negative samples;
4-3-3) calculating the error between the model output information and the real annotation information according to the loss function;
outputting the category probability of s multiplied by k prediction candidate frames corresponding to the first picture and the coordinate offset relative to the corresponding standard size frame according to the model obtained in the step 4-3-1), and calculating the picture prediction informationCorresponding real label informationThe error between; the loss function for the ith picture is defined as follows:
wherein,representing the real category corresponding to the jth prediction candidate frame in the ith position area of the ith picture, if the standard size frame corresponding to the prediction candidate frame is a positive sample, thenThe value is 1; if the standard size frame corresponding to the prediction candidate frame is negative sample, thenThe value is 0;indicating the coordinate offset of the center of the real labeling frame of the ith picture relative to the center of the jth standard size frame of the ith position area,the offset of the width and the height of the real marking frame of the ith picture relative to the width and the height of the jth standard size frame of the ith position area respectively is represented; relative offset of real marking frame of first pictureThe calculation expressions are respectively as follows:
wherein,respectively represent F corresponding to the first pictureimageThe width and height of the original picture;coordinate information of the jth standard size frame of the ith position area of the ith picture relative to the whole input picture,is the coordinate of the center of the ith position area relative to the whole input picture,obtained by k-means clustering; lambda [ alpha ]locA weight coefficient representing the position error loss relative to the class error loss, 0.01 ≦ λloc≤0.2；A Boolean value indicating whether the jth prediction candidate frame of the ith position area is a flame or not, if the standard size frame corresponding to the prediction candidate frame is a positive sample, the prediction candidate frame is divided into a plurality of regionsThe value is 1; if the standard size frame corresponding to the prediction candidate frame is negative sample, thenThe value is 0;a Boolean value indicating whether the jth prediction candidate frame of the ith position area is not the flame, if the standard size frame corresponding to the prediction candidate frame is a positive sample, thenThe value is 0; if the standard size frame corresponding to the prediction candidate frame is negative sample, thenThe value is 1; lambda [ alpha ]noobjλ is 0.01 ≦ for adjusting the weighting factor of the class loss ratio of the positive and negative samplesnoobj≤0.2；
4-3-4) repeating steps 4-3-1) to 4-3-3), and combining the training set FtrainInputting each picture into the Model established in the step 4-1), updating the Model through an error back propagation algorithm to obtain a current Model and recording the current Model as a Modelold；
4-3-5) Using test set FtestFor ModeloldTesting is carried out, and a test set FtestEach picture is input into a Model in sequenceoldCalculating F according to equation (3)testAll pictures in the ModeloldLoss of total Loss ofold；
4-3-6) repeat step 4-3-4), training set FtrainEach picture is input into a Model in sequenceoldObtaining a new current Model and marking as a Modelnew；
4-3-7) repeat step 4-3-5), using test set FtestFor ModelnewTesting was carried out to give FtestAll pictures in the ModelnewLoss of total Loss ofnew；
4-3-8) training stopping condition judgment:
if Lossnew≤LossoldContinuing to train the Model and setting the current ModelnewUpdate to a new ModeloldWill be the current LossnewUpdate to New LossoldAnd returning to the step 4-3-6);
if Lossnew≥LossoldWhen the Model is not trained, the Model stops training and outputs the current ModeloldTaking the flame detection model as a finally trained flame detection model, and entering the step 5);
5) carrying out flame detection on the real-time video by using the flame detection model obtained in the step 4);
5-1) shooting a real-time video, inputting each frame of picture of the real-time video into the flame detection model trained in the step 4) for detection, and outputting the class probability of s multiplied by k prediction candidate frames corresponding to the frame of picture and the coordinate offset relative to the corresponding standard size frame by the modelWhere the superscript o represents the flame detection model;
5-2) setting a flame judgment threshold value, wherein the value range is 0-1, and judging the category probability of a prediction candidate frame output by each frame of picture: if it isThen no flame is present in the prediction candidate box; if it isThe predicted candidate frame has a flame and the amount of coordinate offset of the predicted candidate frame with respect to the corresponding standard-sized frame is determined based on the flameAnd corresponding standard size frame coordinatesCalculating to obtain the vertex coordinates of the upper left corner and the lower right corner of the actually predicted flame area as
Wherein,respectively representing the original width and height of the frame picture;
5-3) if all the prediction candidate frames of the frame have no flame, returning to the step 5-1) again, and carrying out flame detection on the next frame; if any prediction candidate frame has flame, the model outputs the vertex coordinates of the actually predicted flame area in the upper left corner and the lower right corner of the picture, which are obtained by calculation in the step 5-2), and the coordinates are marked by using a rectangular frame, and then the step 5-1) is returned again to continue to detect the flame of the next frame of picture.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810243014.4A CN108537215B (en) | 2018-03-23 | 2018-03-23 | Flame detection method based on image target detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810243014.4A CN108537215B (en) | 2018-03-23 | 2018-03-23 | Flame detection method based on image target detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108537215A true CN108537215A (en) | 2018-09-14 |
| CN108537215B CN108537215B (en) | 2020-02-21 |
Family
ID=63483653
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810243014.4A Active CN108537215B (en) | 2018-03-23 | 2018-03-23 | Flame detection method based on image target detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108537215B (en) |
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109360369A (en) * | 2018-09-19 | 2019-02-19 | 李琳 | One kind carrying out fire hazard analysis method and device based on cluster result |
| CN109522819A (en) * | 2018-10-29 | 2019-03-26 | 西安交通大学 | A kind of fire image recognition methods based on deep learning |
| CN109568123A (en) * | 2018-11-02 | 2019-04-05 | 广东数相智能科技有限公司 | A kind of acupuncture point localization method based on YOLO target detection |
| CN109614990A (en) * | 2018-11-20 | 2019-04-12 | 成都通甲优博科技有限责任公司 | A kind of object detecting device |
| CN109684982A (en) * | 2018-12-19 | 2019-04-26 | 哈尔滨工业大学（深圳） | The flame detecting method based on video analysis excluded in conjunction with easy mixed target |
| CN109829429A (en) * | 2019-01-31 | 2019-05-31 | 福州大学 | Security protection sensitive articles detection method under monitoring scene based on YOLOv3 |
| CN110059753A (en) * | 2019-04-19 | 2019-07-26 | 北京朗镜科技有限责任公司 | Model training method, interlayer are every recognition methods, device, equipment and medium |
| CN110119682A (en) * | 2019-04-04 | 2019-08-13 | 北京理工雷科电子信息技术有限公司 | A kind of infrared remote sensing Image Fire point recognition methods |
| CN110287982A (en) * | 2019-05-08 | 2019-09-27 | 中国科学技术大学 | A kind of CT images classification method, device and medium based on convolutional neural networks |
| CN110348390A (en) * | 2019-07-12 | 2019-10-18 | 创新奇智(重庆)科技有限公司 | A kind of training method, computer-readable medium and the system of fire defector model |
| CN110569703A (en) * | 2019-05-10 | 2019-12-13 | 阿里巴巴集团控股有限公司 | computer-implemented method and device for identifying damage from picture |
| CN110728186A (en) * | 2019-09-11 | 2020-01-24 | 中国科学院声学研究所南海研究站 | Fire detection method based on multi-network fusion |
| CN110826439A (en) * | 2019-10-25 | 2020-02-21 | 杭州叙简科技股份有限公司 | Electric welding construction detection method based on deep learning image processing |
| CN111127433A (en) * | 2019-12-24 | 2020-05-08 | 深圳集智数字科技有限公司 | Method and device for detecting flame |
| CN111178275A (en) * | 2019-12-30 | 2020-05-19 | 浙江中创天成科技有限公司 | Fire detection method based on convolutional neural network |
| CN111310662A (en) * | 2020-02-17 | 2020-06-19 | 淮阴工学院 | Flame detection and identification method and system based on integrated deep network |
| CN111325047A (en) * | 2018-12-13 | 2020-06-23 | 博西华电器（江苏）有限公司 | Cooking safety auxiliary method and system, kitchen appliance and combination thereof |
| CN111353555A (en) * | 2020-05-25 | 2020-06-30 | 腾讯科技（深圳）有限公司 | Label detection method and device and computer readable storage medium |
| US10885625B2 (en) | 2019-05-10 | 2021-01-05 | Advanced New Technologies Co., Ltd. | Recognizing damage through image analysis |
| CN112560834A (en) * | 2019-09-26 | 2021-03-26 | 武汉金山办公软件有限公司 | Coordinate prediction model generation method and device and graph recognition method and device |
| CN112633061A (en) * | 2020-11-18 | 2021-04-09 | 淮阴工学院 | Lightweight FIRE-DET flame detection method and system |
| CN113012383A (en) * | 2021-03-26 | 2021-06-22 | 深圳市安软科技股份有限公司 | Fire detection alarm method, related system, related equipment and storage medium |
| CN113033553A (en) * | 2021-03-22 | 2021-06-25 | 深圳市安软科技股份有限公司 | Fire detection method and device based on multi-mode fusion, related equipment and storage medium |
| CN113052226A (en) * | 2021-03-22 | 2021-06-29 | 淮阴工学院 | Time-sequence fire identification method and system based on single-step detector |
| CN113221768A (en) * | 2021-05-18 | 2021-08-06 | 北京百度网讯科技有限公司 | Recognition model training method, recognition method, device, equipment and storage medium |
| CN113657250A (en) * | 2021-08-16 | 2021-11-16 | 南京图菱视频科技有限公司 | Flame detection method and system based on monitoring video |
| CN114841920A (en) * | 2022-03-29 | 2022-08-02 | 清华大学 | Flame identification method and device based on image processing and electronic equipment |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102208018A (en) * | 2011-06-01 | 2011-10-05 | 西安工程大学 | Method for recognizing fire disaster of power transmission line based on video variance analysis |
| CN103116746A (en) * | 2013-03-08 | 2013-05-22 | 中国科学技术大学 | Video flame detecting method based on multi-feature fusion technology |
| CN104408469A (en) * | 2014-11-28 | 2015-03-11 | 武汉大学 | Firework identification method and firework identification system based on deep learning of image |
| CN105678332A (en) * | 2016-01-08 | 2016-06-15 | 昆明理工大学 | Converter steel-making endpoint determination method and system based on flame image CNN recognizing and modeling process |
| CN106250845A (en) * | 2016-07-28 | 2016-12-21 | 北京智芯原动科技有限公司 | Flame detecting method based on convolutional neural networks and device |
| CN106934404A (en) * | 2017-03-10 | 2017-07-07 | 深圳市瀚晖威视科技有限公司 | A kind of image flame identifying system based on CNN convolutional neural networks |
| CN107025443A (en) * | 2017-04-06 | 2017-08-08 | 江南大学 | Stockyard smoke monitoring and on-time model update method based on depth convolutional neural networks |
-
2018
- 2018-03-23 CN CN201810243014.4A patent/CN108537215B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102208018A (en) * | 2011-06-01 | 2011-10-05 | 西安工程大学 | Method for recognizing fire disaster of power transmission line based on video variance analysis |
| CN103116746A (en) * | 2013-03-08 | 2013-05-22 | 中国科学技术大学 | Video flame detecting method based on multi-feature fusion technology |
| CN104408469A (en) * | 2014-11-28 | 2015-03-11 | 武汉大学 | Firework identification method and firework identification system based on deep learning of image |
| CN105678332A (en) * | 2016-01-08 | 2016-06-15 | 昆明理工大学 | Converter steel-making endpoint determination method and system based on flame image CNN recognizing and modeling process |
| CN106250845A (en) * | 2016-07-28 | 2016-12-21 | 北京智芯原动科技有限公司 | Flame detecting method based on convolutional neural networks and device |
| CN106934404A (en) * | 2017-03-10 | 2017-07-07 | 深圳市瀚晖威视科技有限公司 | A kind of image flame identifying system based on CNN convolutional neural networks |
| CN107025443A (en) * | 2017-04-06 | 2017-08-08 | 江南大学 | Stockyard smoke monitoring and on-time model update method based on depth convolutional neural networks |
Non-Patent Citations (3)
| Title |
|---|
| BING WANG ET AL.: "The real-time estimation of hazardous gas dispersion by theintegration of gas detectors, neural network and gas dispersion models", 《JOURNAL OF HAZARDOUS MATERIALS》 * |
| YI ZHAO ET AL.: "Saliency Detection and Deep Learning-Based Wildfire Identification in UAV Imagery", 《SENSORS》 * |
| 王勇 等: "基于稀疏自编码深度神经网络的林火图像分类", 《计算机工程与应用》 * |
Cited By (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109360369A (en) * | 2018-09-19 | 2019-02-19 | 李琳 | One kind carrying out fire hazard analysis method and device based on cluster result |
| CN109522819A (en) * | 2018-10-29 | 2019-03-26 | 西安交通大学 | A kind of fire image recognition methods based on deep learning |
| CN109568123A (en) * | 2018-11-02 | 2019-04-05 | 广东数相智能科技有限公司 | A kind of acupuncture point localization method based on YOLO target detection |
| CN109614990A (en) * | 2018-11-20 | 2019-04-12 | 成都通甲优博科技有限责任公司 | A kind of object detecting device |
| CN111325047A (en) * | 2018-12-13 | 2020-06-23 | 博西华电器（江苏）有限公司 | Cooking safety auxiliary method and system, kitchen appliance and combination thereof |
| CN109684982A (en) * | 2018-12-19 | 2019-04-26 | 哈尔滨工业大学（深圳） | The flame detecting method based on video analysis excluded in conjunction with easy mixed target |
| CN109684982B (en) * | 2018-12-19 | 2020-11-20 | 深圳前海中创联科投资发展有限公司 | Flame detection method based on video analysis and combined with miscible target elimination |
| CN109829429A (en) * | 2019-01-31 | 2019-05-31 | 福州大学 | Security protection sensitive articles detection method under monitoring scene based on YOLOv3 |
| CN110119682A (en) * | 2019-04-04 | 2019-08-13 | 北京理工雷科电子信息技术有限公司 | A kind of infrared remote sensing Image Fire point recognition methods |
| CN110059753A (en) * | 2019-04-19 | 2019-07-26 | 北京朗镜科技有限责任公司 | Model training method, interlayer are every recognition methods, device, equipment and medium |
| CN110287982A (en) * | 2019-05-08 | 2019-09-27 | 中国科学技术大学 | A kind of CT images classification method, device and medium based on convolutional neural networks |
| CN110569703A (en) * | 2019-05-10 | 2019-12-13 | 阿里巴巴集团控股有限公司 | computer-implemented method and device for identifying damage from picture |
| US10885625B2 (en) | 2019-05-10 | 2021-01-05 | Advanced New Technologies Co., Ltd. | Recognizing damage through image analysis |
| CN110348390A (en) * | 2019-07-12 | 2019-10-18 | 创新奇智(重庆)科技有限公司 | A kind of training method, computer-readable medium and the system of fire defector model |
| CN110728186A (en) * | 2019-09-11 | 2020-01-24 | 中国科学院声学研究所南海研究站 | Fire detection method based on multi-network fusion |
| CN112560834A (en) * | 2019-09-26 | 2021-03-26 | 武汉金山办公软件有限公司 | Coordinate prediction model generation method and device and graph recognition method and device |
| CN112560834B (en) * | 2019-09-26 | 2024-05-10 | 武汉金山办公软件有限公司 | Coordinate prediction model generation method and device and pattern recognition method and device |
| CN110826439A (en) * | 2019-10-25 | 2020-02-21 | 杭州叙简科技股份有限公司 | Electric welding construction detection method based on deep learning image processing |
| CN111127433B (en) * | 2019-12-24 | 2020-09-25 | 深圳集智数字科技有限公司 | Method and device for detecting flame |
| CN111127433A (en) * | 2019-12-24 | 2020-05-08 | 深圳集智数字科技有限公司 | Method and device for detecting flame |
| CN111178275A (en) * | 2019-12-30 | 2020-05-19 | 浙江中创天成科技有限公司 | Fire detection method based on convolutional neural network |
| CN111310662A (en) * | 2020-02-17 | 2020-06-19 | 淮阴工学院 | Flame detection and identification method and system based on integrated deep network |
| CN111353555A (en) * | 2020-05-25 | 2020-06-30 | 腾讯科技（深圳）有限公司 | Label detection method and device and computer readable storage medium |
| CN112633061B (en) * | 2020-11-18 | 2023-03-24 | 淮阴工学院 | Lightweight FIRE-DET flame detection method and system |
| CN112633061A (en) * | 2020-11-18 | 2021-04-09 | 淮阴工学院 | Lightweight FIRE-DET flame detection method and system |
| CN113033553A (en) * | 2021-03-22 | 2021-06-25 | 深圳市安软科技股份有限公司 | Fire detection method and device based on multi-mode fusion, related equipment and storage medium |
| CN113052226A (en) * | 2021-03-22 | 2021-06-29 | 淮阴工学院 | Time-sequence fire identification method and system based on single-step detector |
| CN113033553B (en) * | 2021-03-22 | 2023-05-12 | 深圳市安软科技股份有限公司 | Multi-mode fusion fire detection method, device, related equipment and storage medium |
| CN113012383B (en) * | 2021-03-26 | 2022-12-30 | 深圳市安软科技股份有限公司 | Fire detection alarm method, related system, related equipment and storage medium |
| CN113012383A (en) * | 2021-03-26 | 2021-06-22 | 深圳市安软科技股份有限公司 | Fire detection alarm method, related system, related equipment and storage medium |
| CN113221768A (en) * | 2021-05-18 | 2021-08-06 | 北京百度网讯科技有限公司 | Recognition model training method, recognition method, device, equipment and storage medium |
| CN113657250A (en) * | 2021-08-16 | 2021-11-16 | 南京图菱视频科技有限公司 | Flame detection method and system based on monitoring video |
| CN114841920A (en) * | 2022-03-29 | 2022-08-02 | 清华大学 | Flame identification method and device based on image processing and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108537215B (en) | 2020-02-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108537215B (en) | Flame detection method based on image target detection | |
| CN110059554B (en) | Multi-branch target detection method based on traffic scene | |
| CN111126136B (en) | Smoke concentration quantification method based on image recognition | |
| CN111754498B (en) | Conveyor belt carrier roller detection method based on YOLOv3 | |
| CN111753828B (en) | Natural scene horizontal character detection method based on deep convolutional neural network | |
| CN110717481B (en) | Method for realizing face detection by using cascaded convolutional neural network | |
| CN110675370A (en) | Welding simulator virtual weld defect detection method based on deep learning | |
| CN114283469B (en) | Improved YOLOv4-tiny target detection method and system | |
| CN111898406A (en) | Face detection method based on focus loss and multitask cascade | |
| CN105574550A (en) | Vehicle identification method and device | |
| CN106408030A (en) | SAR image classification method based on middle lamella semantic attribute and convolution neural network | |
| CN113569788B (en) | Building semantic segmentation network model training method, system and application method | |
| CN109948527B (en) | Small sample terahertz image foreign matter detection method based on integrated deep learning | |
| CN106022254A (en) | Image recognition technology | |
| CN112132005A (en) | Face detection method based on cluster analysis and model compression | |
| CN107392251B (en) | Method for improving target detection network performance by using classified pictures | |
| CN110503092B (en) | Improved SSD monitoring video target detection method based on field adaptation | |
| CN113139500A (en) | Smoke detection method, system, medium and device | |
| CN102867183A (en) | Method and device for detecting littered objects of vehicle and intelligent traffic monitoring system | |
| WO2023160666A1 (en) | Target detection method and apparatus, and target detection model training method and apparatus | |
| CN116824335A (en) | YOLOv5 improved algorithm-based fire disaster early warning method and system | |
| CN114841920A (en) | Flame identification method and device based on image processing and electronic equipment | |
| CN118097755A (en) | Intelligent face identity recognition method based on YOLO network | |
| CN115311601A (en) | Fire detection analysis method based on video analysis technology | |
| CN111368625B (en) | Pedestrian target detection method based on cascade optimization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |