CN117576369A - Fire detection method, fire detection device and storage medium - Google Patents
Fire detection method, fire detection device and storage medium Download PDFInfo
- Publication number
- CN117576369A CN117576369A CN202311441867.6A CN202311441867A CN117576369A CN 117576369 A CN117576369 A CN 117576369A CN 202311441867 A CN202311441867 A CN 202311441867A CN 117576369 A CN117576369 A CN 117576369A
- Authority
- CN
- China
- Prior art keywords
- fog
- training
- image
- target area
- segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 41
- 230000011218 segmentation Effects 0.000 claims abstract description 117
- 239000003595 mist Substances 0.000 claims abstract description 52
- 238000000034 method Methods 0.000 claims abstract description 35
- 230000008859 change Effects 0.000 claims abstract description 15
- 238000012549 training Methods 0.000 claims description 105
- 230000006870 function Effects 0.000 claims description 29
- 238000012545 processing Methods 0.000 claims description 19
- 238000013527 convolutional neural network Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000001914 filtration Methods 0.000 claims description 5
- 230000008569 process Effects 0.000 claims description 3
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 6
- 239000013598 vector Substances 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000009499 grossing Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 3
- 241000287196 Asthenes Species 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000000779 smoke Substances 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 230000008685 targeting Effects 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a fire detection method, a fire detection device and a storage medium. The method comprises the steps of obtaining an original image in a cloud and fog environment; carrying out cloud and mist removal treatment on the original image through a segmentation model insensitive to mist change to obtain a clear image; and identifying the potential fire target area in the clear image, and obtaining and outputting the position and the category of the potential fire target area, wherein the category is used for indicating whether the potential fire target area has a fire or not. The embodiment of the invention is beneficial to improving the accuracy of image identification in a cloud and fog environment, and further effectively detecting and early warning fire.
Description
Technical Field
The present invention relates to the field of computer image and vision technologies, and in particular, to a fire detection method, apparatus, and storage medium.
Background
Fire is an important ring in natural disasters, and early detection and timely intervention of fire have important significance for reducing loss and protecting lives and properties of people. In the field of computer vision, the convolutional neural network has the advantages of high-efficiency feature extraction capability, parameter sharing, hierarchical structure and the like, so that the convolutional neural network can be a powerful tool for processing complex visual tasks. This provides the possibility of fire early warning through fire image recognition. However, in the practical application scene, due to the problems of cloud, small images and the like, the images shot in the fog can show reduced color quality, low contrast and other artifacts associated with low visibility, so that the traditional convolutional neural network does not perform well in the foggy scene, and effective fire detection cannot be performed.
Disclosure of Invention
The embodiment of the invention provides a fire detection method, a fire detection device and a storage medium, which are beneficial to improving the accuracy of image identification in a cloud and fog environment, so as to effectively detect and early warn the fire.
In one aspect, the embodiment of the invention provides a fire detection method, which includes:
acquiring an original image in a cloud environment;
carrying out cloud and mist removal treatment on the original image through a segmentation model insensitive to mist change to obtain a clear image;
and identifying the potential fire target area in the clear image, and obtaining and outputting the position and the category of the potential fire target area, wherein the category is used for indicating whether the potential fire target area has a fire or not.
The cloud and mist removing treatment is carried out on the original image through a segmentation model insensitive to mist change, so that a clear image is obtained, and the method comprises the following steps:
determining the similarity between the fog-invariant feature parameters in the fog-change insensitive segmentation model and the pixels in the original image;
determining a calculation weight of each pixel included in the original image based on the similarity;
a sharp image is determined based on the product between each pixel and the calculated weight.
The identifying the potential fire target area in the clear image to obtain and output the position and the category of the potential fire target area includes:
extracting shallow detail features and deep semantic features in the clear image by using a convolutional neural network to obtain a feature map;
generating a plurality of candidate boxes on the feature map by a candidate box generator, each of the candidate boxes representing a potential fire target area;
determining whether a target exists in each candidate frame to confirm the category of the potential fire target area;
and outputting the position and the category of each potential fire target area.
Wherein prior to said outputting the location and category of each of said potential fire target areas, said method further comprises:
calculating the boundary regression offset of each candidate frame to obtain the accurate coordinates of each potential fire target area;
and taking the accurate coordinates as the position of the potential fire target area.
The method further comprises the following steps of:
acquiring a training set and a fog-pass filter;
alternately training a segmentation network and a fog pass filter through the training set to obtain fog invariant feature parameters;
and inputting the fog invariant feature parameters into an initial segmentation model to obtain a segmentation model insensitive to fog changes.
The training set is used for alternately training a segmentation network and a fog pass filter to obtain a segmentation model comprising fog invariant feature parameters, and the method comprises the following steps:
fixing parameters of the segmentation network, and training the fog-pass filter through the training set;
the step of fixing parameters of the segmentation network, and training the fog-pass filter through the training set specifically comprises the following steps:
carrying out cloud and mist removal treatment on the same training image under different training domains according to the segmentation network with fixed parameters to obtain feature images under different training domains;
calculating fog factors of the feature map under different training domains through the fog-pass filter;
calculating a fog-pass filtering loss function value corresponding to the fog-pass filter according to fog factors in different training domains;
and updating the fog filter according to the fog filter loss function value.
The training set is used for alternately training a segmentation network and a fog pass filter to obtain a segmentation model comprising fog invariant feature parameters, and the method comprises the following steps:
fixing parameters of the fog-pass filter, and training the segmentation network through the training set;
the step of fixing parameters of the fog-pass filter, and training the segmentation network through the training set specifically comprises the following steps:
cloud and mist removing treatment is carried out on the same input training image according to the segmentation network, so that feature images under different training domains are obtained;
calculating fog factors of the feature map under different training domains through a fog-pass filter with fixed parameters;
calculating a fog type matching loss value according to the fog factor;
updating the fog invariant feature parameters of the segmentation network according to the fog matching loss value;
calculating the prediction score of the feature map according to the updated segmentation network;
calculating a segmentation loss value according to the prediction score and a ground truth label of the same input training image;
and updating the segmentation network again according to the segmentation loss value.
In one aspect, an embodiment of the present invention provides a fire detection apparatus, including:
the data acquisition unit is used for acquiring an original image in a cloud and fog environment;
the image processing unit is used for carrying out cloud and mist removal processing on the original image through a segmentation model comprising mist-invariant feature parameters to obtain a clear image, wherein the mist-invariant feature is a feature insensitive to mist change of the original image;
the image recognition unit is used for recognizing the potential fire target area in the clear image to obtain the potential fire target area;
and the data output unit is used for outputting the position and the category of the potential fire target area, wherein the category is used for indicating whether the potential fire target area has a fire or not.
In one aspect, another fire detection apparatus is provided according to an embodiment of the present invention, including: a processor and a memory;
the processor is connected to a memory, wherein the memory is configured to store a computer program, and the processor is configured to invoke the computer program to perform a method as in one aspect of an embodiment of the present invention.
An aspect of an embodiment of the present invention provides a computer-readable storage medium storing a computer program comprising program instructions which, when executed by a processor, perform a method as in an aspect of an embodiment of the present invention.
Before the image is identified, the embodiment of the invention carries out cloud and mist removal treatment on the acquired original image in the cloud and mist environment through the segmentation model insensitive to the mist type change to obtain a clear image, thereby improving the quality of the image and retaining other characteristics except the cloud and mist; meanwhile, when the image is identified, the image is divided, and only the potential fire target area is identified, but not the whole image is identified, so that the accurate detection is performed, and the subsequent identification is facilitated; therefore, the improvement of the accuracy of image identification in the cloud and fog environment can be finally realized, and further effective detection and early warning of fire are facilitated.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of a network architecture according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of a fire detection method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a fire detection device according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of another fire detection device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a schematic diagram of a network architecture according to an embodiment of the present invention. The network architecture may include a plurality of servers and a plurality of terminal devices (as shown in fig. 1, specifically including a terminal device 100a, a terminal device 100b, a terminal device 100c, a server 200a, and a server 200 b), where the server 200a may perform data transmission with each terminal device through a network, and each server or terminal device may install a fire detection application. The terminal devices may include cell phones, tablet computers, notebook computers, palm computers, mobile internet devices (mobile internet device, MID), wearable devices (e.g., smart watches, smart bracelets, etc.), satellites, sensors, etc.
In one embodiment of the present invention, the terminal device 100a is a notebook computer, and the fire detection application is installed; terminal device 100b is a satellite; server 200a may be a background server for the fire detection application. Each terminal device may perform data transmission with the server 200a, for example, the terminal device 100b captures an original image in a cloud environment and transmits the original image to the terminal device 100a in a wireless manner. When the fire detection application on the terminal device 100a is started, the original image is transmitted to the server 200a by a wired or wireless manner for processing. The server 200a performs cloud and mist removal processing on the original image through a segmentation model comprising mist-invariant feature parameters to obtain a clear image; identifying a potential fire target area in the clear image to obtain the potential fire target area; and outputting the position and the category of the potential fire target area to terminal equipment 100c, wherein the terminal equipment 100c is a mobile phone of a related manager, so that early warning of the related manager is realized.
Referring to fig. 2, a flow chart of a fire detection method according to an embodiment of the present invention is shown. As shown in fig. 2, the method comprises the steps of:
step 201: acquiring an original image in a cloud environment;
the application scene of the invention can be under the field condition of forests and the like, and can also occur in cities. Therefore, the cloud and fog environment can be caused by natural reasons such as open air weather, industrial reasons such as city haze, and local dense smoke caused by man-made fire.
In the present invention, the source of the original image is not particularly limited. The original image may be satellite detected, for example in the case of a forest or the like in the field; if the image is generated in the city, the original image can be detected by an image sensor or photographed by a camera, a mobile phone and other devices with camera modules.
In a practical application scene, an original image in a cloud environment has the problems of cloud, small target and the like, and the original image can show reduced color quality, low contrast and other artifacts associated with low visibility due to shooting in the cloud environment. Because of the loss of relevant characteristic information, the direct input into the existing segmentation network can lead to semantic confusion.
Step 202: carrying out cloud and mist removal treatment on the original image through a segmentation model insensitive to mist change to obtain a clear image;
wherein, the fog type refers to fog type, and can be classified into clear weather, synthetic fog and real fog; it may also be classified as a mist concentration, for example, a dense mist, a medium mist, a thin mist. The fog type change is insensitive, namely the influence of different types of fog on the segmentation model is small, the segmentation model comprises fog invariant feature parameters, and the original image is processed through the parameters, so that a clear image can be obtained.
In the method, the segmentation model insensitive to fog change is a trained segmentation model. The method is not a special algorithm for cloud removal and defogging, but is used for training a segmentation network to ensure that the segmentation network has fog invariance, so that the image segmentation performance in foggy weather is improved. The images in the foggy weather can be better processed through the trained segmentation network without a separate cloud and mist removal algorithm, and the effect of the cloud and mist removal algorithm is equivalent to that of cloud and mist removal before identification.
Step 203: and identifying the potential fire target area in the clear image, and obtaining and outputting the position and the category of the potential fire target area, wherein the category is used for indicating whether the potential fire target area has a fire or not.
The clear image may be divided into a plurality of areas, the potential fire target area is a partial area in the plurality of areas, the potential fire target area in the clear image may be identified, only the partial area may be identified, and other partial areas that are obviously not the potential fire target area may not be identified, thereby improving the identification speed. Other partial areas that are not significantly potential fire target areas may be determined from the gray values or pixel values of the pixels, e.g., pixel values that are significantly lower than red are not potential fire target areas.
The position of the potential fire target area may be one or more coordinate values, if one coordinate value is one, the center point of the potential fire target area is indicated, and if a plurality of coordinate values are provided, the area surrounded by the coordinate values is the potential fire target area.
Wherein, the category of the potential fire target area can be fire or no fire, and can be determined by a classification method.
It can be seen that before the image is identified, the embodiment of the invention carries out cloud and mist removal processing on the acquired original image in the cloud and mist environment through the segmentation model insensitive to the mist change to obtain a clear image, thereby improving the quality of the image and retaining other characteristics except the cloud and mist; meanwhile, when the image is identified, the image is divided, and only the potential fire target area is identified, but not the whole image is identified, so that the accurate detection is performed, and the subsequent identification is facilitated; therefore, the improvement of the accuracy of image identification in the cloud and fog environment can be finally realized, and further effective detection and early warning of fire are facilitated.
In one embodiment of the present invention, the performing cloud and mist removing processing on the original image by using the segmentation model insensitive to mist change to obtain a clear image includes:
determining the similarity between the fog-invariant feature parameters in the fog-change insensitive segmentation model and the pixels in the original image;
determining a calculation weight of each pixel included in the original image based on the similarity;
a sharp image is determined based on the product between each pixel and the calculated weight.
The original image is an image subjected to semantic segmentation, and each pixel in the image is assigned with a category according to whether the pixel is an interested fire target or not. The similarity between the fog invariant feature parameters and pixels in the original image may be measured using a correlation coefficient or cosine similarity. A higher similarity will result in a higher weight for the pixel. Once weights are calculated for each pixel, these weights can be applied to the smoothing operation of the image. For example, each pixel is multiplied by its calculated weight to obtain a new pixel, and the new pixel is used as a pixel of a clear image to obtain the clear image, thereby realizing smoothing processing.
Specifically, the determining the similarity between the fog-invariant feature parameters in the fog-change-insensitive segmentation model and the pixels in the original image includes: determining the spatial domain similarity of the fog-invariant feature parameters and the spatial distance between pixels in the fog-change insensitive segmentation model, and determining the value domain similarity of the fog-invariant feature parameters and the pixel values in the fog-change insensitive segmentation model; and determining the overall similarity between the fog-unchanged characteristic parameters in the fog-change-insensitive segmentation model and the pixels in the original image according to the airspace similarity and the value range similarity. For example, the overall similarity may be equal to the product of the spatial similarity and the value range similarity, or may be equal to the product of the weight corresponding to the spatial similarity and the value range similarity plus the product of the weight corresponding to the value range similarity, and the sum of the weight corresponding to the spatial similarity and the weight corresponding to the value range similarity is 1.
In the embodiment of the invention, the fog invariant feature parameters can be used for adjusting the weight of pixel value similarity so as to better preserve edge information. Before the weighted smoothing operation, the weights need to be in a proper range to avoid excessively intense smoothing or excessively intense defogging, and then the weight calculation and smoothing processing are performed on each pixel of the image, so that the personalized processing on each region in the image is ensured to reduce or remove the influence of the defogging.
In one embodiment of the present invention, the identifying the potential fire target area in the clear image, obtaining and outputting the location and the category of the potential fire target area includes:
extracting shallow detail features and deep semantic features in the clear image by using a convolutional neural network to obtain a feature map;
generating a plurality of candidate boxes on the feature map by a candidate box generator, each of the candidate boxes representing a potential fire target area;
determining whether a target exists in each candidate frame to confirm the category of the potential fire target area;
and outputting the position and the category of each potential fire target area.
The shallow detail features refer to features extracted from a shallow network in the convolutional neural network, are close to an input clear image, contain more pixel point information and some fine-grained information such as color, texture, edges and angles. Deep semantic features refer to features extracted from deep networks in convolutional neural networks, which are closer to the output, and some coarse-grained information, such as semantic information. The feature map is composed of two parts, namely shallow detail features and deep semantic features, and compared with single shallow detail features or deep semantic features, the feature map can more comprehensively and completely represent images, so that the information of the images is not lost too much.
Specifically, the extracting shallow detail features and deep semantic features in the clear image by using a convolutional neural network to obtain a feature map includes:
calculating to obtain a spatial attention feature through a spatial attention function and original features in the clear image; calculating to obtain the channel attention feature through the channel attention function and the original feature in the clear image;
fusing the spatial attention characteristic and the channel attention characteristic to obtain an attention characteristic;
and fusing the attention features with the original features to obtain fused features, wherein the fused features form a feature map.
For example, assume that the original features are represented asThe spatial attention features are expressed as +.>Characteristic, channel attention characteristic is expressed as +.>。/>And->Representing the spatial attention function and the channel attention function, respectively, feature extraction can be expressed as:
attention features are expressed asThen:
fusion features are expressed asThen:
in this embodiment, by emphasizing the feature and spatial position information related to the object, interference of the background information can be reduced.
The objective of the candidate box generation phase is to search for locations in the feature map that may contain fire targets, which are also called regions of interest (ROIs). After generating a plurality of candidate frames on the feature map by a candidate frame generator, for each candidate frame, region of interest (ROI) pooling is performed, converting the different sized candidate frames into feature vectors of a fixed size for subsequent classification and regression operations. The candidate boxes are passed through the RoI pooling layer to generate feature maps of fixed size, which are then passed to the classification and regression layer for target detection. This preserves semantic information in the candidate boxes, facilitating further processing of the targeting and classification tasks.
Determining whether there is a target in each of the candidate boxes, i.e. classifying each candidate box using a classification neural network to determine whether they contain a fire target.
In one embodiment of the present invention, before the outputting the location and the category of each of the potential fire target areas, the method further includes:
calculating the boundary regression offset of each candidate frame to obtain the accurate coordinates of each potential fire target area;
and taking the accurate coordinates as the position of the potential fire target area.
Further, before the accurate coordinate is used as the position of the potential fire target area, the accurate coordinate can be subjected to positioning accuracy evaluation, and the accurate coordinate is used as the accurate coordinate after the accurate coordinate is qualified.
The accuracy evaluation may be performed by a method of calculating a cross ratio (Intersection Over Union ratio, iou) defining an overlap frame between two detected frames, the calculation formula being
When the iou value of the frame to be detected is higher than the calibration threshold valueAt this time, soft-NMS reduces the confidence level of the frame based on the iou value, which can be represented by a fractional reset function
However, since the function is not a continuous function, errors in the fraction of the detected frame to be detected are easily caused, and thus a Gaussian function is introduced, and the final function is as follows
Wherein,is super parameter, D is a group of detection frames, iou (A,/B>) Indicating how well the ith detected frame coincides with a.
In the embodiment of the invention, the error deletion of the detection frame is reduced and the generalization performance of the small-target forest fire detection is improved by improving the traditional non-maximum suppression algorithm.
In one embodiment of the present invention, before the performing the cloud and mist removing treatment on the original image by using the segmentation model insensitive to the mist change to obtain the clear image, the method further includes:
acquiring a training set and a fog-pass filter;
alternately training a segmentation network and a fog pass filter through the training set to obtain fog invariant feature parameters;
and inputting the fog invariant feature parameters into an initial segmentation model to obtain a segmentation model insensitive to fog changes.
The training set comprises three training image fields, namely sunny (CW), synthetic Fog (SF) and Real Fog (RF). The fog filter can learn fog patterns according to the three training image domains, extract fog factors, extract fog related information from images and obtain fog invariant feature parameters. The fog-invariant feature parameters are input into the segmentation model as auxiliary information to provide additional information about fog conditions, thereby implementing the cloud and fog removal processing of the original image.
The haze factor is used to measure the concentration or density of haze, and can affect the visual quality and visibility of an image, and can be regarded as an additional image feature to provide condition information about haze so as to improve the segmentation performance.
In the embodiment of the invention, an alternate optimization method is adopted, and the coupling degree between the fog pass filter and the segmentation model can be gradually reduced by continuously iterating and optimizing the fog pass filter and the segmentation model, so as to achieve the globally optimal or approximately optimal solution. In addition, as only one model (a fog filter or a segmentation model) is optimized at a time, the alternative optimization can reduce the overall complexity and accelerate the convergence rate of the optimization process.
In one embodiment of the present invention, the alternately training the segmentation network and the fog-pass filter by the training set to obtain a segmentation model including fog-invariant feature parameters includes:
fixing parameters of the segmentation network, and training the fog-pass filter through the training set;
the step of fixing parameters of the segmentation network, and training the fog-pass filter through the training set specifically comprises the following steps:
carrying out cloud and mist removal treatment on the same training image under different training domains according to the segmentation network with fixed parameters to obtain feature images under different training domains;
calculating fog factors of the feature map under different training domains through the fog-pass filter;
calculating a fog-pass filtering loss function value corresponding to the fog-pass filter according to fog factors in different training domains;
and updating the fog filter according to the fog filter loss function value.
For a pair of training imagesAnd->The function of the fog-pass filter is to calculate a series of fog factors +.>Andto enable the segmented network image to distinguish between different fog conditions. Where a and b are the two images in the image pair and l is the index of the layer.
Wherein the input of the fog-pass filter is a gram matrix, marked asThe correlation between the c channels whose input features map is captured. ?>Element representation +.>And->Correlation between characteristic channels, calculated as +.>，/>Is the vector of the i-th channel of the input feature map. Since the gram matrix is symmetrical, only the vector form using the triangular part of the matrix need be chosen as input for the fog-pass filter. Therefore, the fog factor is calculated by convolving the vector upper triangle portion of the feature mapped glamer matrix with a fog pass filter.
Specifically, the calculation process is as follows:
for an image, computing a feature mapped gram matrix. Use of fog-pass filter F pair->Convolving the triangular part on the vector of (2) to obtain fog factor +.>=F(/>). Similarly, for an image->Computing a feature mapped gram matrix +.>And convolving it with a fog filter F, resulting in a fog factor = F (/ -)>)。
Wherein for a pair of images from different fog fieldsAnd->And a fog-pass filter +.>Calculating fog factor of the image, wherein fog-pass filtering loss function is
Wherein d (·) is a cosine distance, m is an edge, I (a, b) representsAnd->And returning to 1 when the two types belong to the same fog domain, otherwise, returning to 0.
In one embodiment of the present invention, the alternately training the segmentation network and the fog-pass filter by the training set to obtain a segmentation model including fog-invariant feature parameters includes:
fixing parameters of the fog-pass filter, and training the segmentation network through the training set;
the step of fixing parameters of the fog-pass filter, and training the segmentation network through the training set specifically comprises the following steps:
cloud and mist removing treatment is carried out on the same input training image according to the segmentation network, so that feature images under different training domains are obtained;
calculating fog factors of the feature map under different training domains through a fog-pass filter with fixed parameters;
calculating a fog type matching loss value according to the fog factor;
updating the fog invariant feature parameters of the segmentation network according to the fog matching loss value;
calculating the prediction score of the feature map according to the updated segmentation network;
calculating a segmentation loss value according to the prediction score and a ground truth label of the same input training image;
and updating the segmentation network again according to the segmentation loss value.
Wherein, fog formula matching loss function is:
wherein,is the dimension of fog factor, +.>Indicate->The spatial size of the feature map.
Wherein the segmentation loss function is as follows:
wherein n is the number of pixels,and->Representing the prediction score and ground truth label of class j at pixel i, respectively.
In order to make the different domains before calculating the segmentation loss function valueImages with the same semantic layout have similar prediction results, so that the domains can be aligned in learning to improve the performance of the model when performing transfer learning and domain self-adaptive tasks between different domains, and the prediction consistency loss function value can be calculated. The CW image and the SF image have the same semantic layout, and are provided withAnd->Class probability vector for segmentation model for pixel i in class c, loss is forced +.>And->The consistency between them, its corresponding predictive consistency loss function is as follows:
where KLdiv () is Kullback-leibler divergence.
In summary, combining the fog-like matching loss function, the segmentation loss function and the predictive consistency loss function, the segmentation network corresponds to a training strategy that a small batch is given, the same number of image pairs are respectively sampled from CW-SF, CW-RF and SF-RF, and the segmentation network is trained by a minimization method. The total loss function is as follows:
wherein,and->To balance hyper-parameters->And->Representing the real labels of CW and SF domains, respectively. For other pairs of input domain pairs, the loss consists of split and fog matches, as
Wherein,。
fig. 3 is a schematic structural diagram of a fire detection device according to an embodiment of the invention. The device comprises:
a data acquisition unit 301, configured to acquire an original image in a cloud environment;
an image processing unit 302, configured to perform cloud and mist removal processing on the original image through a segmentation model including a mist-invariant feature parameter, so as to obtain a clear image, where the mist-invariant feature is a feature insensitive to mist variation of the original image;
an image recognition unit 303, configured to recognize a potential fire target area in the clear image, so as to obtain the potential fire target area;
the data output unit 304 is configured to output a location and a category of the potential fire target area, where the category is used to indicate whether the potential fire target area has a fire.
Specifically, in terms of performing cloud and mist removal processing on the original image through the segmentation model insensitive to mist change to obtain a clear image, the image processing unit 302 is specifically configured to:
determining the similarity between the fog-invariant feature parameters in the fog-change insensitive segmentation model and the pixels in the original image;
determining a calculation weight of each pixel included in the original image based on the similarity;
a sharp image is determined based on the product between each pixel and the calculated weight.
Specifically, in the aspect of identifying the potential fire target area in the clear image, obtaining and outputting the position and the category of the potential fire target area, the image identifying unit 303 is specifically configured to:
extracting shallow detail features and deep semantic features in the clear image by using a convolutional neural network to obtain a feature map;
generating a plurality of candidate boxes on the feature map by a candidate box generator, each of the candidate boxes representing a potential fire target area;
determining whether a target exists in each candidate frame to confirm the category of the potential fire target area;
and outputting the position and the category of each potential fire target area.
Further, before said outputting the location and category of each of said potential fire target areas, said apparatus further comprises a location determining unit 305 for:
calculating the boundary regression offset of each candidate frame to obtain the accurate coordinates of each potential fire target area;
and taking the accurate coordinates as the position of the potential fire target area.
Further, before the cloud and mist removing processing is performed on the original image through the segmentation model insensitive to the mist change to obtain a clear image, the device further includes a model training unit 306, specifically configured to:
acquiring a training set and a fog-pass filter;
alternately training a segmentation network and a fog pass filter through the training set to obtain fog invariant feature parameters;
and inputting the fog invariant feature parameters into an initial segmentation model to obtain a segmentation model insensitive to fog changes.
Specifically, in the aspect that the training set is used for alternately training the segmentation network and the fog filter to obtain a segmentation model including the fog-invariant feature parameters, the model training unit 306 is specifically configured to:
fixing parameters of the segmentation network, and training the fog-pass filter through the training set;
the step of fixing parameters of the segmentation network, and training the fog-pass filter through the training set specifically comprises the following steps:
carrying out cloud and mist removal treatment on the same training image under different training domains according to the segmentation network with fixed parameters to obtain feature images under different training domains;
calculating fog factors of the feature map under different training domains through the fog-pass filter;
calculating a fog-pass filtering loss function value corresponding to the fog-pass filter according to fog factors in different training domains;
and updating the fog filter according to the fog filter loss function value.
Specifically, in the aspect that the training set is used for alternately training the segmentation network and the fog filter to obtain a segmentation model including the fog-invariant feature parameters, the model training unit 306 is specifically configured to:
fixing parameters of the fog-pass filter, and training the segmentation network through the training set;
the step of fixing parameters of the fog-pass filter, and training the segmentation network through the training set specifically comprises the following steps:
cloud and mist removing treatment is carried out on the same input training image according to the segmentation network, so that feature images under different training domains are obtained;
calculating fog factors of the feature map under different training domains through a fog-pass filter with fixed parameters;
calculating a fog type matching loss value according to the fog factor;
updating the fog invariant feature parameters of the segmentation network according to the fog matching loss value;
calculating the prediction score of the feature map according to the updated segmentation network;
calculating a segmentation loss value according to the prediction score and a ground truth label of the same input training image;
and updating the segmentation network again according to the segmentation loss value.
Referring to fig. 4, fig. 4 is a schematic structural diagram of another fire detection device according to an embodiment of the invention. The fire detection apparatus 400 may include: processor 401, network interface 404, and memory 405, and in addition, the fire detection apparatus 400 may further include: a user interface 403, and at least one communication bus 402. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 403 may include a Display screen (Display) and a Keyboard (Keyboard), and the optional user interface 403 may further include a standard wired interface and a wireless interface. The network interface 404 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 404 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 405 may also optionally be at least one storage device located remotely from the aforementioned processor 401. As shown in fig. 4, an operating system, a network communication module, a user interface module, and a device control application may be included in the memory 405, which is a type of computer-readable storage medium.
In the fire detection apparatus 400 shown in fig. 4, the network interface 404 may provide a network communication function; while user interface 403 is primarily an interface for providing input to a user; the processor 401 may be configured to invoke the device control application stored in the memory 405 to implement the fire detection method in the embodiment corresponding to fig. 3, which is not described herein. In addition, the description of the beneficial effects of the same method is omitted.
Furthermore, it should be noted here that: the embodiment of the present invention further provides a computer readable storage medium, in which a computer program executed by the aforementioned fire detection device is stored, and the computer program includes program instructions, when executed by the processor, can execute the description of the fire detection method in the embodiment corresponding to fig. 3, and therefore, a detailed description thereof will not be provided herein. In addition, the description of the beneficial effects of the same method is omitted. For technical details not disclosed in the embodiments of the computer-readable storage medium according to the present invention, please refer to the description of the method embodiments of the present invention.
Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), or the like.
The foregoing disclosure is illustrative of the present invention and is not to be construed as limiting the scope of the invention, which is defined by the appended claims.
Claims (10)
1. A fire detection method, the method comprising:
acquiring an original image in a cloud environment;
carrying out cloud and mist removal treatment on the original image through a segmentation model insensitive to mist change to obtain a clear image;
and identifying the potential fire target area in the clear image, and obtaining and outputting the position and the category of the potential fire target area, wherein the category is used for indicating whether the potential fire target area has a fire or not.
2. The method according to claim 1, wherein said performing a cloud and mist removal process on said original image by using a segmentation model insensitive to mist changes to obtain a clear image comprises:
determining the similarity between the fog-invariant feature parameters in the fog-change insensitive segmentation model and the pixels in the original image;
determining a calculation weight of each pixel included in the original image based on the similarity;
a sharp image is determined based on the product between each pixel and the calculated weight.
3. The method of claim 1, wherein identifying the potential fire target area in the clear image, obtaining and outputting the location and the category of the potential fire target area, comprises:
extracting shallow detail features and deep semantic features in the clear image by using a convolutional neural network to obtain a feature map;
generating a plurality of candidate boxes on the feature map by a candidate box generator, each of the candidate boxes representing a potential fire target area;
determining whether a target exists in each candidate frame to confirm the category of the potential fire target area;
and outputting the position and the category of each potential fire target area.
4. The method of claim 2, wherein prior to said outputting the location and category of each of said potential fire target areas, said method further comprises:
calculating the boundary regression offset of each candidate frame to obtain the accurate coordinates of each potential fire target area;
and taking the accurate coordinates as the position of the potential fire target area.
5. The method of claim 1, wherein the removing cloud and mist from the original image by the segmentation model insensitive to mist changes, before obtaining a clear image, further comprises:
acquiring a training set and a fog-pass filter;
alternately training a segmentation network and a fog pass filter through the training set to obtain fog invariant feature parameters;
and inputting the fog invariant feature parameters into an initial segmentation model to obtain a segmentation model insensitive to fog changes.
6. The method of claim 5, wherein the alternately training the segmentation network and the fog-pass filter by the training set results in a segmentation model comprising fog-invariant feature parameters, comprising:
fixing parameters of the segmentation network, and training the fog-pass filter through the training set;
the step of fixing parameters of the segmentation network, and training the fog-pass filter through the training set specifically comprises the following steps:
carrying out cloud and mist removal treatment on the same training image under different training domains according to the segmentation network with fixed parameters to obtain feature images under different training domains;
calculating fog factors of the feature map under different training domains through the fog-pass filter;
calculating a fog-pass filtering loss function value corresponding to the fog-pass filter according to fog factors in different training domains;
and updating the fog filter according to the fog filter loss function value.
7. The method of claim 5, wherein the alternately training the segmentation network and the fog-pass filter by the training set results in a segmentation model comprising fog-invariant feature parameters, comprising:
fixing parameters of the fog-pass filter, and training the segmentation network through the training set;
the step of fixing parameters of the fog-pass filter, and training the segmentation network through the training set specifically comprises the following steps:
cloud and mist removing treatment is carried out on the same input training image according to the segmentation network, so that feature images under different training domains are obtained;
calculating fog factors of the feature map under different training domains through a fog-pass filter with fixed parameters;
calculating a fog type matching loss value according to the fog factor;
updating the fog invariant feature parameters of the segmentation network according to the fog matching loss value;
calculating the prediction score of the feature map according to the updated segmentation network;
calculating a segmentation loss value according to the prediction score and a ground truth label of the same input training image;
and updating the segmentation network again according to the segmentation loss value.
8. A fire detection device, the device comprising:
the data acquisition unit is used for acquiring an original image in a cloud and fog environment;
the image processing unit is used for carrying out cloud and mist removal processing on the original image through a segmentation model comprising mist-invariant feature parameters to obtain a clear image, wherein the mist-invariant feature is a feature insensitive to mist change of the original image;
the image recognition unit is used for recognizing the potential fire target area in the clear image to obtain the potential fire target area;
and the data output unit is used for outputting the position and the category of the potential fire target area, wherein the category is used for indicating whether the potential fire target area has a fire or not.
9. A fire detection apparatus, comprising: a processor and a memory;
the processor is connected to a memory, wherein the memory is adapted to store a computer program, the processor being adapted to invoke the computer program to perform the method according to any of claims 1-7.
10. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program comprising program instructions which, when executed by a processor, perform the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311441867.6A CN117576369A (en) | 2023-11-01 | 2023-11-01 | Fire detection method, fire detection device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311441867.6A CN117576369A (en) | 2023-11-01 | 2023-11-01 | Fire detection method, fire detection device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117576369A true CN117576369A (en) | 2024-02-20 |
Family
ID=89889019
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311441867.6A Pending CN117576369A (en) | 2023-11-01 | 2023-11-01 | Fire detection method, fire detection device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117576369A (en) |
-
2023
- 2023-11-01 CN CN202311441867.6A patent/CN117576369A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109584248B (en) | Infrared target instance segmentation method based on feature fusion and dense connection network | |
| CN111178183B (en) | Face detection method and related device | |
| CN109255317B (en) | Aerial image difference detection method based on double networks | |
| CN108062525B (en) | Deep learning hand detection method based on hand region prediction | |
| WO2023015743A1 (en) | Lesion detection model training method, and method for recognizing lesion in image | |
| CN113936256A (en) | Image target detection method, device, equipment and storage medium | |
| CN110706266A (en) | Aerial target tracking method based on YOLOv3 | |
| CN111160407A (en) | Deep learning target detection method and system | |
| CN115797736B (en) | Training method, device, equipment and medium for target detection model and target detection method, device, equipment and medium | |
| Li et al. | Unsupervised domain adaptation with self-attention for post-disaster building damage detection | |
| CN113537180B (en) | Tree obstacle identification method and device, computer equipment and storage medium | |
| EP3553700A2 (en) | Remote determination of containers in geographical region | |
| Wang et al. | A feature-supervised generative adversarial network for environmental monitoring during hazy days | |
| CN114332977A (en) | Key point detection method and device, electronic equipment and storage medium | |
| CN111368634B (en) | Human head detection method, system and storage medium based on neural network | |
| Sun et al. | IRDCLNet: Instance segmentation of ship images based on interference reduction and dynamic contour learning in foggy scenes | |
| Gu et al. | Thermal image colorization using Markov decision processes | |
| Khoshboresh-Masouleh et al. | Robust building footprint extraction from big multi-sensor data using deep competition network | |
| CN116258931B (en) | Visual finger representation understanding method and system based on ViT and sliding window attention fusion | |
| CN115984712A (en) | Multi-scale feature-based remote sensing image small target detection method and system | |
| Fareed et al. | Saliency detection by exploiting multi-features of color contrast and color distribution | |
| CN114998387A (en) | Object distance monitoring method and device, electronic equipment and storage medium | |
| CN117576369A (en) | Fire detection method, fire detection device and storage medium | |
| CN114118127A (en) | Visual scene mark detection and identification method and device | |
| CN115719368B (en) | Multi-target ship tracking method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |