CN116310757A - Multitasking real-time smoke detection method - Google Patents

Abstract

The invention discloses a multitasking real-time smoke detection method, belonging to the field of computer vision and deep learning. The method comprises the following steps: (1) inputting video; (2) image preprocessing; (3) Inputting a smoke picture into a multi-task learning network, comprising two sub-networks: a real-time smoke segmentation network and a lightweight smoke detection network. The real-time smoke segmentation network adopts a BiSeNet-based dual-path network, and a characteristic pyramid module and a lightweight channel attention module are introduced into the dual-path network; the lightweight smoke detection network adopts an FCOS target detection network based on an anchor-free strategy, wherein a backbone network is replaced by a lighter MobileNet V2. The two paths of networks respectively output a smoke segmentation result and a smoke detection result. (4) And (3) combining the smoke segmentation result and the smoke detection result obtained in the previous step, judging whether smoke exists, and further outputting alarm information and position information. The invention balances the high precision and real-time performance of smoke detection performance, and has high practical value.

Description

Multitasking real-time smoke detection method

Technical Field

The invention relates to the field of computer vision and deep learning, in particular to a multitasking real-time smoke detection method. Essentially the problem of object detection and image segmentation.

Background

Fire is a very common disaster, and causes great threat to life safety and property of people. In the early stage of fire, smoke is often generated, so that how to accurately identify the smoke is of great importance for preventing the occurrence of fire. In the current image smoke detection task, how to realize low-delay and high-precision smoke detection and alarm on a monitoring platform is a challenging topic. In addition, the smoke always presents a semitransparent state, so that the background is mixed with the smoke in a highly complex way, the sparse or fine smoke is not obvious in visual sense, the boundary of the smoke is often blurred, and the characteristics of the smoke themselves cause great difficulty for a smoke detection task.

In recent years, with the vigorous development of deep learning technology, there are many methods for image segmentation and object detection, which play an important role in many fields. However, most of these methods cannot meet the requirement of real-time performance, and the recognition effect of smoke in a specific scene cannot meet the social production requirement, so how to improve the accuracy of the smoke detection algorithm and meet the requirement of real-time performance is a problem to be solved.

Disclosure of Invention

The invention aims to solve the problems that the smoke has blurred boundaries in the image and the like, and increases the difficulty of extracting the smoke image from the background. In order to effectively solve the problem of unclear smoke boundary, image segmentation and image detection are combined, detail characteristics of smoke parts in the image are increased, and detection accuracy is improved.

The invention adopts the technical scheme that:

a method of multitasking real-time smoke detection comprising the steps of:

step 1, reading in pictures from a camera subcode stream or video;

step 2, performing image scaling and image denoising preprocessing operation on the read-in picture;

step 3, processing and analyzing the preprocessed picture by adopting a multi-task learning network, wherein the multi-task learning network comprises two branches: a lightweight smoke detection network branch and a real-time smoke segmentation network branch;

in a lightweight smoke detection network branch, a picture passes through a structure of three convolution layers, the size of each convolution core is 3 multiplied by 3, and then a Batchnormalization normalization layer and a ReLU activation function are connected to obtain a feature map F of one eighth of the original picture size ^s The method comprises the steps of carrying out a first treatment on the surface of the Then map F ^s Dividing the feature map into different subareas according to four different proportions through a pyramid pooling module, performing pooling in each subarea to obtain features with different scales, and finally carrying out up-sampling operation on the output pooled features and F ^s Superposing to obtain a new characteristic diagram F ^c ；

In the real-time smoke segmentation network branch, the pictures are subjected to continuous three downsampling operations to generate a characteristic diagram of one sixteenth of the original picture size, which is marked as F ¹ F is to F ¹ Continuing to perform downsampling operation once, and recording the output as F ² The size is thirty-half of the original film size, F ² Continuing to obtain F through global pooling once ³ The method comprises the steps of carrying out a first treatment on the surface of the Then F is carried out ¹ And F ² The method comprises the steps of changing the high-efficiency channel attention module into a one-dimensional channel feature map, carrying out convolution operation on the one-dimensional channel feature map and a convolution layer with a convolution kernel size of K, wherein K represents the coverage range of local cross-channel interaction, respectively obtaining new one-dimensional attention force map, and respectively corresponding the one-dimensional attention force map to F ¹ And F ² Multiplying to obtain a new attention profile F ¹ⁿ And F ²ⁿ F is to F ³ Through up-sampling operation and F ²ⁿ Feature superposition is carried out, and up-sampling operation and F are carried out after the feature superposition ¹ⁿ Stacking, and then passing throughThe up-sampling operation results in feature F ^1n+2n+3 F is to F ^1n+2n+3 Output F with lightweight smoke detection network ^c Superposition to obtain feature F ^con Superimposed feature F ^con Generating a channel attention map through a channel attention mechanism of the feature fusion module, and then combining with F ^con Multiplying to obtain F ^con _{_} ^a Then with F ^con Superposing, generating a new feature map and carrying out loss function calculation;

and step 4, combining the smoke segmentation result and the smoke detection result obtained in the step 3, and judging whether smoke is generated or not.

Compared with the prior art, the invention has the following advantages:

1 has higher detection accuracy. The edge detail characteristics of the smoke area are increased, and the detection accuracy is improved;

2 has higher detection efficiency. The parameter number of the algorithm model is reduced, the calculation force requirement of the model is reduced, and the detection efficiency is improved.

Drawings

Fig. 1 is a flow chart of the method of the present invention.

Detailed Description

In order to make the technical problems, technical solutions and advantages to be solved more apparent, the following detailed description will be given with reference to the accompanying drawings and specific embodiments.

The embodiment provides a method for detecting multi-task real-time smoke, which specifically comprises the following steps as shown in fig. 1:

(1) And reading in pictures from the camera code stream or the video to complete data acquisition.

(2) And image scaling and image denoising operations are adopted for the read-in image, so that the final detection precision is improved.

(3) The method comprises the steps of processing and analyzing pictures by adopting a multi-task learning network, and firstly converting the input pictures into 4-dimensional tensors [ N, C, H and W ], wherein N represents the input quantity, C represents the channel quantity of the feature map, and H and W represent the height and width of the feature map. The pictures are respectively and simultaneously input into two branches of a real-time smoke segmentation network and a lightweight smoke detection network in the form of tensors.

For a real-time smoke segmentation network, a BiSeNet-based dual-path image segmentation network is adopted, the network comprises a space path and a context path, a pyramid pooling module is introduced into the space path, the aim of improving the multi-scale feature extraction capacity of the network is achieved, a lightweight channel attention module is introduced into the context path, the aim of reducing the complexity of an algorithm to meet the requirement of instantaneity is achieved, and finally feature fusion is carried out on feature graphs output by the two paths through a feature fusion module, so that a smoke segmentation result is obtained. For the lightweight smoke detection network, an FCOS (fiber control system) model network based on an anchor-free strategy is adopted, the anchor-free strategy can effectively reduce network parameters, meanwhile, a backbone network of the network is set to be MobileNet V2, a full connection layer is removed, the improvement measure greatly improves the speed of smoke detection on the basis of meeting detection precision, and meanwhile, a characteristic pyramid module is introduced, so that the multi-scale performance of the detection network can be effectively enhanced, and the detection precision is further improved. The specific structure is as follows:

in a lightweight smoke detection network branch, a picture passes through a structure of three convolution layers, the size of each convolution core is 3 multiplied by 3, and then a Batchnormalization normalization layer and a ReLU activation function are connected to obtain a feature map F of one eighth of the original picture size ^s The method comprises the steps of carrying out a first treatment on the surface of the Then map F ^s Dividing the feature map into different subareas according to four different proportions through a pyramid pooling module, performing pooling in each subarea to obtain features with different scales, and finally carrying out up-sampling operation on the output pooled features and F ^s Superposing to obtain a new characteristic diagram F ^c ；

In the real-time smoke segmentation network branch, the pictures are subjected to continuous three downsampling operations to generate a characteristic diagram of one sixteenth of the original picture size, which is marked as F ¹ F is to F ¹ Continuing to perform downsampling operation once, and recording the output as F ² The size is thirty-half of the original film size, F ² Continuing to obtain F through global pooling once ³ The method comprises the steps of carrying out a first treatment on the surface of the Then F is carried out ¹ And F ² The high-efficiency channel attention modules are respectively changed into one-dimensional channel feature graphs to enable one dimension to be communicatedThe trace feature map and a convolution layer with the convolution kernel size of K are subjected to convolution operation, K represents the coverage range of local cross-channel interaction, new one-dimensional attention force diagrams are obtained respectively, and the one-dimensional attention force diagrams are corresponding to F respectively ¹ And F ² Multiplying to obtain a new attention profile F ¹ⁿ And F ²ⁿ F is to F ³ Through up-sampling operation and F ²ⁿ Feature superposition is carried out, and up-sampling operation and F are carried out after the feature superposition ¹ⁿ Overlapping, and then performing up-sampling operation to obtain a feature F ^1n+2n+3 F is to F ^1n+2n+3 Output F with lightweight smoke detection network ^c Superposition to obtain feature F ^con Superimposed feature F ^con Generating a channel attention map through a channel attention mechanism of the feature fusion module, and then combining with F ^con Multiplying to obtain F ^con _{_} ^a Then with F ^con Superposing, generating a new feature map and carrying out loss function calculation;

in this example, the training pictures of both networks were 9300 pieces in total, 7000 pieces were used for training, and 2300 pieces were used for testing. All acquired pictures are 24-bit RGB images, ranging in resolution from 256 x 256 to 1920 x 1080, and are arranged in a uniform naming manner.

The training of the network adopts cross entropy and Dice fusion loss function to calculate errors, and adopts a random gradient descent method to update weights.

(4) And (3) combining the smoke segmentation result obtained in the step (3) with a smoke detection result, judging whether smoke is generated, and outputting alarm information and position information for analysis by a subsequent expert system if the smoke is generated.

Claims (1)

1. A method for multi-tasking real-time smoke detection comprising the steps of:

step 1, reading in pictures from a camera subcode stream or video;

step 2, performing image scaling and image denoising preprocessing operation on the read-in picture;

step 3, processing and analyzing the preprocessed picture by adopting a multi-task learning network, wherein the multi-task learning network comprises two branches: a lightweight smoke detection network branch and a real-time smoke segmentation network branch;

in the lightweight smoke detection network branch, a picture passes through a structure of three convolution layers, the convolution kernel of each layer is 3 multiplied by 3, and then a Batch Normalization normalization layer and a ReLU activation function are connected to obtain a feature map F of one eighth of the original picture size ^s The method comprises the steps of carrying out a first treatment on the surface of the Then map F ^s Dividing the feature map into different subareas according to four different proportions through a pyramid pooling module, performing pooling in each subarea to obtain features with different scales, and finally carrying out up-sampling operation on the output pooled features and F ^s Superposing to obtain a new characteristic diagram F ^c ；

In the real-time smoke segmentation network branch, the pictures are subjected to continuous three downsampling operations to generate a characteristic diagram of one sixteenth of the original picture size, which is marked as F ¹ F is to F ¹ Continuing to perform downsampling operation once, and recording the output as F ² The size is thirty-half of the original film size, F ² Continuing to obtain F through global pooling once ³ The method comprises the steps of carrying out a first treatment on the surface of the Then F is carried out ¹ And F ² The method comprises the steps of changing the high-efficiency channel attention module into a one-dimensional channel feature map, carrying out convolution operation on the one-dimensional channel feature map and a convolution layer with a convolution kernel size of K, wherein K represents the coverage range of local cross-channel interaction, respectively obtaining new one-dimensional attention force map, and respectively corresponding the one-dimensional attention force map to F ¹ And F ² Multiplying to obtain a new attention profile F ¹ⁿ And F ²ⁿ F is to F ³ Through up-sampling operation and F ²ⁿ Feature superposition is carried out, and up-sampling operation and F are carried out after the feature superposition ¹ⁿ Overlapping, and then performing up-sampling operation to obtain a feature F ^1n+2n+3 F is to F ^{1n +2n+3} Output F with lightweight smoke detection network ^c Superposition to obtain feature F ^con Superimposed feature F ^con Generating a channel attention map through a channel attention mechanism of the feature fusion module, and then combining with F ^con Multiplying to obtain F ^con _{_} ^a Then with F ^con Superposition to generate new featuresCarrying out loss function calculation on the graph;

and step 4, combining the smoke segmentation result and the smoke detection result obtained in the step 3, and judging whether smoke is generated or not.

Images