CN114964628A

CN114964628A - Shuffle self-attention light-weight infrared detection method and system for ammonia gas leakage

Info

Publication number: CN114964628A
Application number: CN202210516324.5A
Authority: CN
Inventors: 张印辉; 庄宏; 何自芬
Original assignee: Kunming University of Science and Technology
Current assignee: Kunming University of Science and Technology
Priority date: 2022-05-12
Filing date: 2022-05-12
Publication date: 2022-08-30

Abstract

The invention provides an ammonia gas leakage shuffle self-attention light-weight infrared detection method and system, and relates to the technical field of image processing and gas leakage detection; carrying out normalization processing on an infrared image detection data set, dividing the infrared image detection data set into a training set and a test set, and carrying out ammonia gas leakage infrared data annotation on the infrared image detection data set; constructing a shuffled self-attention network structure; training the shuffled self-attention network structure by using a training set with ammonia gas leakage infrared data labels to obtain a shuffled self-attention network model; the test set is detected through the shuffle self-attention network model, the final shuffle self-attention network model and the test program are called, the ammonia gas leakage infrared image is input to determine the detection result, the real-time accurate monitoring on ammonia gas leakage can be realized at a long distance, the safety of workers is guaranteed, the response is timely made when the ammonia gas leakage occurs, and the technical effect of reducing the economic loss is achieved.

Description

Shuffle self-attention light-weight infrared detection method and system for ammonia gas leakage

Technical Field

The invention relates to the technical field of image processing and gas leakage detection, in particular to a shuffle self-attention light-weight infrared detection method and a shuffle self-attention light-weight infrared detection system for ammonia gas leakage.

Background

The ammonia gas is used as an important basic industrial raw material and widely applied to industrial production such as cold-chain logistics, aerospace and aviation, and the like, but ammonia gas leakage is a main potential safety hazard existing in the production process, personnel poisoning and explosive danger can be caused if the ammonia gas is not treated in time when the ammonia gas leaks, and a real-time ammonia gas leakage detection technology is urgently needed to be developed due to huge economic loss caused by multiple serious ammonia gas leakage accidents.

A conventional ammonia gas leakage detection method adopts a gas sensor, belongs to point location measurement, and has the problems that many areas to be detected cannot reach due to the contact principle, the detection range is small, the sensitivity is poor, the real-time performance is not high, a leakage source cannot be positioned, and the like, so that the safety of workers is greatly threatened. The infrared imaging detection technology can present a harmful gas leakage image on a display, so that the large-range, real-time and visual detection of the harmful gas leakage is possible. The design of a harmful gas leakage non-contact detection model based on an infrared imaging detection technology is the key of the problem.

The ammonia leakage non-contact detection can realize better guarantee of the safety of workers by imaging and positioning the leaked gas at a longer distance, and the model is more complicated and complex, has high implementation difficulty and is difficult to realize ammonia leakage real-time detection although some effects exist at present. Aiming at the characteristics that an infrared image obtained by a thermal infrared imager has high noise and poor detection effect caused by contrast, and ammonia gas leakage has the occurrence time and is unfixed in place, higher requirements are put forward for ensuring safety to real-time performance, the existing detection cannot meet the requirements in the current stage, and how to accurately detect the ammonia gas leakage in real time becomes the technical problem which needs to be solved urgently at present.

Disclosure of Invention

The application aims to provide an ammonia gas leakage shuffling self-attention light-weight infrared detection method and system, which are used for solving the technical problems that a conventional ammonia gas leakage detection device in the prior art is small in detection range, poor in sensitivity, low in real-time performance and incapable of positioning a leakage source, achieving the purpose of realizing real-time accurate monitoring of ammonia gas leakage at a long distance, guaranteeing the safety of workers, responding in time when ammonia gas leaks and reducing the technical effect of economic loss.

In view of the above, the present application provides a shuffle self-attention light-weight infrared detection method and system for ammonia gas leakage.

In a first aspect of the application, there is provided a shuffled self-attention light-weight infrared detection method for ammonia gas leakage, the method comprising: acquiring an infrared image detection data set; normalizing the infrared image detection data set, dividing the infrared image detection data set into a training set and a test set, and performing ammonia gas leakage infrared data annotation on the infrared image detection data set; constructing a shuffled self-attention network structure; training the shuffled self-attention network structure by using the training set with the ammonia leakage infrared data labels to obtain a shuffled self-attention network model; and detecting the test set through the shuffle self-attention network model, calling the final shuffle self-attention network model and a test program, and inputting an ammonia gas leakage infrared image to determine a detection result.

In a second aspect of the application, there is provided an ammonia gas leak shuffled self-attention light-weight infrared detection system, comprising: a first obtaining unit for obtaining an infrared image detection dataset; the first processing unit is used for dividing the infrared image detection data set into a training set and a test set after normalization processing is carried out on the infrared image detection data set, and carrying out ammonia gas leakage infrared data annotation on the infrared image detection data set; a first construction unit for constructing a shuffled self-attention network structure; a second processing unit for training the shuffled self-attention network structure using the training set with the ammonia leakage infrared data annotation, to obtain a shuffled self-attention network model; and the third processing unit is used for detecting the test set through the shuffled self-attention network model, calling a final shuffled self-attention network model and a test program, and inputting an ammonia gas leakage infrared image to determine a detection result.

In a third aspect of the present application, there is provided an ammonia gas leakage shuffled self-attention lightweight infrared detection system, comprising: a processor coupled to a memory for storing a program that, when executed by the processor, causes a system to perform the steps of the method according to the first aspect.

One or more technical solutions provided in the present application have at least the following technical effects or advantages:

according to the shuffle self-attention light-weight infrared detection method for ammonia gas leakage, an infrared image detection data set is obtained; dividing the infrared image detection data set into a training set and a test set after normalization processing, and carrying out ammonia gas leakage infrared data annotation on the infrared image detection data set; constructing a shuffled self-attention network structure; training the shuffled self-attention network structure by using the training set with the ammonia leakage infrared data labels to obtain a shuffled self-attention network model; and detecting the test set through the shuffle self-attention network model, calling the final shuffle self-attention network model and a test program, and inputting an ammonia gas leakage infrared image to determine a detection result. The technical problems that a conventional ammonia gas leakage detection device in the prior art is small in detection range, poor in sensitivity, low in real-time performance and incapable of positioning a leakage source are solved, real-time accurate monitoring of ammonia gas leakage can be achieved at a long distance, safety of workers is guaranteed, response is timely made when ammonia gas leaks, and economic loss is reduced.

The foregoing description is only an overview of the technical solutions of the present application, and the present application can be implemented according to the content of the description in order to make the technical means of the present application more clearly understood, and the following detailed description of the present application is given in order to make the above and other objects, features, and advantages of the present application more clearly understandable.

Drawings

FIG. 1 is a schematic flow diagram of an ammonia gas leakage shuffled self-attention light-weight infrared detection method provided by the application;

FIG. 2 is a schematic diagram of a shuffled self-attention lightweight infrared detection method for ammonia gas leakage provided by the present application, showing a network structure of a shuffled self-attention network model (SSANet);

fig. 3 is a schematic structural diagram of a lightweight feature extraction network SK5Block module with a step size of 1(Stride 1) in the ammonia gas leakage shuffled self-attention lightweight infrared detection method provided by the present application;

fig. 4 is a schematic structural diagram of a lightweight feature extraction network SK5Block module with a step size of 1(Stride 2) in the ammonia gas leakage shuffled self-attention lightweight infrared detection method provided by the present application;

fig. 5 is a schematic structural diagram of a Transformer coding layer in the ammonia gas leakage shuffled self-attention light-weight infrared detection method provided by the present application;

fig. 6 is a schematic structural diagram of a Transformer module in the shuffled self-attention light-weight infrared detection method for ammonia gas leakage provided by the present application;

FIG. 7 is a schematic structural diagram of an ammonia gas leakage shuffled self-attention light-weight infrared detection system provided in the present application;

fig. 8 is a schematic structural diagram of an exemplary electronic device of the present application.

Description of reference numerals: a first obtaining unit 11, a first processing unit 12, a first constructing unit 13, a second processing unit 14, a third processing unit 15, an electronic device 300, a memory 301, a processor 302, a communication interface 303, and a bus architecture 304.

Detailed Description

The application provides the ammonia gas leakage shuffle self-attention light-weight infrared detection method and system, which are used for solving the technical problems that the conventional ammonia gas leakage detection device in the prior art is small in detection range, poor in sensitivity, low in real-time performance and incapable of positioning a leakage source, achieving the purpose of realizing real-time accurate monitoring of ammonia gas leakage at a far distance, guaranteeing the safety of workers, responding in time when ammonia gas leaks, and reducing the technical effect of economic loss.

In view of the above technical problems, the technical solution provided by the present application has the following general idea:

the method provided by the embodiment of the application comprises the steps of obtaining an infrared image detection data set; normalizing the infrared image detection data set, dividing the infrared image detection data set into a training set and a test set, and performing ammonia gas leakage infrared data annotation on the infrared image detection data set; constructing a shuffled self-attention network structure; training the shuffled self-attention network structure by using the training set with the ammonia leakage infrared data labels to obtain a shuffled self-attention network model; and detecting the test set through the shuffle self-attention network model, calling the final shuffle self-attention network model and a test program, and inputting an ammonia gas leakage infrared image to determine a detection result.

Having described the basic principles of the present application, the technical solutions in the present application will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments of the present application, and the present application is not limited to the exemplary embodiments described herein. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application. It should be further noted that, for the convenience of description, only some but not all of the elements relevant to the present application are shown in the drawings.

Example one

As shown in fig. 1, the present application provides a shuffled self-attention, lightweight infrared detection method for ammonia gas leakage, comprising:

s100: acquiring an infrared image detection data set;

specifically, an image acquisition system is built, and an infrared thermal imager is adopted to acquire an ammonia gas leakage video and then frame taking processing is carried out to obtain an infrared image; aiming at the problems that an ammonia gas leakage infrared image acquired by a thermal infrared imager is high in noise content and low in contrast, the network training time is prolonged due to the existence of singular sample data and the network can not be converged in a possible manner, so that the data processing of the model is facilitated and accelerated, noise points randomly distributed in a background area are removed by a non-local mean denoising method, and meanwhile the local characteristics of edge pixels of an ammonia gas cloud cluster area are well maintained. And then performing contrast enhancement pretreatment on the ammonia gas leakage infrared image by adopting a contrast-limited self-adaptive histogram equalization algorithm to establish an ammonia gas leakage infrared image detection data set.

S200: normalizing the infrared image detection data set, dividing the infrared image detection data set into a training set and a test set, and performing ammonia gas leakage infrared data annotation on the infrared image detection data set;

specifically, before data processing, normalization processing is performed on the data, adverse effects caused by singular sample data are eliminated, the speed of solving an optimal solution through gradient descent is increased, and the accuracy of model processing data is improved.

And dividing the image size in the infrared image detection data set into a training set and a test set after normalization processing, and performing ammonia gas leakage infrared data labeling on the obtained infrared image detection data set by using an open source tool.

S300: constructing a shuffle self-attention network structure;

s400: training the shuffled self-attention network structure by using the training set with the ammonia leakage infrared data labels to obtain a shuffled self-attention network model;

specifically, a shuffled self-attention network model (SSANet) is constructed in the embodiment of the application to realize infrared real-time non-contact detection of ammonia gas leakage; in order to improve the detection precision of the ammonia gas leakage infrared detection model while meeting the detection speed, a 3 x 3 depth separable convolution kernel is designed into a 5 x 5 SK5Block module to reconstruct a feature extraction network, the model reasoning calculation amount is reduced according to the leakage region expansion feeling of ammonia gas leakage at different moments, the model weight is compressed, and the model real-time detection is achieved; the SSANet model adopts a Transformer module as a bottleneck layer of a model feature pyramid from bottom to top to realize multi-head attention fusion from bottom to top in a leakage area, and multi-scale feature information is obtained, so that the fusion of global features and local features is realized, and the precision of the model is improved.

And training and learning the shuffling self-attention network structure by using a training set formed by labeling the ammonia leakage infrared data to obtain a shuffling self-attention network model.

S500: and detecting the test set through the shuffle self-attention network model, calling the final shuffle self-attention network model and a test program, and inputting an ammonia gas leakage infrared image to determine a detection result.

Specifically, the shuffled self-attention network model obtained in the above steps is tested through a test set formed by labeling the ammonia leakage infrared data, and when the output result of the model reaches the expected output result, the final shuffled self-attention network model and a test program are called, and the ammonia leakage infrared light-weight real-time detection is realized by inputting an ammonia leakage infrared image into the final shuffled self-attention network model and outputting a detection result, so that an effective real-time detection method is provided for developing an ammonia leakage non-contact detection device to ensure the safe production and the stable operation of ammonia-related enterprises.

The method provided by the application comprises the steps of obtaining an infrared image detection data set; normalizing the infrared image detection data set, dividing the infrared image detection data set into a training set and a test set, and performing ammonia gas leakage infrared data annotation on the infrared image detection data set; constructing a shuffled self-attention network structure; training the shuffled self-attention network structure by using the training set with the ammonia leakage infrared data labels to obtain a shuffled self-attention network model; and detecting the test set through the shuffle self-attention network model, calling a final shuffle self-attention network model and a test program, and inputting an ammonia gas leakage infrared image to determine a detection result. The technical problems that a conventional ammonia gas leakage detection device in the prior art is small in detection range, poor in sensitivity, low in real-time performance and incapable of positioning a leakage source are solved, real-time accurate monitoring of ammonia gas leakage can be achieved at a long distance, safety of workers is guaranteed, response is timely made when ammonia gas leaks, and economic loss is reduced.

Step S100 in the method provided in the embodiment of the present application includes:

s110: acquiring an ammonia gas leakage video through a thermal infrared imager, and performing frame taking processing on the ammonia gas leakage video to obtain an infrared image set;

s120: carrying out non-local mean denoising on the infrared image set to obtain a denoised infrared image set;

s130: and performing adaptive histogram equalization processing on the basis of the de-noised infrared image set to obtain the infrared image detection data set.

In particular, the infrared imaging detection technology can present a harmful gas leakage image on a display, so that the large-range, real-time and visual detection of the harmful gas leakage is possible.

Acquiring an ammonia gas leakage video by using a thermal infrared imager, and then performing frame taking processing to obtain an infrared image; aiming at the problems of high noise content and low contrast of an ammonia gas leakage infrared image collected by a thermal infrared imager, noise points randomly distributed in a background area are removed by a non-local mean denoising method, and meanwhile, the local characteristics of edge pixels of an ammonia gas cloud area are well maintained. And then, performing contrast enhancement pretreatment on the ammonia gas leakage infrared image by adopting a contrast-limiting self-adaptive histogram equalization algorithm to establish an ammonia gas leakage infrared detection data set.

Illustratively, a long-wave thermal infrared imager with the wavelength of 8-14 mu m is adopted to collect an ammonia gas leakage video, and then a frame is taken to obtain an infrared image; performing noise removal and Contrast enhancement pretreatment on the ammonia leakage infrared image by adopting a Non-Local mean denoising (NL-Means) and Contrast Limiting Adaptive Histogram Equalization (CLAHE) algorithm;

the non-local mean de-noising algorithm fully utilizes redundant information and information with similar structures in an image, and eliminates noise by searching all similar blocks in the whole range of the image and carrying out weighted average on the similar structures, and simultaneously can keep the detail characteristics of the image to the maximum extent. And (3) carrying out image denoising treatment on the ammonia gas leakage infrared image by adopting a non-local mean denoising algorithm so as to reduce the infrared image noise. The infrared image is obtained after the processing of the non-local mean denoising algorithm, the image noise white point disappears in a large area, so that the image definition is high, the details are not lost, and the denoising effect is obvious; on the basis of carrying out non-local mean value noise reduction on the ammonia gas leakage infrared image, in order to further improve the image quality, a contrast-limiting self-adaptive histogram equalization algorithm is adopted to process the ammonia gas leakage infrared image, the CLAHE enables the processed area to be thinner from the original image, and noise can be suppressed while the contrast is enhanced. The CLAHE firstly divides the image into sub-blocks, calculates the histogram in the sub-blocks, and then reasonably distributes the part of each sub-block exceeding the clipping amplitude limit to other areas in the histogram by adopting a method of limiting the contrast, thereby achieving the effect of reallocating each sub-block of the histogram; on the basis of the infrared image subjected to non-local mean value noise reduction, the self-adaptive histogram equalization algorithm for limiting the contrast is used for processing the ammonia gas leakage infrared image, so that the gas cloud cluster of ammonia gas leakage is more obvious, the details are richer, and the model detection is facilitated.

Further, in order to analyze the effectiveness of the image preprocessing algorithm, quantitative evaluation indexes of three images, namely peak signal to noise ratio (PSNR), Average Gradient (AG) and Information Entropy (IE), are selected to carry out quantitative averaging evaluation on the infrared image enhanced by the method. The peak signal-to-noise ratio is a reference image quality evaluation index, the difference degree and the noise resistance between the denoised image and the original image are described, the larger the value is, the better the denoising effect is, and the whole visual effect of human eyes is better; the average gradient represents the contrast of a fine part of an image, and the image is clearer when the evaluation gradient is larger; the information entropy represents the information quantity of the image, and the greater the information entropy, the richer the detail information of the image. The ammonia gas leakage infrared original image and the image preprocessing image were evaluated by using the above three indexes, and the results are shown in table 1.

TABLE 1 quantitative evaluation index for image preprocessing

As can be seen from Table 1, the peak signal-to-noise ratio of the preprocessed image compared with the original image is up to 23.40dB, which shows that the anti-noise performance of the image is improved and the de-noising effect is better. The average gradient and the information entropy index of the preprocessed image are higher than those of the original image, so that the contrast of the preprocessed image is effectively improved, the image is clearer, and the detail texture is richer. The performance of the algorithm is shown in Table 2, and the model obtained by image training after image preprocessing is marked as Prep-YOLOv5 s.

TABLE 2 network Performance comparison before and after image preprocessing

As can be seen from Table 2, the model of Prep-YOLOv5s has an improved mAP by 1.00% compared with the model trained by using the infrared original image on the premise that the speeds are basically consistent. The image preprocessing method is shown to improve the detection precision of the ammonia gas leakage on the premise of ensuring the detection speed of the model to be unchanged. Experiments prove that the denoising effect is obvious without losing details through image preprocessing, and the gas cloud cluster is more obvious in contribution to ammonia gas leakage detection.

Step S200 in the method provided in the embodiment of the present application further includes:

s210: carrying out image size preprocessing on the infrared image detection data in the infrared image detection data set;

s220: determining a target detection area based on the infrared image detection data after size preprocessing, and setting a boundary frame according to the target detection area;

s230: generating a label file with a preset data set format according to the boundary frame of the target detection area;

s240: marking the preset data set format label file, wherein the marking information comprises the ammonia gas leakage position and the size of a leakage area of a boundary frame;

s250: and performing format conversion on the preset data set format label file, and determining the ammonia gas leakage infrared data label, wherein the ammonia gas leakage infrared data label is a first preset format label file and comprises a boundary frame central point coordinate.

Specifically, the acquired ammonia gas leakage infrared image is divided into a training set and a testing set after size normalization processing, an open source tool LabeImg is used for manually marking the acquired ammonia gas infrared image data, namely, all ammonia gas leakage areas needing to be detected on each image are marked with corresponding target boundary boxes, and an xml label file in a data set format conforming to VOC2007 is generated. And the marking information comprises the position and the size of the leaked ammonia gas in the coordinate area of the marking boundary frame, and the xml file is converted into a txt label file which comprises the coordinate of the center point of the boundary frame of the ammonia gas leakage area.

Illustratively, a 906 × 720 image output by an infrared camera is converted into a 640 × 640 size image suitable for network model input. Manually labeling the obtained ammonia infrared image data by using an open source tool LabeImg, namely labeling the corresponding target boundary boxes of all ammonia leakage areas needing to be detected on each image, and generating an xml label file in a data set format conforming to VOC 2007. And the marking information comprises the position and the size of the leaked ammonia gas in the coordinate area of the marking boundary frame, and the xml file is converted into a txt label file which comprises the coordinate of the center point of the boundary frame of the ammonia gas leakage area.

As shown in fig. 2, step S300 in the method provided in the embodiment of the present application further includes:

s310: constructing a detection network model SSANet;

s320: constructing a feature extraction network of the detection network model SSANet by using a lightweight SK5Block module, wherein the feature extraction network comprises a 5 x 5 depth separable convolution and channel shuffling structure;

s330: adding a Transformer module with a self-attention mechanism on the feature extraction network as a bottleneck layer of a model feature pyramid from bottom to top, wherein the Transformer module comprises an image slicing part, a data embedding part and an encoding layer part.

Specifically, ammonia gas is a colorless, flammable and explosive gas with toxicity, and sensitivity and real-time performance of detection of the gas by the characteristics of occurrence time of leakage and unfixed place of the gas put forward a very high requirement on designing a shuffled self-attention network model (SSANet) to realize infrared real-time non-contact detection of ammonia gas leakage;

in order to enable the ammonia gas detection model to have less reasoning calculation amount and accelerate the reasoning speed of the model, the real-time detection of ammonia gas leakage is realized. The feature extraction network with the reconstructed model reduces the calculation amount and the compression model weight during model reasoning. Through comparing indexes of different feature extraction networks, the SK5-YOLOv5s has obvious advantages in performance of model weight, speed, precision and the like, and the feature extraction network which preferably adopts the light weight SK5Block module to construct the model in the embodiment of the application achieves balance between speed and precision. Specifically, in the embodiment, different feature extraction network evaluation indexes are compared, and a comparison experiment is performed with the method of the present invention after replacing the feature extraction network of the YOLOv5s model with the lightweight architecture ghestnet, MobileNetv3, and ShuffleNetv2, respectively, during the experiment, the feature extraction network of the SK5Block module construction model follows the same training and testing scheme, and the experiment result is shown in table 3.

TABLE 3 comparison of network evaluation indexes for different feature extractions

As can be seen from Table 3, the weight of the SK5-YOLOv5s model with improved light weight is compressed to 3.4M, the detection speed of a single image reaches 2.7ms, the reference quantity is slightly increased by 4% compared with that of the ShuffleNetv2-YOLOv5s model, the precision is improved to 94.4%, the validity of a feature extraction network adopting the SK5Block module to construct the model is verified, meanwhile, the expansion of a 3 x 3 depth separable convolution kernel to 5 x 5 is verified, the sensitivity field is enlarged, the validity of the model detection precision is improved while too much calculation amount is not increased, and the balance between the light weight of the model and the model detection precision is met. When the feature extraction network using the MobileNetv3 and the GhostNet as the models is used for lightening the features, the detection accuracy of the models is not as good as that of the SK5-YOLOv5s model although the detection speed of the models is improved and the weight of the models is reduced, and the number of parameters and the weight of the models are larger and the speed is lower than that of the SK5-YOLOv5s model. By comprehensively comparing the parameter number, the model weight, the speed and the precision of the model, the SK5-YOLOv5s has obvious advantages in all aspects of performance. Therefore, an SK5-YOLOv5s structure is selected as a feature extraction network of the ammonia gas leakage lightweight infrared detection model.

The SK5Block module is mainly composed of a 1 × 1 normal Convolution (Conv1 × 1), a 3 × 3 depth separable Convolution kernel (DWConv) and a Channel Shuffle (Channel Shuffle) structure, which are designed to expand to 5 × 5.

The method aims at the defect that when ammonia leaks, the model has the defects that local features are concerned and the detection precision is low when the global features are ignored due to the fact that the phenomenon that the contrast ratio of irregular motion of ammonia cloud clusters and diffusion regions in infrared images is low is not obvious. A Transformer module with a self-attention mechanism is introduced, infrared images can be converted into a sequence, and the sequence can be input into a model for processing. On the basis of a feature extraction network reconstructed by an SK5Block module, a Transformer is used as a bottleneck layer of a model feature pyramid from bottom to top, and multi-scale feature information is obtained, so that fusion of global features and local features is achieved.

The Transformer module can capture global information and rich context information and execute global reasoning on the image and the specific predicted target, and the application of the Transformer module to the low-resolution feature map can reduce expensive calculation and storage cost and improve the detection precision of the leakage feature importance improvement model. Specifically, in this embodiment, performance comparison is performed on the characteristic pyramid of the pair of model hack part by adding different bottleneck layer structures from bottom to top to the bottleneck layer, and the added different bottleneck layer structures are cspbotttleneck, ghost bottleneck, CbamBottleNeck, and transformer bottleneck, respectively. The model obtained by training with the addition of a Transformamer BottleNeck structure was designated as SSANet, and the experimental results are shown in Table 4.

TABLE 4 comparison of network Performance between different BottleNeck structures

As can be seen from Table 4, the influence of adding different bottleneck layer structures to the upper bottleneck layer from the bottom on the characteristic pyramid of the Neck part of the model is large, the detection speed of the SSANet model is reduced by 1.85% by adopting a Transformer module compared with that of the SK5-YOLOv5s model, the detection speed of a single image reaches 3.2ms, and the detection precision is improved by 1.90% and reaches 96.30%. And other different bottleneck layer structures are added, so that the detection speed is improved, but the detection precision is not as good as that of a transform module structure. Experiments prove that the Transformer module adopted by the system fuses feature embedding to aggregate feature map information of different scales, and meanwhile, attention to an ammonia gas leakage target is enhanced by fully utilizing the feature interaction of cross-space and scales through a multi-head attention mechanism. Therefore, a Transformer module is selected as a characteristic pyramid of the Neck part of the model for carrying out characteristic fusion from bottom to top aiming at the light-weight infrared real-time detection of ammonia leakage, and the improvement of target attention guarantee precision is extracted. The Transformer module comprises an image slice part, a data embedding part and an encoding layer part.

Before step S310 in the method provided in the embodiment of the present application, the method further includes:

step 1: obtaining a first sample according to the infrared image detection data set, and taking the first sample as a first initialization clustering center;

step 2: calculating the distance between each data sample in the infrared image detection data set and the first initialization clustering center, and selecting the minimum distance from all sample distances as a first cluster;

and step 3: determining a maximum distance sample based on the calculated distances of all samples, and taking the maximum distance sample as a second cluster center;

and 4, step 4: repeating the step 2 to the step 3 until K clustering centers are obtained, and performing clustering calculation on the K clustering centers to obtain the size of a labeling frame;

and 5: and obtaining ammonia gas leakage size information according to the size of the marked frame, wherein the ammonia gas leakage size information is the area and the aspect ratio information of the leakage area of the ammonia gas leakage at different moments, and the ammonia gas leakage size information is used as a candidate frame parameter of the detection network model SSANet.

Specifically, a K-means algorithm is adopted to perform cluster analysis on the ammonia gas leakage infrared data set, and the area and the height-width ratio of an ammonia gas leakage area at different moments are analyzed so as to be suitable for a candidate frame for ammonia gas leakage infrared detection and preset target detection model candidate frame parameter model parameters;

the ammonia gas leakage detection type has only one type, the ammonia gas leakage is diffused from nothing to nothing, the distance is used as a similarity index in an iterative type clustering K-means clustering algorithm, therefore K types in a given data set are found, the center of each type is obtained according to the mean value of all numerical values in the type, and the center of each type is described by a clustering center. And gradually updating the value of the clustering center by an iterative method until the best clustering result is obtained. Randomly selecting a sample in the ammonia gas leakage infrared detection data set as a first initialization clustering center;

calculating the distance between each sample point in the sample and the initialized clustering center, and selecting the shortest distance as a first cluster;

determining a maximum distance sample based on the calculated distances of all samples, and taking the maximum distance sample as a second cluster center;

repeating the two steps until k clustering centers are selected;

calculating a final clustering result for the K clustering centers by using a K-means algorithm to obtain the size of a labeling frame;

and obtaining the area and the height-width ratio of leakage areas of the ammonia gas leakage at different moments by using the size of the labeling frame obtained by clustering analysis, and modifying the parameters of the light-weight non-contact type ammonia gas leakage infrared real-time detection SSANet model candidate frame.

Preferably, in the embodiment of the application, 9 clustering centers are adopted to cluster the sizes of the labeling frames in the ammonia gas leakage infrared detection data set, the sizes of the labeling frames close to the central point are classified, the values of the clustering centers are gradually updated through an iterative method until an optimal clustering result is obtained, and the sizes of the labeling frames obtained through clustering analysis are used for modifying the candidate frame parameters of the target detection model.

Further, the ammonia gas leakage detection type is only one type, the ammonia gas leakage is diffused from nothing to nothing, and the size of the candidate frame is changed from small to large regularly. The parameters of the candidate frame of the YOLOv5s basic model cannot meet the actual requirement of ammonia gas leakage infrared image detection, and the parameters of the candidate frame which regularly changes from small to large in size need to be redesigned to meet the size requirement of the ammonia gas leakage process. And performing clustering calculation on the real frame size marked by the data set by adopting a K-means clustering algorithm. The K-means clustering algorithm is an iterative clustering algorithm, so that the characteristics of the target can be better reflected by selecting a more accurate candidate frame by the model, blind searching of the model during training is avoided, the detection effect of the model is improved, and the model can be fast converged to achieve real-time performance.

YOLOv5s has three detection layers in total, each detection layer has 3 candidate frames with different aspect ratios for identifying and positioning the target, and has 9 candidate frames in total, so that a 640 × 640 pixel image is taken as an input. And performing K-means cluster analysis on the ammonia gas leakage infrared detection data set image by taking the number of the candidate boxes as 9. The ratio of the sizes of the three candidate boxes of the detection layer before and after clustering is shown in table 5.

TABLE 5 detection of layer initial candidate frame size

As can be seen from Table 5, the distribution range of the aspect ratio of the candidate frame of the base model of YOLOv5s before clustering ranges from 0.70 to 2.03, and the distribution range of the aspect ratio of the candidate frame obtained after cluster analysis of the ammonia leakage infrared detection data set ranges from 0.19 to 1.17. The change of the height-to-width ratio distribution range of the candidate frames before and after clustering is large, and the change condition that the marked real frames gradually increase from small to large is met, so that the effect of clustering analysis on the ammonia gas leakage infrared detection data set by adopting the K-means algorithm is obvious, the candidate frame parameters obtained by clustering are beneficial to identifying and positioning the target by the model, and the convergence speed and accuracy of the model can be improved.

The YOLOv5s model was trained using pre-processed images that did not use the original images of the K-means cluster candidate boxes, and the K-means cluster candidate boxes. The model obtained by training the original image of the K-means clustering candidate frame is recorded as Kms-YOLOv5s, the model obtained by training the preprocessed image of the K-means clustering candidate frame is recorded as Kms-Prep-YOLOv5s, and the performance comparison experiments of the three are shown in Table 6.

TABLE 6 comparison of network Performance before and after clustering

As can be seen from table 6, after the frame candidate parameters are preset again, there is no influence on the size of the model weight and the detection speed, all the model weights are 14.40M and the single image detection speed is maintained for 3.6 ms. The detection precision is obviously improved, and the average precision mAP of a Kms-YOLOv5s model obtained by training the original image of the K-means cluster candidate frame is improved by nearly 1.00%; the average precision mAP of Kms-Prep-YOLOv5s obtained by adopting the pre-processing image training of the K-means clustering candidate frame is improved by 1.50 percent, which shows that the aspect ratio of the candidate frame preset by adopting the K-means clustering algorithm accords with the real size of ammonia leakage and effectively improves the accuracy of the model for detecting the ammonia leakage.

Step S400 in the method provided in the embodiment of the present application further includes:

s410: performing size clustering on the ammonia gas leakage infrared data set in the training set by using K clustering centers to obtain a clustering result, and using the clustering result as a model candidate frame parameter;

s420: extracting the characteristics of the ammonia gas leakage infrared data through the characteristic extraction network to obtain characteristic detection extraction information;

s430: performing bottom-up fusion on the feature detection extraction information through the Transformer module to obtain multi-scale feature information;

s440: and training the detection network model SSANet by using infrared data of ammonia gas leakage in the training set to obtain the shuffled self-attention network model, wherein the shuffled self-attention network model is used for extracting the multi-scale feature information to detect ammonia gas leakage information.

Specifically, firstly, carrying out cluster analysis on the ammonia gas leakage infrared data set by adopting a K-means algorithm, and analyzing the area and the height-width ratio of an ammonia gas leakage area at different moments so as to be suitable for a candidate frame for ammonia gas leakage infrared detection and preset target detection model candidate frame parameter model parameters; secondly, aiming at the problem that ammonia gas is colorless, flammable and explosive and has toxic gas, and the detection real-time performance is provided by the characteristics of occurrence time and unfixed place of leakage, a shuffle self-attention network (SSANet) model is designed, a 3 x 3 depth separable convolution kernel is designed into a 5 x 5 SK5Block module to reconstruct a feature extraction network, the receptive field is enlarged, the detection precision is improved, and the weight of the model is reduced; and finally, a Transformer module is adopted as a bottleneck layer of the model feature pyramid from bottom to top to realize multi-head attention fusion from bottom to top in a leakage area, and multi-scale feature information is obtained so as to realize the fusion of global features and local features and improve the accuracy of the model. And (4) training an ammonia gas leakage infrared detection data set by using SSANet to obtain a final ammonia gas leakage infrared detection model. Illustratively, the specific steps for detecting the ammonia gas leakage information by taking K ═ 9 as an example are as follows:

step 1: clustering the sizes of the marking frames in the ammonia gas leakage infrared detection data set by adopting 9 clustering centers, classifying the sizes of the marking frames close to the central point, gradually updating the values of the clustering centers by an iterative method until an optimal clustering result is obtained, and modifying the candidate frame parameters of the target detection model by using the sizes of the marking frames obtained by clustering analysis;

step 2: the method aims at the problem that the ammonia gas is colorless, flammable and explosive and has toxicity, and the detection real-time performance is greatly required by the characteristics of occurring time and unfixed place of leakage of the ammonia gas, so that the ammonia gas detection model has less reasoning calculation amount, the reasoning speed of the model is accelerated, and the ammonia gas leakage real-time detection is realized. The invention reconstructs the feature extraction network of the model to reduce the calculated amount and the weight of the compression model during model inference. The Shuffle self-attention network model (SSANet) designs a 3 × 3 depth separable Convolution kernel as a 5 × 5 SK5Block module to reconstruct a lightweight Shuffle feature extraction network, and the SK5Block module mainly adopts a 1 × 1 ordinary Convolution and a 3 × 3 depth separable Convolution kernel to carry out design expansion into a 5 × 5 depth separable Convolution (DWConv) and Channel Shuffle (Channel Shuffle) structure.

According to different moving steps (Stride) of convolution kernels, two lightweight convolution modules are designed, wherein one lightweight convolution module is a lightweight SK5Block module with the step size of 1(Stride equal to 1), and the other lightweight convolution module is a lightweight SK5Block module with the step size of 2(Stride equal to 2);

and step 3: the defect that detection precision is low due to the fact that a model focuses on local features and ignores global features caused by the fact that irregular movement of ammonia cloud clusters and the phenomenon that the contrast ratio of diffusion regions in infrared images is low are not obvious when ammonia leaks is overcome. A Transformer module with a self-attention mechanism is introduced, infrared images can be converted into a sequence, and the sequence can be input into a model for processing. On the basis of a feature extraction network reconstructed by an SK5Block module, a Transformer is used as a bottleneck layer of a model feature pyramid from bottom to top, and multi-scale feature information is obtained, so that fusion of global features and local features is achieved. The Transformer module can capture global information and rich context information and execute global reasoning on the image and the predicted specific target, and the detection precision of the leakage characteristic importance improvement model can be improved while the expensive calculation and storage cost can be reduced by applying the Transformer module to the low-resolution characteristic diagram;

and 4, step 4: after the original input sequence is subjected to a plurality of groups of self-attention processing processes by a transform coding layer part SSANet model, two full-connection layers (Linear) are used for replacing the original layer normalization processing, and the influence of two times of Linear full-connection transformation on reducing the calculation complexity and effectively reducing the sample batch size is reduced. Therefore, the feature space extracted by the backbone network is enriched by aggregating the feature information of different branches, the Transformer module integrates feature embedding to aggregate the feature map information of different scales, and meanwhile, the attention of the ammonia gas leakage target is enhanced by fully utilizing the feature interaction of cross-space and scale through a multi-head attention mechanism. Compared with other different bottleneck layer structures, a Transformer module is selected as a feature pyramid of the Neck part of the model for carrying out feature fusion from bottom to top aiming at the light-weight infrared real-time detection of ammonia leakage, and the improvement of target attention guarantee precision is extracted.

As shown in fig. 3 and fig. 4, step S420 in the method provided in the embodiment of the present application further includes:

s421: when the first step length lightweight convolution module is used, dividing an input feature map channel into a first branch and a second branch, wherein the first branch is used for keeping self information and transmitting downwards to obtain a first channel feature, and the second branch passes through a 1 × 1 convolution channel and is fused with a 5 × 5 depth separable convolution to obtain a second channel feature;

s422: shuffling through the channel shuffling structure based on the first channel characteristics and the second channel characteristics to obtain the characteristic detection extraction information;

s423: and when the second step length light convolution module is used, dividing the input feature graph into two branches which are mapped equally to obtain two branch features, shuffling the two branch features to obtain the feature detection extraction information, and outputting the feature detection extraction information.

Specifically, the ammonia gas detection model has less inference calculation amount, and the inference speed of the model is increased to realize the real-time detection of ammonia gas leakage. The feature extraction network with the reconstructed model reduces the calculation amount and the compression model weight during model reasoning. The feature extraction network of the model built by the light weight SK5Block module is balanced in speed and precision. The SK5Block module is mainly composed of 1 × 1 normal Convolution and 3 × 3 depth separable Convolution kernel to design and expand to 5 × 5 depth separable Convolution (DWConv) and Channel Shuffle (Channel Shuffle) structures.

According to the difference of the moving step length (Stride) of the convolution kernel, two lightweight convolution modules are designed, one is a lightweight SK5Block module with the step length of 1(Stride being 1), as shown in FIG. 2; the other is a lightweight SK5Block module with step size 2(Stride 2), as shown in fig. 3.

Channel separation (Channel Split) grouping operation is started in a lightweight SK5Block module with the step size of 1(Stride 1), an input characteristic Channel is divided into two branches, wherein one branch directly retains self information and transmits the information downwards, the other branch improves the detection speed of the model through 1 × 1 ordinary convolution, and meanwhile, the operation complexity of model training is reduced by fusing deep separable convolution. Finally, the characteristic interaction among the channels is realized through the channel shuffling, and the purpose of improving the model precision is achieved. The characteristic channel is divided into two parts in the lightweight convolution module, so that the parallel reduction of the model parameter quantity of the network is facilitated, and the running speed is increased. In the lightweight SK5Block module with the step size of 2(Stride 2), the input feature map is divided into two branches which are mapped equally, and a concat operation is adopted for integrating feature map information during output. The channel shuffling operation and the concat operation can be combined into an element-level operation, channel dimensions are enlarged, information among channels can be mutually transmitted, and the operation can effectively improve the generalization of a model and the model speed.

As shown in fig. 5, step S430 in the method provided in the embodiment of the present application further includes:

s431: slicing the input infrared image through the image slicing part, and converting the sliced image sub-blocks into a sequence;

s432: inputting the sequence into a detection network model SSANet, and embedding the positions of the image subblocks through the data embedding part to obtain embedded image information, wherein the position embedding is the position information between the pixel values of the embedded image subblocks;

s433: adding classification marks to the embedded image information, inputting the image information added with the classification marks to a coding layer part, fusing a multi-head attention mechanism to perform feature extraction to obtain multiple groups of self-attention processing features, and aggregating the multiple groups of attention processing features through a full-connection layer to obtain the multi-scale feature information.

Specifically, the defect that detection accuracy is low due to the fact that a model focuses on local features and ignores global features caused by the fact that the phenomenon that the contrast ratio of irregular motion of ammonia cloud clusters and diffusion regions in infrared images is low and the features are not obvious when ammonia leaks is solved. A Transformer module with a self-attention mechanism is introduced, infrared images can be converted into a sequence, and the sequence can be input into a model for processing. On the basis of a feature extraction network reconstructed by an SK5Block module, a Transformer is used as a bottleneck layer of a model feature pyramid from bottom to top, and multi-scale feature information is obtained, so that fusion of global features and local features is achieved.

The Transformer module can capture global information and rich context information and execute global reasoning on images and predicted specific targets, and the application of the Transformer module to the low-resolution feature map can reduce expensive calculation and storage cost and improve the detection precision of the leakage feature importance improvement model at the same time. As shown in fig. 6, the Transformer module is divided into three parts:

(1) an image slice section: the Transformer module firstly divides an input infrared image into N subblocks (Patches) of P multiplied by C, and converts the subblocks into N P subblocks through a flattening (Flatten) operation ² C-dimensional vectors, images are converted into sequences and then can be input into a model for processing;

(2) the data embedding part: after infrared image processing, in order to avoid the influence of the size of the sub-blocks on the model structure, linear mapping is adopted to convert different flattened sub-blocks into D-dimensional vectors, and Position Embedding (Position Embedding) is added to Position information between pixel values of the image without loss. The position embedding adopts a trainable parameter, and the dimension is the same as that of image transformation, so that the relationship between pixels can be captured in a high-dimensional vector space, and the dependence of a model on the number of input images is reduced;

(3) transformer coding layer part: under the infrared imaging technology, the phenomenon that the black and gray cloud cluster characteristics presented in the ammonia gas leakage area and the low contrast characteristics of the background are not obvious exists, the transform coding layer can consider various attention distributions and pay attention to different aspects of leakage information, and the ammonia gas leakage detection precision of the model under a complex scene is improved while less space-time complexity is kept. Before image data is input into a transform coding layer (Encoder), a class token for classification needs to be added, and a Multi-Head Attention mechanism (Multi-Head Attention) is fused to extract features.

After the original input sequence is subjected to a plurality of groups of self-attention processing processes by a transform coding layer part SSANet model, two full-connection layers (Linear) are used for replacing the original layer normalization processing, and the influence of two times of Linear full-connection transformation on reducing the calculation complexity and effectively reducing the sample batch size is reduced. Therefore, the feature space extracted by the backbone network is enriched by aggregating the feature information of different branches.

In summary, the embodiment of the present application has at least the following technical effects:

1. the method provided by the embodiment of the application comprises the steps of obtaining an infrared image detection data set; normalizing the infrared image detection data set, dividing the infrared image detection data set into a training set and a test set, and performing ammonia gas leakage infrared data annotation on the infrared image detection data set; constructing a shuffled self-attention network structure; training the shuffled self-attention network structure by using the training set with the ammonia leakage infrared data labels to obtain a shuffled self-attention network model; and detecting the test set through the shuffle self-attention network model, calling the final shuffle self-attention network model and a test program, and inputting an ammonia gas leakage infrared image to determine a detection result. The technical problems that a conventional ammonia gas leakage detection device in the prior art is small in detection range, poor in sensitivity, low in real-time performance and incapable of positioning a leakage source are solved, real-time accurate monitoring of ammonia gas leakage can be achieved at a long distance, safety of workers is guaranteed, response is timely made when ammonia gas leaks, and economic loss is reduced.

2. The gas sensor aims at the problems that a plurality of areas to be detected cannot reach due to the contact principle of a conventional gas sensor adopting point position measurement for ammonia gas leakage detection, the detection range is small, the sensitivity is poor, the real-time performance is not high, leakage sources cannot be positioned and the like, and the safety of workers is greatly threatened. The invention combines an infrared imaging detection technology with ammonia gas leakage detection, and provides a shuffle self-attention network model (SSANet) for realizing infrared real-time non-contact detection of ammonia gas leakage; because the ammonia gas leakage image obtained by the thermal infrared imager has high noise content and low contrast, the method firstly removes noise points randomly distributed in a background area by adopting a non-local mean de-noising method, and simultaneously well maintains the local characteristics of edge pixels of an ammonia gas cloud area. And then, performing contrast enhancement pretreatment on the ammonia gas leakage infrared image by adopting a contrast-limiting self-adaptive histogram equalization algorithm to establish an ammonia gas leakage infrared detection data set.

3. According to the method, the marked real frame size is calculated in a clustering mode through a K-means clustering algorithm according to the actual diffusion condition of ammonia gas leakage from the beginning, the area and the height-width ratio of an ammonia gas leakage area at different moments are analyzed, and the candidate frame suitable for infrared detection of ammonia gas leakage and the candidate frame parameters of a preset target detection model are used, so that the detection precision of the model is improved.

4. The method aims at the problem that ammonia gas is colorless, flammable and explosive and has toxic gas, the detection real-time performance of the ammonia gas is greatly required by the characteristics of occurrence time and unfixed place of leakage, a shuffle self-attention network model (SSANet) is designed, a 3 x 3 depth separable convolution kernel is designed into a 5 x 5 SK5Block module to reconstruct a feature extraction network, the receptive field is enlarged, the detection precision is improved, and meanwhile, the model weight is reduced; and finally, a Transformer module is adopted as a bottleneck layer of the model characteristic pyramid from bottom to top to realize multi-head attention fusion of a leakage area from bottom to top, multi-scale characteristic information is obtained, so that fusion of global characteristics and local characteristics is realized, the precision of the model is improved, an SSANet is used for training an ammonia leakage infrared detection data set, and finally the ammonia leakage infrared detection model is obtained for detecting ammonia leakage. The weight of the SSANet model is reduced by 76.4 percent compared with the weight of the YOLOv5s basic model, and is reduced to 3.4M; the average detection speed of a single image is increased by 1.1% and reaches 3.2 ms; the average detection precision is improved by 3.5 percent and reaches 96.3 percent. The invention realizes the infrared lightweight real-time detection of ammonia gas leakage, and provides an effective real-time detection method for developing a non-contact detection device for ammonia gas leakage to ensure the safe production and stable operation of ammonia-related enterprises.

Example two

Based on the same inventive concept as the ammonia gas leakage shuffled self-attention light-weight infrared detection method in the foregoing embodiment, as shown in fig. 7, the present application provides an ammonia gas leakage shuffled self-attention light-weight infrared detection system, wherein the system includes:

a first obtaining unit 11 for obtaining an infrared image detection dataset;

the first processing unit 12 is configured to divide the infrared image detection data set into a training set and a test set after normalization processing is performed on the infrared image detection data set, and perform ammonia gas leakage infrared data labeling on the infrared image detection data set;

a first construction unit 13 for constructing a shuffled self-attention network structure;

a second processing unit 14, configured to train the shuffled self-attention network structure using the training set with the ammonia leakage infrared data labels, to obtain a shuffled self-attention network model;

and a third processing unit 15, configured to detect the test set through the shuffled self-attention network model, call a final shuffled self-attention network model and a test program, and input an ammonia gas leakage infrared image to determine a detection result.

Further, the system further comprises:

the fourth processing unit is used for acquiring an ammonia gas leakage video through the thermal infrared imager, and performing frame taking processing on the ammonia gas leakage video to obtain an infrared image set;

the fifth processing unit is used for carrying out non-local mean value denoising on the infrared image set to obtain a denoised infrared image set;

a sixth processing unit, configured to perform adaptive histogram equalization processing on the denoised infrared image set to obtain the infrared image detection data set.

Further, the system further comprises:

a seventh processing unit, configured to perform image size preprocessing on the infrared image detection data in the infrared image detection data set;

an eighth processing unit, configured to determine a target detection area based on the infrared image detection data subjected to size preprocessing, and set a bounding box according to the target detection area;

a ninth processing unit, configured to generate a preset data set format tag file according to the bounding box of the target detection area;

a tenth processing unit, configured to label the preset data set format label file, where the label information includes a boundary frame ammonia gas leakage position and a leakage area size;

and the eleventh processing unit is used for carrying out format conversion on the preset data set format label file and determining the ammonia gas leakage infrared data label, wherein the ammonia gas leakage infrared data label is a first preset format label file and comprises a boundary frame center point coordinate.

Further, the system further comprises:

a second construction unit for constructing a detection network model SSANet;

a twelfth processing unit to construct a feature extraction network of the detection network model SSANet with a lightweight SK5Block module, wherein the feature extraction network comprises a 5 × 5 depth separable convolution, channel shuffle structure;

a thirteenth processing unit, configured to add a transform module with a self-attention mechanism as a bottom-up bottleneck layer of a model feature pyramid on the feature extraction network, where the transform module includes an image slice portion, a data embedding portion, and an encoding layer portion.

Further, the system further comprises:

a fourteenth processing unit for performing step 1: obtaining a first sample according to the infrared image detection data set, and taking the first sample as a first initialization clustering center;

a fifteenth processing unit configured to perform step 2: calculating the distance between each data sample in the infrared image detection data set and the first initialization clustering center, and selecting the minimum distance from all sample distances as a first cluster;

a sixteenth processing unit configured to perform step 3: determining a maximum distance sample based on the calculated distances of all samples, and taking the maximum distance sample as a second cluster center;

a seventeenth processing unit configured to perform step 4: repeating the step 2 to the step 3 until K clustering centers are obtained, and performing clustering calculation on the K clustering centers to obtain the size of a labeling frame;

an eighteenth processing unit for performing step 5: and obtaining ammonia gas leakage size information according to the size of the marked frame, wherein the ammonia gas leakage size information is the area and the aspect ratio information of the leakage area of the ammonia gas leakage at different moments, and the ammonia gas leakage size information is used as a candidate frame parameter of the detection network model SSANet.

Further, the system further comprises:

a nineteenth processing unit, configured to perform label frame size clustering on the ammonia leakage infrared data set in the training set by using K clustering centers, obtain a clustering result, and use the clustering result as a model candidate frame parameter;

a twentieth processing unit, configured to perform feature extraction on the ammonia gas leakage infrared data through the feature extraction network, and obtain feature detection extraction information;

a twenty-first processing unit, configured to perform bottom-up fusion on the feature detection extraction information through the Transformer module, so as to obtain multi-scale feature information;

a twenty-second processing unit, configured to train the detection network model SSANet with infrared data of each ammonia gas leak in the training set, to obtain the shuffled self-attention network model, where the shuffled self-attention network model is used to extract the multi-scale feature information to detect ammonia gas leak information.

Further, the system further comprises:

a twenty-third processing unit, configured to, when the first step size and lightweight convolution module is used, divide an input feature map channel into a first branch and a second branch, where the first branch is to retain self information and transmit the information downward to obtain a first channel feature, and the second branch passes through a 1 × 1 convolution channel and is fused with a 5 × 5 depth separable convolution to obtain a second channel feature;

a twenty-fourth processing unit configured to perform shuffling by the channel shuffling structure based on the first channel feature and the second channel feature, and obtain the feature detection extraction information;

a twenty-fifth processing unit, configured to, in the case of the second step-size and light-weight convolution module, divide an input feature map into two branches that are mapped equally, obtain two branch features, shuffle the two branch features, obtain the feature detection extraction information, and output the feature detection extraction information.

Further, the system further comprises:

a twenty-sixth processing unit, configured to slice the input infrared image through the image slicing portion, and convert the sliced image sub-blocks into a sequence;

a twenty-seventh processing unit, configured to input the sequence into a detection network model SSANet, perform position embedding on the image subblock by using the data embedding part, and obtain embedded image information, where the position embedding is position information between pixel values of the embedded image subblock;

and the twenty-eighth processing unit is used for adding a classification mark to the embedded image information, inputting the image information added with the classification mark to a coding layer part, performing feature extraction by combining a multi-head attention mechanism to obtain multiple groups of self-attention processing features, and aggregating the multiple groups of attention processing features through a full-connection layer to obtain the multi-scale feature information.

EXAMPLE III

Based on the same inventive concept as the ammonia gas leakage shuffled self-attention light-weight infrared detection method in the foregoing embodiments, the present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method as in the first embodiment.

Exemplary electronic device

The electronic device of the present application is described below with reference to figure 8,

based on the same inventive concept as the ammonia gas leakage shuffled self-attention light-weight infrared detection method in the foregoing embodiment, the present application also provides an ammonia gas leakage shuffled self-attention light-weight infrared detection system, including: a processor coupled to a memory, the memory storing a program that, when executed by the processor, causes the system to perform the steps of the method of embodiment one.

The electronic device 300 includes: processor 302, communication interface 303, memory 301. Optionally, the electronic device 300 may also include a bus architecture 304. Wherein, the communication interface 303, the processor 302 and the memory 301 may be connected to each other through a bus architecture 304; the bus architecture 304 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus architecture 304 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 8, but this is not intended to represent only one bus or type of bus.

Processor 302 may be a CPU, microprocessor, ASIC, or one or more integrated circuits for controlling the execution of programs in accordance with the teachings of the present application.

The communication interface 303 may be any device, such as a transceiver, for communicating with other devices or communication networks, such as an ethernet, a Radio Access Network (RAN), a Wireless Local Area Network (WLAN), a wired access network, and the like.

The memory 301 may be, but is not limited to, a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, an electrically erasable Programmable read-only memory (EEPROM), a compact-read-only-memory (CD-ROM) or other optical disk storage, optical disk storage (including compact disk, laser disk, optical disk, digital versatile disk, blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory may be self-contained and coupled to the processor through a bus architecture 304. The memory may also be integral to the processor.

The memory 301 is used for storing computer-executable instructions for implementing the present application, and is controlled by the processor 302 to execute. The processor 302 is configured to execute computer-executable instructions stored in the memory 301, so as to implement the ammonia gas leakage shuffled self-attention light-weighted infrared detection method provided in the foregoing embodiments of the present application.

Those of ordinary skill in the art will understand that: the various numbers of the first, second, etc. mentioned in this application are for convenience of description and are not intended to limit the scope of this application nor to indicate the order of precedence. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. "at least one" means one or more. At least two means two or more. "at least one," "any," or similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one (one ) of a, b, or c, may represent: a, b, c, a-b, a-c, b-c, or a-b-c, wherein a, b, c may be single or multiple.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the procedures or functions described in accordance with the present application are generated, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device including one or more available media integrated servers, data centers, and the like. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

The various illustrative logical units and circuits described in this application may be implemented or operated through the design of a general purpose processor, a digital signal processor, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor may be a microprocessor, but, in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other similar configuration.

The steps of a method or algorithm described in this application may be embodied directly in hardware, in a software element executed by a processor, or in a combination of the two. The software cells may be stored in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. For example, a storage medium may be coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC, which may be disposed in a terminal. In the alternative, the processor and the storage medium may reside in different components within the terminal. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Although the present application has been described in conjunction with specific features and embodiments thereof, it will be evident that various modifications and combinations can be made thereto without departing from the spirit and scope of the application. Accordingly, the specification and figures are merely exemplary of the application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of the application. It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the present application and its equivalent technology, it is intended that the present application include such modifications and variations.

Claims

1. A shuffled self-attention light-weight infrared detection method for ammonia gas leakage, characterized by comprising:

acquiring an infrared image detection data set;

normalizing the infrared image detection data set, dividing the infrared image detection data set into a training set and a test set, and performing ammonia gas leakage infrared data annotation on the infrared image detection data set;

constructing a shuffled self-attention network structure;

training the shuffled self-attention network structure by using the training set with the ammonia leakage infrared data labels to obtain a shuffled self-attention network model;

and detecting the test set through the shuffle self-attention network model, calling the final shuffle self-attention network model and a test program, and inputting an ammonia gas leakage infrared image to determine a detection result.

2. The method of claim 1, wherein said obtaining an infrared image detection dataset comprises:

acquiring an ammonia gas leakage video through a thermal infrared imager, and performing frame taking processing on the ammonia gas leakage video to obtain an infrared image set;

carrying out non-local mean denoising on the infrared image set to obtain a denoised infrared image set;

and carrying out self-adaptive histogram equalization processing on the basis of the de-noised infrared image set to obtain the infrared image detection data set.

3. The method of claim 1, wherein performing ammonia gas leak infrared data annotation on the infrared image detection dataset comprises:

carrying out image size preprocessing on the infrared image detection data in the infrared image detection data set;

determining a target detection area based on the infrared image detection data after size preprocessing, and setting a boundary frame according to the target detection area;

generating a label file with a preset data set format according to the boundary frame of the target detection area;

marking the preset data set format label file, wherein marking information comprises the ammonia gas leakage position and the size of a leakage area of a boundary frame;

and performing format conversion on the preset data set format label file, and determining the ammonia gas leakage infrared data label, wherein the ammonia gas leakage infrared data label is a first preset format label file and comprises a boundary frame central point coordinate.

4. The method of claim 1, wherein said constructing a shuffled self-attention network structure comprises:

constructing a detection network model SSANet;

constructing a feature extraction network of the detection network model SSANet by using a lightweight SK5Block module, wherein the feature extraction network comprises a 5 x 5 depth separable convolution and channel shuffling structure;

adding a Transformer module with a self-attention mechanism on the feature extraction network as a bottleneck layer of a model feature pyramid from bottom to top, wherein the Transformer module comprises an image slicing part, a data embedding part and an encoding layer part.

5. The method of claim 4, wherein prior to constructing the detection network model SSANet, comprising:

6. The method of claim 4, wherein the training the shuffled self-attention network structure with the training set having the ammonia leak infrared data annotation, obtaining a shuffled self-attention network model, comprises:

performing size clustering on the ammonia gas leakage infrared data set in the training set by using K clustering centers to obtain a clustering result, and using the clustering result as a model candidate frame parameter;

extracting the characteristics of the ammonia gas leakage infrared data through the characteristic extraction network to obtain characteristic detection extraction information;

performing bottom-up fusion on the feature detection extraction information through the Transformer module to obtain multi-scale feature information;

and training the detection network model SSANet by using infrared data of ammonia gas leakage in the training set to obtain the shuffled self-attention network model, wherein the shuffled self-attention network model is used for extracting the multi-scale feature information to detect ammonia gas leakage information.

7. The method of claim 6, wherein the feature extraction network comprises a first length and weight reduction convolution module and a second length and weight reduction convolution module, and the feature extraction network is used for performing feature extraction on the ammonia gas leakage infrared data to obtain feature detection extraction information and comprises:

when the first step length lightweight convolution module is used, dividing an input feature map channel into a first branch and a second branch, wherein the first branch is used for keeping self information and transmitting downwards to obtain a first channel feature, and the second branch passes through a 1 × 1 convolution channel and is fused with a 5 × 5 depth separable convolution to obtain a second channel feature;

shuffling through the channel shuffling structure based on the first channel characteristics and the second channel characteristics to obtain the characteristic detection extraction information;

and when the second step length light convolution module is used, dividing the input feature graph into two branches which are mapped equally to obtain two branch features, shuffling the two branch features to obtain the feature detection extraction information, and outputting the feature detection extraction information.

8. The method of claim 6, wherein the bottom-up fusion of the feature detection extraction information by the transform module to obtain multi-scale feature information comprises:

slicing the input infrared image through the image slicing part, and converting the sliced image sub-blocks into a sequence;

inputting the sequence into a detection network model SSANet, and embedding the positions of the image subblocks through the data embedding part to obtain embedded image information, wherein the position embedding is the position information between the pixel values of the embedded image subblocks;

adding classification marks to the embedded image information, inputting the image information added with the classification marks to a coding layer part, fusing a multi-head attention mechanism to perform feature extraction to obtain multiple groups of self-attention processing features, and aggregating the multiple groups of attention processing features through a full-connection layer to obtain the multi-scale feature information.

9. An ammonia gas leak shuffled self-attention light-weight infrared detection system, characterized in that the system comprises:

a first obtaining unit for obtaining an infrared image detection dataset;

the first processing unit is used for dividing the infrared image detection data set into a training set and a test set after normalization processing is carried out on the infrared image detection data set, and carrying out ammonia gas leakage infrared data annotation on the infrared image detection data set;

a first construction unit for constructing a shuffled self-attention network structure;

a second processing unit for training the shuffled self-attention network structure using the training set with the ammonia leakage infrared data annotation, to obtain a shuffled self-attention network model;

and the third processing unit is used for detecting the test set through the shuffled self-attention network model, calling a final shuffled self-attention network model and a test program, and inputting an ammonia gas leakage infrared image to determine a detection result.

10. An ammonia gas leakage shuffled self-attention light-weight infrared detection system, characterized by comprising: a processor coupled to a memory, the memory for storing a program that, when executed by the processor, causes a system to perform the steps of the method of any of claims 1 to 8.