CN115761332A - Smoke and flame detection method, device, equipment and storage medium - Google Patents
Smoke and flame detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN115761332A CN115761332A CN202211430723.6A CN202211430723A CN115761332A CN 115761332 A CN115761332 A CN 115761332A CN 202211430723 A CN202211430723 A CN 202211430723A CN 115761332 A CN115761332 A CN 115761332A
- Authority
- CN
- China
- Prior art keywords
- neural network
- smoke
- target
- image data
- feature extraction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of computer vision, in particular to a smoke and flame detection method, a device, equipment and a storage medium, wherein the smoke and flame detection method comprises the following steps: preprocessing the marked image data of the marked target object, and establishing a training data set by using the preprocessed marked image data, wherein the target object comprises smoke and flame; training an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism through a training data set to obtain a target neural network; and acquiring to-be-detected image data of an environment to be detected, and inputting the to-be-detected image data into the target neural network model to obtain a detection result output by the target neural network model. The invention realizes the improvement of the accuracy of the smoke and flame detection.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a smoke and flame detection method, a smoke and flame detection device, smoke and flame detection equipment and a computer readable storage medium.
Background
With the rapid development of deep learning in the field of computer vision, the smoke and flame detection algorithm based on deep learning also becomes the mainstream smoke and flame detection method at present. The smoke and flame detection method of deep learning is divided into One-Stage and Two-Stage on the network structure. The Two-Stage divides the detection problem into Two stages, firstly, a candidate region is obtained to carry out position refinement and classification, and the stages represent that the network comprises R-CNN, faster R-CNN, mask-RCNN and the like. The One-Stage method does not have a Stage of selecting a candidate region, directly generates the class probability and the position coordinate of an object, has higher speed compared with the Two-Stage method, and represents that the network has a YOLO series algorithm, SSD, centerNet and the like.
However, due to the fact that the target in the image data of the detection area may have huge deformation, the target in the image data is unclear when the distance between the detection area and the image sensor is too far, and the characteristic information in the image data of the detection area is small, and the like, the current smoke and flame detection algorithm is prone to phenomena such as missing detection, false detection and the like, and the detection result of smoke and flame is inaccurate.
Disclosure of Invention
The invention mainly aims to provide a smoke and flame detection method, a smoke and flame detection device, smoke and flame detection equipment and a computer readable storage medium, and aims to improve the accuracy of smoke and flame detection.
To achieve the above object, the present invention provides a smoke and flame detection method, comprising the steps of:
preprocessing marked image data of a marked target object, and establishing a training data set by using the preprocessed marked image data, wherein the target object comprises smoke and flame;
training an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism through the training data set to obtain a target neural network model;
and acquiring to-be-detected image data of an environment to be detected, and inputting the to-be-detected image data into the target neural network model to obtain a detection result output by the target neural network model.
Optionally, the initial neural network model includes a feature extraction network and a feedforward prediction network, where the feature extraction network includes a plurality of feature extraction blocks, and each feature extraction block includes a convolution block and an attention mechanism block;
the step of training an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism through the training data set to obtain a target neural network model comprises the following steps:
inputting the training data set into the feature extraction network, and sequentially performing feature extraction through a plurality of feature extraction blocks in the feature extraction network to obtain training features, wherein the output result of a last feature extraction block in the plurality of feature extraction blocks is input data of a next feature extraction block;
fusing a plurality of training features to obtain learning features;
and adjusting network parameters in the initial neural network model according to the training result to obtain the target neural network model.
Optionally, the step of performing feature extraction by using any one of the target extraction blocks to obtain the training features includes:
performing feature extraction through the convolution block of the target extraction block to obtain feature data;
and performing local feature reinforcement on the feature data extracted by the rolling block through an attention mechanism block of the target extraction block to obtain the training features.
Optionally, the convolution block comprises a first convolution layer and a pooling layer.
Optionally, the feed forward prediction network comprises a second convolutional layer, a hidden layer and a linear prediction layer; wherein the content of the first and second substances,
the second convolution layer is used for performing feature extraction on the target feature through a 3-by-3 convolution kernel;
the hidden layer comprises three full-connection layers, wherein the first full-connection layer and the second full-connection layer are respectively connected with a connection activation function.
Optionally, the step of preprocessing the marked image data includes:
performing data enhancement on the marked image data through geometric transformation;
performing blending processing on the marked image data of different target objects;
the marked image data of different image backgrounds are mixed.
Optionally, the detection result includes a target object category, a target object existence probability, and a target object coordinate.
Further, to achieve the above object, the present invention also provides a smoke and flame detecting device including:
the system comprises a preprocessing module, a data processing module and a data processing module, wherein the preprocessing module is used for preprocessing marked image data of a marked target object and establishing a training data set by using the preprocessed marked image data, and the target object comprises smoke and flame;
the training module is used for training an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism through the training data set to obtain a target neural network model;
and the detection module is used for acquiring image data to be detected of an environment to be detected, inputting the image data to be detected into the target neural network model and obtaining a detection result output by the target neural network model.
Furthermore, to achieve the above object, the present invention also provides a smoke and flame detection apparatus comprising a memory, a processor and a smoke and flame detection program stored on the memory and operable on the processor, the smoke and flame detection program when executed by the processor implementing the steps of the smoke and flame detection method described above.
Furthermore, to achieve the above object, the present invention also provides a computer readable storage medium having a smoke and flame detection program stored thereon, which when executed by a processor, implements the steps of the smoke and flame detection method described above.
The method comprises the steps of preprocessing marked image data of a marked target object, establishing a training data set by using the preprocessed marked image data, training an initial neural network model established on the basis of a convolutional neural network and a multi-head attention mechanism through the training data set to obtain a target neural network, acquiring to-be-detected image data of an environment to be detected, inputting the to-be-detected image data into the target neural network model, and obtaining a detection result output by the target neural network model.
According to the invention, an initial neural network model is constructed by combining a convolutional neural network and a multi-head attention mechanism, a convolutional block can learn local characterization information, and an architecture of the convolutional block combined with the multi-head attention mechanism has strong capability of capturing low-frequency signals, so that a target neural network model obtained by training can sense global information of image data to be detected, and the phenomena of missing detection and false detection caused by incomplete characteristic information are avoided.
Drawings
FIG. 1 is a schematic flow chart of a first embodiment of a smoke and flame detection method of the present invention;
FIG. 2 is a schematic flow chart diagram of one embodiment of a smoke and flame detection method of the present invention;
FIG. 3 is a schematic block diagram of an embodiment of a smoke and flame detection method of the present invention;
FIG. 4 is a schematic block diagram of an embodiment of a smoke and flame detection method of the present invention;
FIG. 5 is a schematic diagram of functional modules of a smoke and flame detection device according to an embodiment of the invention;
FIG. 6 is a schematic diagram of a smoke and flame detection apparatus according to an embodiment of the invention;
fig. 7 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
An embodiment of the present invention provides a smoke and flame detection method, and as shown in fig. 1, fig. 1 is a schematic flow chart of a first embodiment of the smoke and flame detection method of the present invention.
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings in which the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application.
In this embodiment, the device for performing the smoke and flame detection method according to the embodiment of the present invention may be a detection device, such as a camera, in a field of an environment to be detected, or may be a device for establishing a communication connection with the detection device, such as a smart phone, a PC (Personal Computer), a tablet Computer, a portable Computer, and the like, and the following description omits to perform the main description for convenience of description. The smoke and flame detection method of the embodiment comprises the following steps:
step S10: preprocessing marked image data of a marked target object, and establishing a training data set by using the preprocessed marked image data, wherein the target object comprises smoke and flame;
in this embodiment, the flame and the smoke are referred to as a target, and image data of the marked target (hereinafter referred to as marked data to show distinction) is acquired.
In a specific embodiment, the pre-processing of the marked image data may be to eliminate information irrelevant to the target object in the marked image data, or to perform data enhancement on the marked image data, which is not limited herein and may be set according to actual requirements.
In this embodiment, after the labeled image data is preprocessed, a training data set is established by using the preprocessed labeled image data.
Step S20: training an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism through the training data set to obtain a target neural network model;
in this embodiment, an initial neural network model is constructed based on a convolutional neural network and a multi-head attention mechanism. Specifically, in the present embodiment, the backbone structure of the initial neural network model may include a feature extraction network and a feedforward prediction network.
And training an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism through a training data set to obtain a target neural network model. In particular embodiments, training the initial neural network model may be based on a loss function.
Step S30: and acquiring to-be-detected image data of an environment to be detected, and inputting the to-be-detected image data into the target neural network model to obtain a detection result output by the target neural network model.
In this embodiment, to-be-detected image data of an environment to be detected is obtained, and the to-be-detected image data is input to the target neural network model to obtain a detection result output by the target neural network model.
In a specific embodiment, the image data to be detected may be acquired by an image sensor disposed in the region to be detected, for example, a camera.
In particular embodiments, the detection result may be a category including an object, such as smoke or flame, an occurrence probability of the object, and coordinates of the object.
Further, in an embodiment, when the detection result is that a target object exists in the image data to be detected, alarm information can be sent out so that a user can carry out safety processing in time.
Further, a possible embodiment is provided, and referring to fig. 2, fig. 2 is a schematic flow chart of an embodiment of the smoke and flame detection method of the present invention. In this embodiment, the labeled image data of the labeled target object is preprocessed, and a training data set is established using the preprocessed labeled image data. The initial neural network model constructed based on the convolutional neural network and the multi-head attention mechanism is trained through a training data set to obtain a target neural network model (that is, after the data shown in fig. 2 is preprocessed, the CNN detects and identifies the target by combining the multi-head attention mechanism).
Acquiring image data to be detected of an environment to be detected, inputting the image data to be detected into the target neural network model, and obtaining a detection result (namely, judging whether smoke and flame exist as shown in fig. 2) output by the target neural network model.
In the embodiment, if smoke and flame exist in the image data to be detected, an alarm is given in the detection system; if the smoke and the flame do not exist in the image data to be detected, the alarm does not need to be given in a detection system, and the detection of the image data to be detected is finished.
Further, in some possible embodiments, in step S10 above: pre-processing the marked image data, comprising:
step S101: performing data enhancement on the marked image data through geometric transformation;
in this embodiment, data enhancement is performed on the marked image data, specifically, in this embodiment, single sample data enhancement is performed through geometric transformation. In a specific embodiment, the geometric transformation includes performing cropping, rotation, or scaling on the marked image data, which is not described herein in detail.
Step S102: performing blending processing on the marked image data of different target objects;
in this embodiment, multi-sample data enhancement is further applied to the labeled image data for processing, and specifically, in this embodiment, the labeled image data of different targets are subjected to hybrid processing, so that the number of training data sets is expanded, and the generalization capability and the test robustness of the target neural network model are improved.
Specifically, in this embodiment, a Mixup algorithm may be used to perform a Mixup process, in which two pieces of marked image data with different target marks are combined together with a certain probability, and the calculation formula is: x is the number of mixed ＝λ*x 1 +(1-λ)*x 2 Wherein λ is a mixing coefficient calculated by the beta distribution, and can be set according to actual requirements, and x 1 、x 2 Is the marked image data.
Step S103: the marked image data of different image backgrounds are mixed.
In the embodiment, the marked image data of different image backgrounds are mixed, so that the number of training data sets is expanded, and the generalization capability and the test robustness of the target neural network model are improved. For a specific processing procedure, refer to step S102, which is not described herein.
It should be noted that, in this embodiment, by performing single sample data enhancement on the marked image data, performing mixed processing on the marked image data of different target objects, and performing mixed processing on the marked image data of different image backgrounds, the number of training data sets can be expanded, and the generalization capability of the target neural network model and the robustness of the test can be improved.
Further, in some possible embodiments, in this embodiment, the detection result includes a category of the target object, an occurrence probability of the target object, and a coordinate of the target object. In a specific embodiment, the coordinates of the target are the coordinates of the center point of the target.
In a specific embodiment, the coordinates in the detection result may be coordinates in feature data extracted from the target neural network model by the target object, coordinates in the feature data by the target object may also be coordinates in which the coordinates in the feature data by the target object are mapped in the image data to be detected, and coordinates in the feature data by the target object are mapped in the environment to be detected.
Further, in the specific embodiment, the detection result may further include the width and the height of the target object, and the area of the target object is obtained based on the width and the height of the target object, so as to determine the occurrence range of smoke and flame, and make the result of smoke and flame detection more practical.
It should be noted that, in this embodiment, the detection result includes the detection result including the type of the target object, the occurrence probability of the target object, and the coordinate of the target object, and compared with the case of detecting only the occurrence probability of the target object, the embodiment improves the detail of the detection result, so that the practicability of the detection result is stronger.
In the embodiment, marked image data of a marked target object is preprocessed, a training data set is established through the preprocessed marked image data, wherein the target object comprises smoke and flames, an initial neural network model established based on a convolutional neural network and a multi-head attention mechanism is trained through the training data set, the target neural network model obtains image data to be detected of an environment to be detected, the image data to be detected is input into the target neural network model to obtain a detection result output by the target neural network.
In the embodiment, the initial neural network model is constructed by combining the convolutional neural network and the multi-head attention mechanism, the convolutional block can learn local characterization information, the architecture of the convolutional block combined with the multi-head attention mechanism has strong capability of capturing low-frequency signals, so that the initial neural network model can learn global information, the trained target neural network model can sense the global information of image data to be detected, and the phenomena of missed detection and false detection caused by incomplete characteristic information are avoided.
Further, a second embodiment of the smoke and flame detection method of the present invention is proposed based on the first embodiment described above, in which the initial neural network model includes a feature extraction network and a feedforward prediction network, the feature extraction network includes a plurality of feature extraction blocks, and each feature extraction block includes a convolution block and an attention mechanism block.
In this embodiment, the step S20: training an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism through the training data set to obtain a target neural network model, wherein the training data set comprises the following steps:
step S201: inputting the training data set into the feature extraction network, and sequentially performing feature extraction through a plurality of feature extraction blocks in the feature extraction network to obtain training features, wherein the output result of a last feature extraction block in the plurality of feature extraction blocks is input data of a next feature extraction block;
in this embodiment, the training data set is input into the feature extraction network, and the training features are obtained by sequentially performing feature extraction by a plurality of feature extraction blocks in the feature extraction network.
Specifically, in this embodiment, each feature extraction block may be connected in series, and the process of sequentially performing feature extraction by a plurality of feature extraction blocks in the feature extraction network may be: the training features extracted by any one of the feature extraction blocks are input into the next feature extraction block connected with the target extraction block, that is, the output result of the last feature extraction block in the feature extraction blocks is the input data of the next feature extraction block.
For example, in an embodiment, when two feature extraction blocks are arranged in the feature extraction network, the two feature extraction blocks may be connected in series, specifically, a first feature extraction block performs feature extraction on a training data set, and training features extracted by the first feature extraction block are used as input data of a second feature extraction block for feature extraction by the second feature extraction block.
Step S202: fusing a plurality of training features to obtain a learning feature;
in this embodiment, after feature extraction is performed on the training data set through the feature extraction network to obtain training features corresponding to each feature extraction block, a plurality of training features are fused to obtain learning features.
In a specific embodiment, the manner of performing feature fusion may be cascade feature fusion, series feature fusion, parallel policy fusion, and the like, which is not described herein again.
Step S203: and adjusting network parameters in the initial neural network model according to the training result to obtain the target neural network model.
In this embodiment, the learning characteristics are input into the feedforward prediction network to learn, and the network parameters in the initial neural network model are adjusted according to the training result to obtain the target neural network model. Specifically, in an embodiment, the network parameters in the initial neural network model may be adjusted based on the loss function, and in this embodiment, when the loss function starts to converge, it is determined that the training of the initial neural network model is completed, so as to obtain the target neural network model. In particular embodiments, the feed forward prediction network may include a convolutional layer, a hidden layer, and a linear prediction layer.
It should be noted that, in this embodiment, an initial neural network model is constructed by combining a convolutional neural network and a multi-head attention mechanism, a convolution block may learn local characterization information, and an architecture of the convolution block combined with the multi-head attention mechanism has a very strong capability of capturing a low-frequency signal, so that a target neural network model obtained through training may sense global information of image data to be detected, thereby avoiding the occurrence of missed detection and false detection due to incomplete feature information.
Further, in some possible embodiments, in step S201: the step of extracting the features through any target extraction block in each feature extraction block to obtain the training features comprises the following steps:
step S2011: performing feature extraction through the convolution block of the target extraction block to obtain feature data;
in this embodiment, any one of the target extraction blocks in each feature extraction block includes a volume block and an attention mechanism block. And performing feature extraction through the convolution block of the target extraction block to obtain feature data. In a specific embodiment, the step size of the convolution block and the convolution kernel are not limited, and may be set according to actual requirements.
Step S2012: and performing local feature enhancement on the feature data extracted by the rolling block through an attention mechanism block of the target extraction block to obtain the training features.
In this embodiment, the feature data extracted by the convolution block is subjected to local feature enhancement by the attention mechanism block of the target extraction block, so as to obtain a training feature.
Specifically, in this embodiment, a multi-head attention mechanism mode is used, so that the initial neural network model can focus on feature information of a subspace in feature data obtained by convolution in different feature extraction blocks, and can capture richer feature information.
In a specific embodiment, after straightening and feature splicing are performed on feature data obtained by convolution, three matrixes needed for calculating self attention are generated to calculate attention weight on a target block. The formulaic calculation of self-attention can be expressed as:
where Q represents a series matrix of features, K is a key of the information, V is a value of the information, and S is a softmax (normalization) operation. In this embodiment, the V value corresponding to the K key is returned by comparing the similarity between Q and K. According to the above formula, the larger the similarity between Q and K, the more attention is weighted.
Further, in some possible embodiments, the convolution block includes a first convolution layer and a pooling layer.
In this embodiment, the convolution block in the feature extraction block includes a first convolution layer and a pooling layer. The feature information is extracted through convolution of the first convolution layer, and feature downsampling is performed through the pooling layer, so that the calculated amount of a multi-head attention mechanism can be reduced, the length of a feature sequence can be greatly shortened, and multi-level rich features can be captured.
Specifically, referring to fig. 3, fig. 3 is a schematic structural diagram of an embodiment of the smoke and flame detection method of the present invention. The embodiment designs four extraction blocks, each feature extraction block comprises a convolution block and an attention mechanism block, and the convolution block comprises a first convolution layer and a pooling layer. In this embodiment, the feature data of each feature extraction block is gradually reduced, and the feature data of each layer in the feature extraction block is also gradually reduced, which is represented by the pyramid-type structure shown in fig. 3.
Further, in some possible embodiments, the feed-forward prediction network includes a second convolutional layer, a hidden layer, and a linear prediction layer; wherein the content of the first and second substances,
the second convolution layer is used for performing feature extraction on the target feature through a convolution kernel of 3 x 3;
the hidden layer comprises three full-connection layers, wherein the first full-connection layer and the second full-connection layer are respectively connected with a connection activation function.
In this embodiment, the feedforward prediction network includes a second convolutional layer, a hidden layer, and a linear prediction layer.
Specifically, in this embodiment, the second convolution layer is used to perform feature extraction on the target feature through the 3 × 3 convolution kernel, so that the feature characterization capability of the target neural network model can be further improved.
In this embodiment, the hidden layer in the feedforward prediction network includes three fully-connected layers, and a connection activation function is connected to each of the fully-connected layers after the first fully-connected layer and the second fully-connected layer.
The full connection layer is responsible for converting the feature data output by convolution into vectors, so that an end-to-end learning process is realized, and each node of the full connection layer is connected with all nodes of the previous layer. The full-connection layer can reduce the influence of the characteristic position on the detection result and improve the robustness of the whole target neural network model. In a specific embodiment, the dimension of the hidden layer may be set according to an actual requirement of a user, and is not limited herein.
The activation function determines what is to be transmitted to the next fully-connected layer, introducing non-linear behavior into the neural network. In this embodiment, the activation function may be a ReLU function.
Further, in a possible embodiment, the initial neural network model includes a feature extraction network and a feed-forward prediction network, wherein the feature extraction network includes a plurality of feature extraction blocks, each of which includes a convolution block and an attention mechanism block. In this embodiment, the convolution block includes a first convolution layer and a pooling layer. The feedforward prediction network comprises a second convolution layer, a hidden layer and a linear prediction layer, wherein the second convolution layer is used for carrying out feature extraction on target features through a 3 x 3 convolution kernel, the hidden layer comprises three layers of fully-connected layers, and the activation function ReLU is connected to each layer of the fully-connected layers behind the first layer and the second layer.
Further, in a possible embodiment, referring to fig. 4, fig. 4 is a schematic flowchart of an embodiment of the smoke and flame detection method of the present invention, in this embodiment, the process of detecting the image data to be detected may be: and inputting the data of the image to be detected into a feature extraction module in a target neural network model, wherein the feature extraction module is constructed on the basis of a convolutional neural network and a multi-head attention mechanism, and inputting the extracted features to be detected into a feedforward prediction network to obtain a detection result.
In this embodiment, the initial neural network model includes a feature extraction network and a feedforward prediction network, where the feature extraction network includes a plurality of feature extraction blocks, and each feature extraction block includes a convolution block and an attention mechanism block. In this embodiment, a training data set is input into a feature extraction network, and feature extraction is sequentially performed by a plurality of feature extraction blocks in the feature extraction network to obtain training features, where an output result of a previous feature extraction block in the plurality of feature extraction blocks is input data of a next feature extraction block, a plurality of training features are fused to obtain learning features, and a network parameter in an initial neural network model is adjusted according to a training result to obtain a target neural network model.
In the embodiment, the convolution neural network is combined with the multi-head attention mechanism to construct the initial neural network model, the convolution block can learn local characterization information, the framework of the convolution block combined with the multi-head attention mechanism has strong capacity of capturing low-frequency signals, the initial neural network model can learn global information, the trained target neural network model can sense the global information of image data to be detected, and the phenomena of missed detection and false detection caused by incomplete characteristic information are avoided.
In addition, the invention also provides a smoke and flame detection device, and referring to fig. 5, fig. 5 is a schematic diagram of functional modules of the smoke and flame detection device according to the embodiment of the invention. The smoke and flame detection device of the present invention comprises:
a preprocessing module 10, configured to preprocess marked image data of a marked target object, and establish a training data set using the preprocessed marked image data, where the target object includes smoke and flames;
the training module 20 is configured to train, through the training data set, an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism, so as to obtain a target neural network model;
the detection module 30 is configured to obtain image data to be detected of an environment to be detected, and input the image data to be detected into the target neural network model to obtain a detection result output by the target neural network model.
Further, the fourth invoking module 40 is further configured to:
further, the initial neural network model comprises a feature extraction network and a feedforward prediction network, wherein the feature extraction network comprises a plurality of feature extraction blocks, and each feature extraction block comprises a convolution block and an attention mechanism block;
the training module 20 is further configured to:
inputting the training data set into the feature extraction network, and sequentially performing feature extraction through a plurality of feature extraction blocks in the feature extraction network to obtain training features, wherein the output result of a last feature extraction block in the plurality of feature extraction blocks is input data of a next feature extraction block;
fusing a plurality of training features to obtain a learning feature;
and adjusting network parameters in the initial neural network model according to the training result to obtain the target neural network model.
Further, the training module 20 is further configured to:
performing feature extraction through the convolution block of the target extraction block to obtain feature data;
and performing local feature enhancement on the feature data extracted by the rolling block through an attention mechanism block of the target extraction block to obtain the training features.
Further, the convolution block includes a first convolution layer and a pooling layer.
Further, the feedforward prediction network comprises a second convolutional layer, a hidden layer and a linear prediction layer; wherein, the first and the second end of the pipe are connected with each other,
the second convolution layer is used for performing feature extraction on the target feature through a convolution kernel of 3 x 3;
the hidden layer comprises three full-connection layers, wherein the first full-connection layer and the second full-connection layer are respectively connected with a connection activation function.
Further, the preprocessing module 10 is further configured to:
performing data enhancement on the marked image data through geometric transformation;
performing blending processing on the marked image data of different target objects;
the marked image data of different image backgrounds are mixed.
Further, the detection result comprises a target object category, a target object existence probability and a target object coordinate.
Wherein, when each functional module of the smoke and flame detection device is operated, the steps of the smoke and flame detection method are realized.
In addition, the invention also provides smoke and flame detection equipment. Referring to fig. 6, fig. 6 is a schematic structural diagram of a smoke and flame detection device according to an embodiment of the present invention. The smoke and flame detection apparatus of embodiments of the invention may particularly be an apparatus for locally operating a smoke and flame detection system.
As shown in fig. 6, a smoke and flame detection apparatus according to an embodiment of the present invention may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to implement connection communication among these components. The user interface 1003 may include a Display (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., a Wi-Fi interface).
A memory 1005 is provided on the smoke and flame detection apparatus body, the memory 1005 having stored thereon a program which when executed by the processor 1001 effects corresponding operations. The memory 1005 is also used to store parameters for use by the smoke and flame detection apparatus. The memory 1005 may be a high-speed RAM memory or a non-volatile memory such as a disk memory. The memory 1005 may alternatively be a storage device separate from the processor 1001.
It will be appreciated by those skilled in the art that the smoke and flame detection apparatus arrangement shown in figure 6 does not constitute a limitation of the smoke and flame detection apparatus and may include more or fewer components than shown, or some components in combination, or a different arrangement of components.
As shown in fig. 6, a memory 1005, which is one type of storage medium, may include an operating system, a network processing module, a user interface module, and a smoke and flame detection program therein.
In the smoke and flame detection apparatus shown in fig. 6, the processor 1001 may be configured to invoke a smoke and flame detection program stored in the memory 1005 and perform the following operations:
preprocessing marked image data of a marked target object, and establishing a training data set by using the preprocessed marked image data, wherein the target object comprises smoke and flame;
training an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism through the training data set to obtain a target neural network model;
and acquiring to-be-detected image data of an environment to be detected, and inputting the to-be-detected image data into the target neural network model to obtain a detection result output by the target neural network model.
Further, the initial neural network model includes a feature extraction network and a feed-forward prediction network, wherein the feature extraction network includes a plurality of feature extraction blocks. The processor 1001 may also be configured to invoke a smoke and flame detection program stored in the memory 1005 and perform the following operations:
inputting the training data set into the feature extraction network, and sequentially performing feature extraction through a plurality of feature extraction blocks in the feature extraction network to obtain training features, wherein the output result of a last feature extraction block in the plurality of feature extraction blocks is input data of a next feature extraction block;
fusing a plurality of training features to obtain learning features;
and adjusting network parameters in the initial neural network model according to the training result to obtain the target neural network model.
Further, the processor 1001 may be further configured to invoke a smoke and flame detection program stored in the memory 1005 and perform the following operations:
performing feature extraction through the convolution block of the target extraction block to obtain feature data;
and performing local feature reinforcement on the feature data extracted by the rolling block through an attention mechanism block of the target extraction block to obtain the training features.
Further, the convolution block includes a first convolution layer and a pooling layer.
Further, the feedforward prediction network comprises a second convolutional layer, a hidden layer and a linear prediction layer; wherein the content of the first and second substances,
the second convolution layer is used for performing feature extraction on the target feature through a 3-by-3 convolution kernel;
the hidden layer comprises three full-connection layers, wherein the first full-connection layer and the second full-connection layer are respectively connected with a connection activation function.
Further, the processor 1001 may be further configured to invoke a smoke and flame detection program stored in the memory 1005 and perform the following operations:
performing data enhancement on the marked image data through geometric transformation;
performing blending processing on the marked image data of different target objects;
and performing mixing processing on the marked image data of different image backgrounds.
Further, the detection result comprises a target object category, a target object existence probability and a target object coordinate.
In addition, the invention also provides a computer readable storage medium. Referring to fig. 7, fig. 7 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention. The computer readable storage medium has stored thereon a smoke and flame detection program which, when executed by the processor, implements the steps of the smoke and flame detection method as described above.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrases "comprising one of 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. With this understanding in mind, the technical solutions of the present invention may be embodied in the form of a software product stored in a computer-readable storage medium (such as ROM/RAM, magnetic disk, optical disk) as described above, and including instructions for causing a smoke and flame detection device (which may be a mobile phone, a computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A smoke and flame detection method, characterized in that it comprises the steps of:
preprocessing marked image data of a marked target object, and establishing a training data set by using the preprocessed marked image data, wherein the target object comprises smoke and flame;
training an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism through the training data set to obtain a target neural network model;
and acquiring to-be-detected image data of an environment to be detected, and inputting the to-be-detected image data into the target neural network model to obtain a detection result output by the target neural network model.
2. The smoke and flame detection method of claim 1, wherein the initial neural network model comprises a feature extraction network and a feed-forward prediction network, wherein the feature extraction network comprises a plurality of feature extraction blocks, each of the feature extraction blocks comprising a convolution block and an attention mechanism block;
the step of training an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism through the training data set to obtain a target neural network model comprises the following steps:
inputting the training data set into the feature extraction network, and sequentially performing feature extraction through a plurality of feature extraction blocks in the feature extraction network to obtain training features, wherein the output result of a last feature extraction block in the plurality of feature extraction blocks is input data of a next feature extraction block;
fusing a plurality of training features to obtain a learning feature;
inputting the learning characteristics into the feedforward prediction network to obtain a training result;
and adjusting network parameters in the initial neural network model according to the training result to obtain the target neural network model.
3. The smoke and flame detection method of claim 2, wherein the step of extracting features by any one of the object extraction blocks to obtain the training features comprises:
performing feature extraction through the convolution block of the target extraction block to obtain feature data;
and performing local feature enhancement on the feature data extracted by the rolling block through an attention mechanism block of the target extraction block to obtain the training features.
4. The smoke and flame detection method of claim 3 wherein the roll-up block comprises a first roll-up layer and a pooling layer.
5. The smoke and flame detection method of claim 2, wherein the feed forward prediction network comprises a second convolutional layer, a hidden layer, and a linear prediction layer; wherein the content of the first and second substances,
the second convolution layer is used for performing feature extraction on the target feature through a convolution kernel of 3 x 3;
the hidden layer comprises three full-connection layers, wherein the first full-connection layer and the second full-connection layer are respectively connected with a connection activation function.
6. A smoke and flame detection method according to any of claims 1 to 5 wherein the step of pre-processing the marked image data comprises:
performing data enhancement on the marked image data through geometric transformation;
performing blending processing on the marked image data of different target objects;
and performing mixing processing on the marked image data of different image backgrounds.
7. A smoke and flame detection method according to any of claims 1 to 5 wherein the detection results include target class, target presence probability and target coordinates.
8. A smoke and flame detection device, comprising:
the system comprises a preprocessing module, a data processing module and a data processing module, wherein the preprocessing module is used for preprocessing marked image data of a marked target object and establishing a training data set by using the preprocessed marked image data, and the target object comprises smoke and flame;
the training module is used for training an initial neural network model constructed based on a convolutional neural network and a multi-head attention mechanism through the training data set to obtain a target neural network model;
and the detection module is used for acquiring image data to be detected of an environment to be detected, inputting the image data to be detected into the target neural network model and obtaining a detection result output by the target neural network model.
9. A smoke and flame detection apparatus, characterized in that it comprises: a memory, a processor and a smoke and flame detection program stored on the memory and executable on the processor, the smoke and flame detection program being configured to implement the steps of the smoke and flame detection method of any one of claims 1 to 7.
10. A computer readable storage medium having stored thereon a smoke and flame detection program which when executed by a processor implements the steps of the smoke and flame detection method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211430723.6A CN115761332A (en) | 2022-11-14 | 2022-11-14 | Smoke and flame detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211430723.6A CN115761332A (en) | 2022-11-14 | 2022-11-14 | Smoke and flame detection method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115761332A true CN115761332A (en) | 2023-03-07 |
Family
ID=85371474
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211430723.6A Pending CN115761332A (en) | 2022-11-14 | 2022-11-14 | Smoke and flame detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115761332A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116503715A (en) * | 2023-06-12 | 2023-07-28 | 南京信息工程大学 | Forest fire detection method based on cascade network |
-
2022
- 2022-11-14 CN CN202211430723.6A patent/CN115761332A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116503715A (en) * | 2023-06-12 | 2023-07-28 | 南京信息工程大学 | Forest fire detection method based on cascade network |
| CN116503715B (en) * | 2023-06-12 | 2024-01-23 | 南京信息工程大学 | Forest fire detection method based on cascade network |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107529650B (en) | Closed loop detection method and device and computer equipment | |
| CN109522945B (en) | Group emotion recognition method and device, intelligent device and storage medium | |
| CN111259940B (en) | Target detection method based on space attention map | |
| CN111814902A (en) | Target detection model training method, target identification method, device and medium | |
| CN112801146B (en) | Target detection method and system | |
| CN110929080B (en) | Optical remote sensing image retrieval method based on attention and generation countermeasure network | |
| CN115129848B (en) | Method, device, equipment and medium for processing visual question-answering task | |
| CN106650615B (en) | A kind of image processing method and terminal | |
| CN107784288A (en) | A kind of iteration positioning formula method for detecting human face based on deep neural network | |
| US20230401833A1 (en) | Method, computer device, and storage medium, for feature fusion model training and sample retrieval | |
| CN113487610B (en) | Herpes image recognition method and device, computer equipment and storage medium | |
| CN113628059A (en) | Associated user identification method and device based on multilayer graph attention network | |
| CN112784066A (en) | Information feedback method, device, terminal and storage medium based on knowledge graph | |
| CN111539456B (en) | Target identification method and device | |
| CN115761332A (en) | Smoke and flame detection method, device, equipment and storage medium | |
| CN110414516B (en) | Single Chinese character recognition method based on deep learning | |
| WO2022063076A1 (en) | Adversarial example identification method and apparatus | |
| CN111079930A (en) | Method and device for determining quality parameters of data set and electronic equipment | |
| CN114066899A (en) | Image segmentation model training method, image segmentation device, image segmentation equipment and image segmentation medium | |
| CN111476144B (en) | Pedestrian attribute identification model determining method and device and computer readable storage medium | |
| CN112699809B (en) | Vaccinia category identification method, device, computer equipment and storage medium | |
| CN115760775A (en) | Tomato disease detection method, device, equipment and storage medium | |
| CN115620083A (en) | Model training method, face image quality evaluation method, device and medium | |
| CN114120050A (en) | Method, device and equipment for extracting surface ecological data and storage medium | |
| CN117574314B (en) | Information fusion method, device and equipment of sensor and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |