CN114972732A - Smoke and fire detection method, device, equipment and computer readable storage medium - Google Patents
Smoke and fire detection method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN114972732A CN114972732A CN202210534499.9A CN202210534499A CN114972732A CN 114972732 A CN114972732 A CN 114972732A CN 202210534499 A CN202210534499 A CN 202210534499A CN 114972732 A CN114972732 A CN 114972732A
- Authority
- CN
- China
- Prior art keywords
- image
- detected
- smoke
- prediction frame
- convolution
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Fire-Detection Mechanisms (AREA)
Abstract
The application relates to a smoke and fire detection method, a smoke and fire detection device, smoke and fire detection equipment and a computer readable storage medium, belongs to the technical field of image recognition, and comprises the steps of obtaining an image to be detected; inputting the image to be detected into a target detection model, and outputting the prediction frame information of the image to be detected based on the target detection model; the target detection model comprises a common convolution module, a variable convolution module, a feature fusion module and a post-processing module, and the prediction frame information comprises a target category, a confidence coefficient and position information of the prediction frame in the image to be detected; marking the smoke and fire area in the image to be detected according to the prediction frame information to obtain a marked image; the variable convolution module is utilized in the target detection model, so that the target detection model can conveniently train a convolution kernel form matched with the actual shape of smoke and fire, and the smoke and fire can be more accurately detected; this application has the effect of being convenient for detect firework target.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a smoke and fire detection method, a smoke and fire detection device, smoke and fire detection equipment and a computer readable storage medium.
Background
The firework detection is mainly used for detecting fireworks and preventing fire, so that people can know fire information in time, and the fire is controlled and processed in time. At present, most of video-based smoke and fire detection methods firstly adopt an interframe difference method, a background modeling method and the like to detect a candidate motion area, then extract characteristics such as color, texture and the like from the candidate motion area, and judge whether the candidate motion area is smoke and fire by using a classifier. The target detection is image segmentation based on target geometry and statistical characteristics, the accuracy and the real-time performance of the target detection are important capabilities of the whole system, and especially in a complex scene, when a plurality of targets need to be processed in real time, the automatic extraction and identification of the targets are very important; if the target detection method is applied to firework detection, the target detection method is often utilized to achieve a good detection effect in a complex scene.
In the process of implementing the present application, the inventors found that at least the following problems exist: target detection usually depends on a bounding box based on feature extraction, and when the target detection is adopted to detect smoke and fire, the method can not well adapt to the actual form of the smoke and fire, so that the detection effect of the target detection method on irregular targets such as the smoke and fire is not ideal.
Disclosure of Invention
To facilitate detection of a pyrotechnic object, a pyrotechnic detection method, apparatus, device, and computer-readable storage medium are provided.
In a first aspect, the application provides a smoke and fire detection method, which adopts the following technical scheme:
a method of smoke detection, comprising,
acquiring an image to be detected;
inputting the image to be detected into a target detection model to generate prediction frame information of the image to be detected; the target detection model comprises a common convolution module, a variable convolution module, a feature fusion module and a post-processing module;
and marking the smoke and fire area in the image to be detected according to the prediction frame information to obtain a marked image.
By adopting the technical scheme, the acquired image to be detected is input into the target detection model, the target detection model outputs the prediction frame information, the variable convolution module is applied to the target detection model, when the target detection model is trained, a dynamic convolution kernel can be trained, the form of the convolution kernel is not limited to a rectangle, namely, the detection range of the target detection model can be more fit with the actual shape of smoke and fire, the smoke and fire area in the image to be detected is marked according to the prediction frame information, the marked image is obtained, and the effect of conveniently detecting smoke and fire is realized.
Optionally, the step of marking the firework area in the image to be detected according to the prediction frame information specifically includes:
segmenting the image to be detected into a plurality of sub-images according to the preset segmentation overlapping degree; the preset segmentation overlapping degree is the overlapping degree of adjacent sub-graphs after segmentation;
inputting a plurality of subgraphs into the target detection model respectively to generate a plurality of subgraph prediction frame information;
obtaining the target category and the confidence coefficient of the image to be detected based on the prediction frame information of the image to be detected;
respectively obtaining target categories and confidence degrees corresponding to the multiple subgraphs based on the multiple subgraph prediction frame information;
judging whether the target type of any subgraph is consistent with the target type of the image to be detected, if so, judging whether the confidence coefficient of the subgraph with the consistent target type and the confidence coefficient of the image to be detected are both greater than a preset confidence coefficient, and if so, marking the smoke and fire area in the image to be detected according to the prediction frame information of the image to be detected.
By adopting the technical scheme, the method for segmenting the image to be detected is utilized, the image to be detected has more obvious characteristics after being amplified by the sub-image, so that a more accurate detection result can be obtained in the sub-image detection, whether the target type of the sub-image is the same as the target type of the detected image is judged by judging the target type, if the target type of the sub-image is the same as the target type of the detected image, the confidence coefficient and the preset confidence coefficient of the sub-image with the same target type are compared, and the confidence coefficient and the preset confidence coefficient of the image to be detected are judged, if the confidence coefficient and the preset confidence coefficient are both greater than the preset confidence coefficient, the information of the prediction frame is considered to be accurate, at the moment, the smoke and fire area in the image to be detected is marked according to the corresponding information of the prediction frame, and the effect that the marked smoke and fire area is more accurate is realized.
Optionally, after the step of determining that the confidence of the subgraphs with the same target class and the confidence of the image to be detected are both greater than a preset confidence, the method further includes:
obtaining the region overlapping degree of the prediction frame of the image to be detected and the prediction frame of the subgraph with the same target type based on the prediction frame information of the image to be detected and the subgraph prediction frame information with the same target type;
and judging whether the area overlapping degree is greater than a preset area overlapping degree, if so, marking the smoke and fire area in the image to be detected according to the prediction frame information of the image to be detected.
By adopting the technical scheme, after the reliability is judged, whether the region overlapping degree of the image prediction frame to be detected and the sub-image prediction frame is greater than the preset region overlapping degree is judged, so that whether the frame sides in the image to be detected and the sub-image are in the same region is conveniently detected, if the region overlapping degree is greater than the preset region overlapping degree, the frame sides in the image to be detected and the sub-image are judged to be in the same region, the local region detected in the image to be detected is the same as the region detected in the sub-image, and at the moment, the firework region in the image to be detected is marked, so that the accuracy of the displayed prediction frame information is improved.
Optionally, the common convolution module is configured to perform a preliminary common convolution on the input image to be detected, and output an initial feature map;
the variable convolution module is used for processing the input initial characteristic diagram and outputting a dynamic characteristic diagram;
the characteristic fusion module is used for fusing the initial characteristic diagram and the dynamic characteristic diagram and extracting detection characteristics;
and the post-processing module is used for processing the detection characteristics and outputting the information of the prediction frame after the processing is finished.
By adopting the technical scheme, the initial common convolution processing is carried out on the input image by using the common convolution module to generate an initial characteristic diagram, the initial characteristic diagram is further processed by using the variable convolution module, and the convolution kernel of the deformable convolution can be more close to the target to be detected because the direction vector is added compared with the common convolution kernel, thereby being convenient for generating a dynamic characteristic diagram which is more close to the shape of flame and smoke; and finally, the post-processing module is used for processing the detection characteristics to obtain the information of the prediction frame, so that the detection of flames and smoke with irregular shapes is facilitated.
Optionally, the variable convolution module, in particular comprising,
the channel segmentation layer is used for carrying out channel segmentation on the initial characteristic diagram;
a first convolution layer, which is used for processing the initial characteristic diagram after the channel segmentation by utilizing a first convolution operation and outputting a first characteristic diagram;
the channel attention unit is used for carrying out channel attention mechanism processing on the first characteristic image, carrying out channel fusion and outputting a second characteristic image;
the second convolution layer is used for processing the second characteristic diagram and the initial characteristic diagram after the channel segmentation by utilizing a second convolution operation and outputting a third characteristic diagram;
and the deformable convolution layer is used for performing deformable convolution on the third feature map and outputting the dynamic feature map.
By adopting the technical scheme, the initial characteristic diagram is subjected to channel segmentation, the initial characteristic diagram is segmented into a plurality of channels, the initial characteristic diagram and the convolution kernel are subjected to first convolution operation, so that a first characteristic diagram is obtained, the channel attention unit is utilized to endow the channels in the first characteristic diagram with weight values, the weight values are larger as the importance degree is higher, so that a second characteristic diagram with high importance degree is conveniently extracted, then, second convolution operation is carried out on the second characteristic diagram and the initial characteristic diagram, then, the deformable convolution layer is processed, the form of the convolution kernel is more fit with the shapes of actual flame and smoke through the deformable convolution layer, a dynamic characteristic diagram is obtained, and the effect of detecting irregular flame and smoke is better.
Optionally, the first volume operation specifically includes,
sequentially performing a first depth separable convolution and a second depth separable convolution on the initial feature map;
performing channel fusion on the result of the first depth separable convolution and the result of the second depth separable convolution;
wherein the first depth separable convolution is a 1x1 depth separable convolution and the second depth separable convolution is a 3x3 depth separable convolution, a 5x5 depth separable convolution, or a 7x7 depth separable convolution.
By adopting the technical scheme, the number of channels of the initial feature map is adjusted by utilizing the first depth separable convolution, the number of channels of the initial feature map is adjusted to the number of input channels of the convolution kernel, then the adjusted initial feature map is subjected to the second depth separable convolution, the features in the initial feature map are extracted, the result of the first depth separable convolution and the result of the second depth separable convolution are subjected to channel fusion, and the first feature map is generated after the channels are fused.
Optionally, the second convolution operation specifically includes,
and performing 1x1 depth separable convolution on the second feature map and the initial feature map after channel segmentation, and performing channel scrambling.
Through adopting above-mentioned technical scheme, through the operation of passageway in disorder for the information of different passageways obtains the circulation in the characteristic map, more does benefit to the expression of characteristic, thereby is convenient for follow-up extracting the characteristic, in order to reach better firework detection effect.
In a second aspect, the application provides a smoke and fire detection device, which adopts the following technical scheme:
a smoke and fire detection device comprises an image acquisition unit, a target detection unit and a prediction frame marking unit;
the image acquisition unit is used for acquiring an image to be detected;
the target detection unit inputs the image to be detected into a target detection model to generate prediction frame information;
and the prediction frame marking unit is used for marking the smoke and fire area in the image to be detected according to the prediction frame information to obtain a marked image.
By adopting the technical scheme, the image to be detected is obtained by the image obtaining unit, the image to be detected is input into the target detection model, the prediction frame information is output by the target detection model, and the prediction frame information is marked by the prediction frame marking unit, so that the effect of detecting the firework target is realized.
In a third aspect, the present application provides a computer device, which adopts the following technical solution:
a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor performing a method of smoke detection as described in the first aspect.
In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solutions:
a computer readable storage medium comprising a computer program stored thereon which is loadable by a processor and adapted to carry out any of the methods of the first aspect.
Drawings
FIG. 1 is a schematic flow chart of a smoke and fire detection method according to an embodiment of the present application.
FIG. 2 is a first flowchart of an embodiment of the present application for displaying tag prediction box information.
FIG. 3 is a diagram of a second process for presenting tagged prediction box information in one implementation of the present application.
FIG. 4 is a block diagram of a target detection model according to an embodiment of the present disclosure.
Fig. 5 is a block diagram of a variable convolution module according to an embodiment of the present application.
FIG. 6 is a block flow diagram of a variable convolution module according to an embodiment of the present application.
Fig. 7 is a block diagram of a detection apparatus according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is further described in detail below with reference to fig. 1-7 and the embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The embodiment of the application discloses a firework detection method.
Referring to fig. 1, a method of smoke detection comprising:
step S101, obtaining an image to be detected;
the image to be detected can be acquired from the image acquisition equipment in real time or can be acquired from a storage module of the image acquisition equipment; the image acquisition apparatus may be installed in an outdoor environment where fire frequently occurs, or may be installed indoors, for example, in residential houses, shopping malls, farms, and the like.
Step S102, inputting an image to be detected into a target detection model, and generating prediction frame information of the image to be detected;
the target detection model comprises a common convolution module, a variable convolution module, a feature fusion module and a post-processing module; the prediction frame information includes an object class, a confidence level, and position information of the prediction frame, and zero, one, or more pieces of prediction frame information may be included in the same image to be detected.
Specifically, the position information of the prediction frame may be coordinates (a, b) of the upper left corner of the frame and coordinates (c, d) of the lower right corner of the frame, and since the frame is a rectangular frame, the coordinates (a, d) of the lower left corner of the frame and the coordinates (c, b) of the upper right corner of the frame can be obtained, and the frame range of the prediction frame can be accurately obtained from the four coordinates; the position information of the prediction frame may be coordinates (x, y) of the center of the prediction frame, the width w of the frame side, and the height h of the frame side, and the frame side range of the prediction frame can be accurately obtained in the same manner.
And step S103, marking the firework area in the image to be detected according to the prediction frame information to obtain a marked image.
The position of the prediction frame in the image to be detected is determined according to the prediction frame position information of the prediction frame information, and the target type of the prediction frame information represents the type of the detection target in the prediction frame range, such as: smoke, flame, etc., the confidence of the prediction box information indicates the confidence that the detection target is included in the prediction box, i.e., the accuracy of the prediction box.
In the above embodiment, the deformable convolution is applied to the target detection model, and compared with the ordinary convolution, the direction vector for adjusting the convolution kernel is added, and when the target detection model is trained, a dynamic convolution kernel can be trained, but the ordinary convolution kernel is usually of a fixed size and a fixed size, for example, 3x3, 5x5, and the like, that is, the ordinary convolution kernel is a regular rectangle, and when the region to be detected is an irregular shape, adaptive change cannot be made, so that the actual shapes of smoke and flame can be better fitted by using the convolution kernel of the deformable convolution kernel, and a more accurate effect of smoke and fire detection is realized.
It should be noted that the detection target of the target detection model is not limited to the detection of smoke and flame, and the target detection model is trained by using different training sets of sample images, and can also realize the identification of other targets; when smoke and fire are detected, the training set of the target detection model needs to contain flame and smoke, and the trained target detection model has the effect of detecting the smoke and the flame; similarly, if the training set of the sample image includes the cloud, the trained target detection model has the effect of detecting the cloud.
Referring to fig. 2, the step of marking the smoke and fire area in the image to be detected according to the prediction frame information specifically includes:
step S201, segmenting an image to be detected into a plurality of sub-images according to preset segmentation overlapping degree;
the preset segmentation overlapping degree is the overlapping degree of adjacent sub-graphs after segmentation; specifically, the number of the sub-graphs to be cut is determined by the size of the original graph and the actual deployment condition, and the sub-graphs can be cut into 4 sub-graphs, or 9 sub-graphs or other numbers; the segmentation overlap may be preset to 0.2 or 0.3 or other values, depending on the actual situation.
The sub-graphs are segmented according to the preset segmentation overlapping degree, so that a complete detection target is prevented from being segmented on two or more sub-graphs to a certain extent, and the possibility of retaining the complete detection target is increased by means of overlapping segmentation of adjacent sub-graphs.
Step S202, inputting a plurality of subgraphs into the target detection model respectively, and generating a plurality of subgraph prediction frame information.
The subgraph prediction frame information comprises a target class, confidence and position information of a prediction frame, and a plurality of prediction frames may exist in each subgraph.
And step S203, acquiring the target category and the confidence coefficient of the image to be detected based on the prediction frame information of the image to be detected.
And step S204, respectively obtaining target categories and confidence degrees corresponding to the multiple subgraphs based on the multiple subgraph prediction frame information.
Step S205, judging whether the target type of any subgraph is consistent with the target type of the image to be detected, if so, executing step 206, and if not, not executing operation;
step S206, determining whether the confidence of the subgraph with the same target class and the confidence of the image to be detected are both greater than a preset confidence, if yes, executing step 207, and if not, not executing the operation.
The preset confidence may be set to 0.4 or 0.5 or other values, if the preset confidence is too small, irrelevant prediction frame information may not be effectively filtered, and if the preset confidence is too large, the possibility of missing detection of the detection target may occur, so that the preset confidence needs to be determined according to the actual deployment situation and the historical experience of the staff.
And step S207, marking the firework area in the image to be detected according to the prediction frame information of the image to be detected.
In the embodiment, the image to be detected is segmented, and has more obvious characteristics after being amplified by the sub-image, so that a more accurate detection result can be obtained in the sub-image detection, the sub-image prediction frame information is compared with the prediction frame information of the image to be detected, whether the target type of the sub-image is the same as the target type of the image to be detected is judged by judging the target type, and if the target type of the sub-image is the same as the target type of the image to be detected, the detection of the target type is accurate; and comparing the confidence coefficient of the subgraph and the confidence coefficient of the image to be detected with a preset confidence coefficient, if the confidence coefficient is greater than the preset confidence coefficient, the detection of the confidence coefficient is accurate, and marking the corresponding firework area in the image to be detected according to the prediction frame information of the image to be detected, so that the accuracy of marking the prediction frame information in the image to be detected is improved.
Referring to fig. 3, the step of marking the smoke and fire area in the image to be detected according to the prediction box information further includes:
step S301, obtaining the region overlapping degree of the prediction frame of the image to be detected and the prediction frame of the sub-image with the consistent target type based on the prediction frame information of the image to be detected and the sub-image prediction frame information with the consistent target type.
Wherein the region overlap degree IOU = () () # U (); () is the prediction frame area of the image to be detected, and () is the prediction frame area with consistent target types.
Step S302, determining whether the region overlapping degree is greater than a preset region overlapping degree, if so, performing the step S303, and if not, not performing the operation.
And step S303, marking the firework area in the image to be detected according to the prediction frame information of the image to be detected.
The region overlapping degree is set to be 0.5 or 0.6 or other values, the region overlapping degree is set according to historical experience of workers, the larger the region overlapping degree is, the higher the overlapping degree of the sub-image prediction frame and the image prediction frame to be detected is, namely, the larger the partial area of the same region of the sub-image prediction frame and the prediction frame of the image to be detected is.
In the above embodiment, after the confidence level of the image to be detected and the confidence level of the sub-image are determined, it is determined whether the region overlapping degree of the prediction frame of the image to be detected and the region overlapping degree of the prediction frame of the sub-image are greater than the preset region overlapping degree, so that it is convenient to detect whether the frame sides in the image to be detected and the sub-image are the same region, if the region overlapping degree is greater than the preset region overlapping degree, it is determined that the frame sides in the image to be detected and the sub-image are the same region, it is indicated that the detected local region in the image to be detected and the detected region in the sub-image are the same, and at this time, the image to be detected is marked according to the prediction frame information, so that the accuracy of marking the prediction frame information is improved.
It should be noted that, if the confidence degrees of the image to be detected and the sub-image are both greater than the preset confidence degree, the information of the prediction frame is labeled, and it may occur that the detection region of the image to be detected and the detection region of the sub-image are not the same region, for example, the prediction frame of the image to be detected is in the upper left corner, and the prediction frame of the sub-image is in the lower right corner, at this time, although the confidence degrees of the image to be detected and the sub-image are both greater than the preset confidence degree, the detection regions are different, and the problem that the information of the prediction frame is not accurate enough is still determined only by the confidence degrees; and continuously judging the region overlapping degree, ensuring that the prediction frames of the sub-image and the image to be detected are the same region under the condition that the confidence coefficient meets the condition, and marking the prediction frame information on the image to be detected, thereby avoiding the occurrence of different regions detected by the prediction frames under the condition that the target category and the confidence coefficient meet the condition to a certain extent.
Referring to fig. 4, as a further embodiment of the target detection model, a general convolution module is configured to perform a preliminary general convolution on an input image to be detected and output an initial feature map.
And the variable convolution module is used for processing the input initial characteristic diagram and outputting a dynamic characteristic diagram.
And the feature fusion module is used for fusing the initial feature map and the dynamic feature map and extracting detection features.
And the post-processing module is used for processing the detection characteristics and outputting the information of the prediction frame after the processing is finished.
In the above embodiment, the general convolution module is used to perform the preliminary general convolution processing on the input image to generate the initial characteristic diagram, and the variable convolution module is used to further process the initial characteristic diagram; and finally, the post-processing module is used for processing the detection characteristics to obtain the information of the prediction frame, so that the detection of flames and smoke with irregular shapes is facilitated.
Referring to fig. 5, as an embodiment of the variable convolution processing layer, the variable convolution module, specifically including,
and the channel segmentation layer is used for carrying out channel segmentation on the initial characteristic diagram.
And processing the initial characteristic diagram after the channel segmentation by using a first convolution operation and outputting a first characteristic diagram.
And the channel attention unit is used for carrying out channel attention mechanism processing on the first characteristic image, carrying out channel fusion and outputting a second characteristic image.
And the second convolution layer processes the second characteristic diagram and the initial characteristic diagram after the channel segmentation by utilizing a second convolution operation and outputs a third characteristic diagram.
And the deformable convolution layer is used for performing deformable convolution on the third feature map and outputting the dynamic feature map.
In the first convolution operation, the second convolution operation and the deformable convolution operation, the number of channels corresponding to the feature diagram is required to be equal to the number of input channels of the convolution kernel.
In the above embodiment, the channel segmentation is performed on the initial feature map, the initial feature map is segmented into a plurality of channels, and the first convolution operation is performed on the initial feature map and the convolution kernel, so as to obtain a first feature map, the channel attention unit is used to assign weighted values to the plurality of channels in the first feature map, the weighted values are larger as the importance degree is higher, so as to extract a second feature map with a high importance degree, the second convolution operation is performed on the second feature map and the initial feature map, and then the deformable convolution layer is processed, so that the form of the convolution kernel is more fit with the shape of the actual flame and smoke through the deformable convolution layer, so as to obtain a dynamic feature map, which is a feature map with an area having an irregular shape, and thus the effect of detecting the irregular flame and smoke is better.
Referring to fig. 5 and 6, as an embodiment of the first volume operation, specifically including,
the initial feature map is sequentially subjected to a first depth separable convolution and a second depth separable convolution.
Channel merging the result of the first depth separable convolution with the result of the second depth separable convolution.
Wherein the first depth separable convolution is a 1x1 depth separable convolution and the second depth separable convolution is a 3x3 depth separable convolution, a 5x5 depth separable convolution, or a 7x7 depth separable convolution.
In the above embodiment, the number of channels of the initial feature map is adjusted by 1 × 1 depth separable convolution, the number of channels of the initial feature map is adjusted to the number of input channels of the convolution kernel, and then the adjusted initial feature map is subjected to 3x3 depth separable convolution processing, 5x5 depth separable convolution processing, or 7x7 depth separable convolution processing, so as to extract features in the initial feature map, and the first feature map is generated after the channels are fused.
Referring to fig. 5 and 6, as an embodiment of the second convolution operation, specifically including,
and performing 1x1 depth separable convolution on the second feature map and the initial feature map after channel segmentation, and performing channel scrambling.
Among the above-mentioned embodiment, through the operation of channel in disorder for the information of different channels obtains the circulation in the characteristic map, more does benefit to the expression of characteristic, thereby is convenient for follow-up to the characteristic extract, in order to reach better firework detection effect.
The embodiment of the application also discloses a firework detection device.
Referring to fig. 7, a smoke and fire detecting apparatus includes an image acquiring unit, an object detecting unit, and a prediction frame marking unit;
the image acquisition unit is used for acquiring an image to be detected;
the target detection unit inputs the image to be detected into the target detection model to generate prediction frame information;
and the prediction frame marking unit is used for marking the firework area in the image to be detected according to the prediction frame information to obtain a marked image.
In the above embodiment, the image to be detected is acquired by the image acquisition unit, the image to be detected is input into the target detection model, the target detection model outputs the prediction frame information, and the prediction frame marking unit marks the prediction frame information on the image to be detected, so that the effect of detecting the smoke and fire target is realized.
The smoke and fire detection device provided by the embodiment of the invention can realize any method of the detection method, and the specific working process of the smoke and fire detection device can refer to the corresponding process in the detection method embodiment.
In the several embodiments provided by the present invention, it should be understood that the provided method and apparatus can be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative; for example, a module may be divided into only one logical function, and another division may be implemented in practice, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the connections or communication connections shown or discussed above may be indirect couplings or communication connections through some interfaces, devices or units, and may also be electrical, mechanical or other connections.
The embodiment of the application also discloses computer equipment.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor performing a method of smoke detection as described above.
The embodiment of the application also discloses a computer readable storage medium.
A computer readable storage medium comprising a computer program stored thereon which is loadable by a processor and adapted to carry out any of the methods of the first aspect.
Wherein the computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device; program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
It should be noted that, in the foregoing embodiments, descriptions of the respective embodiments have respective emphasis, and reference may be made to relevant descriptions of other embodiments for parts that are not described in detail in a certain embodiment.
The foregoing is a preferred embodiment of the present application and is not intended to limit the scope of the application in any way, and any features disclosed in this specification (including the abstract and drawings) may be replaced by alternative features serving equivalent or similar purposes, unless expressly stated otherwise. That is, unless expressly stated otherwise, each feature is only an example of a generic series of equivalent or similar features.
Claims (10)
1. A method of smoke detection, comprising:
acquiring an image to be detected;
inputting the image to be detected into a target detection model to generate prediction frame information of the image to be detected; the target detection model comprises a common convolution module, a variable convolution module, a feature fusion module and a post-processing module;
and marking the smoke and fire area in the image to be detected according to the prediction frame information to obtain a marked image.
2. A smoke and fire detection method according to claim 1, wherein said step of marking the smoke and fire area in the image to be detected according to the prediction frame information comprises:
segmenting the image to be detected into a plurality of sub-images according to the preset segmentation overlapping degree; the preset segmentation overlapping degree is the overlapping degree of adjacent sub-graphs after segmentation;
inputting a plurality of subgraphs into the target detection model respectively to generate a plurality of subgraph prediction frame information;
obtaining the target category and the confidence coefficient of the image to be detected based on the prediction frame information of the image to be detected;
respectively obtaining target categories and confidence degrees corresponding to the multiple subgraphs based on the multiple subgraph prediction frame information;
judging whether the target type of any subgraph is consistent with the target type of the image to be detected, if so, judging whether the confidence coefficient of the subgraph with the consistent target type and the confidence coefficient of the image to be detected are both greater than a preset confidence coefficient, and if so, marking the smoke and fire area in the image to be detected according to the prediction frame information of the image to be detected.
3. A smoke and fire detection method according to claim 2, wherein said step of marking the smoke and fire area in the image to be detected according to the prediction frame information of the image to be detected further comprises:
obtaining the region overlapping degree of the prediction frame of the image to be detected and the prediction frame of the subgraph with the same target type based on the prediction frame information of the image to be detected and the subgraph prediction frame information with the same target type;
and judging whether the area overlapping degree is greater than a preset area overlapping degree, if so, marking the smoke and fire area in the image to be detected according to the prediction frame information of the image to be detected.
4. A smoke and fire detection method according to claim 1, wherein:
the common convolution module is used for carrying out primary common convolution on the input image to be detected and outputting an initial characteristic diagram;
the variable convolution module is used for processing the input initial characteristic diagram and outputting a dynamic characteristic diagram;
the characteristic fusion module is used for fusing the initial characteristic diagram and the dynamic characteristic diagram and extracting detection characteristics;
and the post-processing module is used for processing the detection characteristics and outputting the information of the prediction frame after the processing is finished.
5. A smoke and fire detection method according to claim 4, characterized in that: the variable convolution module, in particular comprising,
the channel segmentation layer is used for carrying out channel segmentation on the initial characteristic diagram;
the first convolution layer is used for processing the initial characteristic diagram after the channel is segmented by utilizing a first convolution operation and outputting a first characteristic diagram;
the channel attention unit is used for carrying out channel attention mechanism processing on the first characteristic image, carrying out channel fusion and outputting a second characteristic image;
the second convolution layer is used for processing the second characteristic diagram and the initial characteristic diagram after the channel segmentation by utilizing a second convolution operation and outputting a third characteristic diagram;
and the deformable convolution layer is used for performing deformable convolution on the third feature map and outputting the dynamic feature map.
6. A method for fire and smoke detection according to claim 5, characterized in that said first winding operation comprises in particular:
sequentially performing a first depth separable convolution and a second depth separable convolution on the initial feature map;
performing channel fusion on the result of the first depth separable convolution and the result of the second depth separable convolution;
wherein the first depth separable convolution is a 1x1 depth separable convolution and the second depth separable convolution is a 3x3 depth separable convolution, a 5x5 depth separable convolution, or a 7x7 depth separable convolution.
7. A method for fire and smoke detection according to claim 5, characterized in that said second convolution operation comprises in particular:
and performing 1x1 depth separable convolution on the second feature map and the initial feature map after channel segmentation, and performing channel scrambling.
8. A smoke and fire detection device characterized by: the device comprises an image acquisition unit, a target detection unit and a prediction frame marking unit;
the image acquisition unit is used for acquiring an image to be detected;
the target detection unit inputs the image to be detected into a target detection model to generate prediction frame information;
and the prediction frame marking unit is used for marking the smoke and fire area in the image to be detected according to the prediction frame information to obtain a marked image.
9. A computer device, characterized by: comprising a memory, a processor and a computer program stored on said memory and executable on said processor, said processor performing a method of smoke detection according to any one of claims 1-7.
10. A computer-readable storage medium characterized by: a computer program which can be loaded by a processor and which performs the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210534499.9A CN114972732A (en) | 2022-05-17 | 2022-05-17 | Smoke and fire detection method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210534499.9A CN114972732A (en) | 2022-05-17 | 2022-05-17 | Smoke and fire detection method, device, equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114972732A true CN114972732A (en) | 2022-08-30 |
Family
ID=82984223
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210534499.9A Pending CN114972732A (en) | 2022-05-17 | 2022-05-17 | Smoke and fire detection method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114972732A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116977634A (en) * | 2023-07-17 | 2023-10-31 | 应急管理部沈阳消防研究所 | Fire smoke detection method based on laser radar point cloud background subtraction |
-
2022
- 2022-05-17 CN CN202210534499.9A patent/CN114972732A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116977634A (en) * | 2023-07-17 | 2023-10-31 | 应急管理部沈阳消防研究所 | Fire smoke detection method based on laser radar point cloud background subtraction |
| CN116977634B (en) * | 2023-07-17 | 2024-01-23 | 应急管理部沈阳消防研究所 | Fire smoke detection method based on laser radar point cloud background subtraction |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109886130B (en) | Target object determination method and device, storage medium and processor | |
| US10762376B2 (en) | Method and apparatus for detecting text | |
| CN109829398B (en) | Target detection method in video based on three-dimensional convolution network | |
| CN113658132B (en) | Computer vision-based structural part weld joint detection method | |
| KR102406804B1 (en) | Method and system for automatically detecting objects in image based on deep learning | |
| US20210174539A1 (en) | A method for estimating the pose of a camera in the frame of reference of a three-dimensional scene, device, augmented reality system and computer program therefor | |
| CN108416902A (en) | Real-time object identification method based on difference identification and device | |
| CN111222478A (en) | Construction site safety protection detection method and system | |
| CN112528974B (en) | Distance measuring method and device, electronic equipment and readable storage medium | |
| EP3493157B1 (en) | Method and system for visual change detection using multi-scale analysis | |
| CN111738206A (en) | Excavator detection method for unmanned aerial vehicle inspection based on CenterNet | |
| CN115908988B (en) | Defect detection model generation method, device, equipment and storage medium | |
| CN111368682A (en) | Method and system for detecting and identifying station caption based on faster RCNN | |
| CN111652181B (en) | Target tracking method and device and electronic equipment | |
| CN113283408A (en) | Monitoring video-based social distance monitoring method, device, equipment and medium | |
| CN112200056A (en) | Face living body detection method and device, electronic equipment and storage medium | |
| CN113326773A (en) | Recognition model training method, recognition method, device, equipment and storage medium | |
| CN114399734A (en) | Forest fire early warning method based on visual information | |
| CN114972732A (en) | Smoke and fire detection method, device, equipment and computer readable storage medium | |
| CN114419739A (en) | Training method of behavior recognition model, behavior recognition method and equipment | |
| CN115205793B (en) | Electric power machine room smoke detection method and device based on deep learning secondary confirmation | |
| KR101592087B1 (en) | Method for generating saliency map based background location and medium for recording the same | |
| CN113538351B (en) | Method for evaluating defect degree of external insulation equipment by fusing multiparameter electric signals | |
| KR102597692B1 (en) | Method, apparatus, and computer program for measuring volume of objects by using image | |
| JP2019066909A (en) | Object distribution estimation apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |