CN114170575A - Flame identification method and device, electronic equipment and storage medium - Google Patents
Flame identification method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114170575A CN114170575A CN202210126854.9A CN202210126854A CN114170575A CN 114170575 A CN114170575 A CN 114170575A CN 202210126854 A CN202210126854 A CN 202210126854A CN 114170575 A CN114170575 A CN 114170575A
- Authority
- CN
- China
- Prior art keywords
- feature
- fusion
- prediction
- image
- target image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
Abstract
The embodiment of the application discloses a flame identification method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence. Wherein, the method comprises the following steps: acquiring an image to be recognized, and performing data enhancement processing on the image to be recognized to obtain a target image; inputting the target image into a feature extraction model for feature extraction to obtain a first feature, a second feature and a first fusion feature of the target image; the first feature is subjected to transmission fusion through the first fusion feature to obtain a second fusion feature, and the second feature is subjected to transmission fusion through the second fusion feature to obtain a third fusion feature; respectively performing feature prediction processing on the first fusion feature, the second fusion feature and the third fusion feature to obtain a plurality of prediction frames and attribute information of each prediction frame; and determining whether the image to be identified has flames or not according to the plurality of prediction boxes and the attribute information of each prediction box. This application technical scheme can satisfy the demand to the quick discernment of flame, can also improve discernment rate of accuracy.
Description
Technical Field
The embodiment of the application relates to the technical field of artificial intelligence, in particular to a flame identification method and device, electronic equipment and a storage medium.
Background
With the increasing perfection of security protection, the importance of gas gun flame identification and detection in factories is highlighted. The ubiquitous monitoring also brings huge amounts of information. In the face of a long-term massive monitoring picture, the air gun flame monitoring is carried out by using manpower at every moment, so that manpower and material resources are consumed, and meanwhile due to the limitation of human energy, timely reaction to danger at every moment cannot be guaranteed. Therefore, a flame identification method needs to be designed to meet the requirement of rapid and accurate identification and detection of flames.
Disclosure of Invention
The embodiment of the application provides a flame identification method, a flame identification device, electronic equipment and a storage medium, which can meet the requirements of quickly identifying and detecting flame, can also improve the identification accuracy and save the cost of manpower and material resources.
In a first aspect, an embodiment of the present application provides a flame identification method, including:
acquiring an image to be identified, and performing data enhancement processing on the image to be identified to obtain a target image;
inputting the target image into a feature extraction model for feature extraction to obtain a first feature, a second feature and a first fusion feature of the target image;
performing transfer fusion on the first feature through the first fusion feature to obtain a second fusion feature, and performing transfer fusion on the second feature through the second fusion feature to obtain a third fusion feature;
respectively performing feature prediction processing on the first fusion feature, the second fusion feature and the third fusion feature to obtain a plurality of prediction frames and attribute information of each prediction frame;
and determining whether the image to be identified has flames or not according to the plurality of prediction boxes and the attribute information of each prediction box.
In a second aspect, embodiments of the present application provide a flame identification device, including:
the target image determining module is used for acquiring an image to be identified and performing data enhancement processing on the image to be identified to obtain a target image;
the first feature determination module is used for inputting the target image into a feature extraction model for feature extraction to obtain a first feature, a second feature and a first fusion feature of the target image;
the second feature determination module is used for performing transfer fusion on the first feature through the first fusion feature to obtain a second fusion feature, and performing transfer fusion on the second feature through the second fusion feature to obtain a third fusion feature;
the prediction frame determining module is used for respectively performing feature prediction processing on the first fusion feature, the second fusion feature and the third fusion feature to obtain a plurality of prediction frames and attribute information of each prediction frame;
and the flame identification module is used for determining whether the image to be identified has flames or not according to the plurality of prediction boxes and the attribute information of each prediction box.
In a third aspect, an embodiment of the present application provides an electronic device, including:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the method of flame identification as described in any of the embodiments of the present application.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium on which a computer program is stored, where the computer program, when executed by a processor, implements a flame identification method as described in any of the embodiments of the present application.
The embodiment of the application provides a flame identification method, a flame identification device, electronic equipment and a storage medium, wherein the method comprises the following steps: acquiring an image to be recognized, and performing data enhancement processing on the image to be recognized to obtain a target image; inputting the target image into a feature extraction model for feature extraction to obtain a first feature, a second feature and a first fusion feature of the target image; the first feature is subjected to transmission fusion through the first fusion feature to obtain a second fusion feature, and the second feature is subjected to transmission fusion through the second fusion feature to obtain a third fusion feature; respectively performing feature prediction processing on the first fusion feature, the second fusion feature and the third fusion feature to obtain a plurality of prediction frames and attribute information of each prediction frame; and determining whether the image to be identified has flames or not according to the plurality of prediction boxes and the attribute information of each prediction box. Whether this application confirms to treat the discernment image and have flame through adopting to treat the discernment image and carry out image enhancement, feature extraction, feature fusion and feature prediction, has solved prior art and need adopt artifical whenever to carry out the problem that air gun flame monitored, can satisfy the demand to the quick discernment of flame and detection, can also improve the discernment rate of accuracy, the cost of the material resources of using manpower sparingly.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present application, nor do they limit the scope of the present application. Other features of the present application will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
fig. 1 is a first schematic flow chart of a flame identification method according to an embodiment of the present disclosure;
FIG. 2 is a schematic illustration of feature delivery fusion provided by an embodiment of the present application;
FIG. 3 is a second flowchart of a method for identifying flames according to an embodiment of the present disclosure;
FIG. 4 is a schematic structural diagram of a flame identification device according to an embodiment of the present disclosure;
FIG. 5 is a block diagram of an electronic device for implementing a method of flame identification according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Before the embodiments of the present application are described, a brief description will be given of the algorithm used in the flame identification method of the present application. The application uses a Yolo-x algorithm, which comprises four parts: network input end, backbone network, hack structure and network output layer.
Example one
Fig. 1 is a first flowchart of a flame identification method according to an embodiment of the present disclosure, and fig. 2 is a schematic diagram of feature transfer fusion according to an embodiment of the present disclosure. The embodiment can be applied to identifying the flame of the air gun in the production process of a factory, namely identifying whether the image to be identified has the flame. The flame identification method provided by the embodiment of the present application can be performed by the flame identification device provided by the embodiment of the present application, and the device can be implemented by software and/or hardware and integrated in an electronic device for performing the method.
Referring to fig. 1, the method of the present embodiment includes, but is not limited to, the following steps:
and S110, acquiring an image to be recognized, and performing data enhancement processing on the image to be recognized to obtain a target image.
In the embodiment of the present application, the image to be recognized refers to an image acquired from an appliance generating flames. And acquiring an image to be identified of the flame appliance through data acquisition equipment. And after the image to be recognized is acquired, performing data enhancement processing on the image to be recognized to obtain a target image.
Specifically, the specific process of performing data enhancement processing on the image to be recognized is as follows: splicing the images to be recognized for multiple times to obtain a first intermediate image and a second intermediate image; splicing comprises at least one of random scaling, random clipping and random arrangement; filling left and right edges of the first intermediate image to obtain a first candidate image; filling upper and lower edges of the second intermediate image to obtain a second candidate image; and performing weighted fusion on the first candidate image and the second candidate image to obtain a target image.
In the embodiment of the present application, this step is a network input end. After the image to be recognized is obtained, the Mosaic algorithm is adopted to carry out random scaling, random cutting and random arrangement on the image to be recognized, so that multiple splicing is realized, the data of the image to be recognized is enhanced, and a first intermediate image and a second intermediate image are obtained. In addition, a MixUp algorithm is adopted on the basis of the Mosaic algorithm, and left and right edges of the first intermediate image are filled to obtain a first candidate image; filling upper and lower edges of the second intermediate image to obtain a second candidate image; and setting a fusion coefficient, and performing weighted fusion on the first candidate image and the second candidate image to obtain a target image.
And S120, inputting the target image into the feature extraction model for feature extraction to obtain a first feature, a second feature and a first fusion feature of the target image.
In the embodiment of the present application, the step is a backbone network. The feature extraction model may be a backbone network of Yolox-Darknet53, the backbone network of Yolox-Darknet53 is substantially identical to the backbone network of Yolov3 baseline, but a Spatial Pyramid Pool (SPP) layer is added, and the original training strategy is slightly adjusted.
Specifically, inputting a target image into a feature extraction model for feature extraction to obtain a first feature, a second feature and a first fusion feature of the target image, including: performing feature extraction on the target image through a first convolution neural network in the feature extraction model to obtain a first feature of the target image; performing feature extraction on the target image through a second convolutional neural network in the feature extraction model to obtain second features of the target image; and performing feature fusion on the second features through a feature fusion network in the feature extraction model to obtain first fusion features of the target image.
The feature extraction model comprises five layers of convolutional neural networks and an SPP layer, the first convolutional neural network refers to the convolutional neural network of the first three layers, the second convolutional neural network refers to the convolutional neural network of the first four layers, and the feature fusion network refers to the SPP layer behind the fifth convolutional neural network.
As shown in fig. 2, which is a schematic diagram of feature transfer fusion, in the left part of fig. 2, a is a target image, b is a second feature extracted from the target image, c is a first feature extracted from the target image, and d is a first fusion feature extracted from the target image.
S130, carrying out transmission fusion on the first features through the first fusion features to obtain second fusion features, and carrying out transmission fusion on the second features through the second fusion features to obtain third fusion features.
In the embodiment of the present application, this step is a tack structure. After the first feature, the second feature and the first fusion feature of the target image are obtained in step S120, the features are then transferred and fused to obtain a fusion feature.
Specifically, the determining process of the second fusion feature is as follows: and performing upsampling processing on the first fusion characteristic to obtain first intermediate information, and fusing the first characteristic and the first intermediate information to obtain a second fusion characteristic.
Specifically, the determination process of the third fusion feature is as follows: and performing upsampling processing on the second fusion characteristic to obtain second intermediate information, and fusing the second characteristic and the second intermediate information to obtain a third fusion characteristic.
Fig. 2 is a schematic diagram of feature delivery fusion. In the right part of fig. 2, d is a first fusion feature extracted from the target image, e is a second fusion feature obtained after the first fusion feature and the first feature are processed, and f is a third fusion feature obtained after the second fusion feature and the second feature are processed.
S140, respectively carrying out feature prediction processing on the first fusion feature, the second fusion feature and the third fusion feature to obtain a plurality of prediction frames and attribute information of each prediction frame.
In the embodiment of the present application, the step is a network output layer. And setting a prediction network model, and respectively inputting the first fusion characteristic, the second fusion characteristic and the third fusion characteristic into the prediction network model to obtain corresponding prediction frames and attribute information of the prediction frames, so as to obtain a plurality of prediction frames and attribute information of each prediction frame in the plurality of prediction frames.
And S150, determining whether the image to be identified has flames or not according to the plurality of prediction frames and the attribute information of each prediction frame.
In the embodiment of the present application, this step is also a network output layer. And performing data processing on the plurality of prediction frames according to the attribute information of each prediction frame in the plurality of prediction frames to obtain a processing result, and determining whether the image to be identified has flames or not according to the processing result. The data processing may be Loss calculation, and the Yolo-x uses two types of conventional iou _ Loss or giou _ Loss.
According to the technical scheme provided by the embodiment, the target image is obtained by acquiring the image to be identified and performing data enhancement processing on the image to be identified; inputting the target image into a feature extraction model for feature extraction to obtain a first feature, a second feature and a first fusion feature of the target image; the first feature is subjected to transmission fusion through the first fusion feature to obtain a second fusion feature, and the second feature is subjected to transmission fusion through the second fusion feature to obtain a third fusion feature; respectively performing feature prediction processing on the first fusion feature, the second fusion feature and the third fusion feature to obtain a plurality of prediction frames and attribute information of each prediction frame; and determining whether the image to be identified has flames or not according to the plurality of prediction boxes and the attribute information of each prediction box. Whether this application confirms to treat the discernment image and have flame through adopting to treat the discernment image and carry out image enhancement, feature extraction, feature fusion and feature prediction, has solved prior art and need adopt artifical whenever to carry out the problem that air gun flame monitored, can satisfy the demand to the quick discernment of flame and detection, can also improve the discernment rate of accuracy, the cost of the material resources of using manpower sparingly.
Example two
Fig. 3 is a second flowchart of a flame identification method according to an embodiment of the present disclosure. The embodiment of the application is optimized on the basis of the embodiment, and specifically optimized as follows: this embodiment explains the process of feature prediction in detail.
Referring to fig. 3, the method of the present embodiment includes, but is not limited to, the following steps:
s210, obtaining an image to be recognized, and performing data enhancement processing on the image to be recognized to obtain a target image.
In the embodiment of the application, the image to be identified refers to an image acquired from a flame generating instrument (such as an air gun). The image to be identified of the flame appliance is acquired through data acquisition equipment, and the data acquisition equipment can be a laser radar sensor, a millimeter wave radar sensor, an ultrasonic radar sensor or a visual sensor and the like. And after the image to be recognized is obtained, performing data enhancement processing on the image to be recognized to obtain a target image.
Specifically, the specific process of performing data enhancement processing on the image to be recognized is as follows: splicing the images to be recognized for multiple times to obtain a first intermediate image and a second intermediate image; splicing comprises at least one of random scaling, random clipping and random arrangement; filling left and right edges of the first intermediate image to obtain a first candidate image; filling upper and lower edges of the second intermediate image to obtain a second candidate image; and performing weighted fusion on the first candidate image and the second candidate image to obtain a target image.
In the embodiment of the present application, this step is a network input end. After the image to be recognized is obtained, the Mosaic algorithm is adopted to carry out random scaling, random cutting and random arrangement on the image to be recognized, so that multiple splicing is realized, the data of the image to be recognized is enhanced, and a first intermediate image and a second intermediate image are obtained. In addition, a Mix Up algorithm is adopted on the basis of the Mosaic algorithm, and left and right edges of the first intermediate image are filled to obtain a first candidate image; filling upper and lower edges of the second intermediate image to obtain a second candidate image; and setting a fusion coefficient, and performing weighted fusion on the first candidate image and the second candidate image to obtain a target image.
S220, inputting the target image into the feature extraction model for feature extraction to obtain a first feature, a second feature and a first fusion feature of the target image.
In the embodiment of the present application, the step is a backbone network. The feature extraction model may be a backbone network of Yolox-Darknet53, where the backbone network of Yolox-Darknet53 is substantially the same as the backbone network of Yolov3 baseline, but a Spatial Pyramid Pool (SPP) layer is added, and the original training strategy is slightly adjusted, such as adding a weight update of a sliding Average (EMA), a cosine learning rate mechanism, Cross-over Union (IoU) Loss, IoU perceptual branch, etc., and training classification and regression branch by Binary Cross Entropy Loss (BCE Loss), IoU Loss regression branch.
Specifically, inputting a target image into a feature extraction model for feature extraction to obtain a first feature, a second feature and a first fusion feature of the target image, including: performing feature extraction on the target image through a first convolution neural network in the feature extraction model to obtain a first feature of the target image; performing feature extraction on the target image through a second convolutional neural network in the feature extraction model to obtain second features of the target image; and performing feature fusion on the second features through a feature fusion network in the feature extraction model to obtain first fusion features of the target image.
The feature extraction model comprises five layers of convolutional neural networks and an SPP layer, the first convolutional neural network refers to the convolutional neural network of the first three layers, the second convolutional neural network refers to the convolutional neural network of the first four layers, and the feature fusion network refers to the SPP layer behind the five layers of convolutional neural networks.
And S230, carrying out transmission fusion on the first features through the first fusion features to obtain second fusion features, and carrying out transmission fusion on the second features through the second fusion features to obtain third fusion features.
In the embodiment of the present application, this step is a tack structure. Among the nerck structures, the nerck structures of Yolox-Darknet53 and Yolov3 basepine are also structures using Feature Pyramid Network (FPN) layers. After the first feature, the second feature and the first fusion feature of the target image are obtained in step S120, the features are then transferred and fused to obtain a fusion feature.
Specifically, the determining process of the second fusion feature is as follows: and performing upsampling processing on the first fusion characteristic to obtain first intermediate information, and fusing the first characteristic and the first intermediate information to obtain a second fusion characteristic.
Specifically, the determination process of the third fusion feature is as follows: and performing upsampling processing on the second fusion characteristic to obtain second intermediate information, and fusing the second characteristic and the second intermediate information to obtain a third fusion characteristic.
S240, conducting decoupling head processing on the first fusion features to obtain a first prediction result, conducting decoupling head processing on the second fusion features to obtain a second prediction result, and conducting decoupling head processing on the third fusion features to obtain a third prediction result.
In the embodiment of the present application, the step is a network output layer, and the network output layer includes decorupped Head, Anchor Free, label distribution and Loss calculation. First is a decorupped Head, i.e. a decoupling Head. In this embodiment, three decoupling heads are included, and the sizes of prediction frames obtained by different decoupling heads are different.
Further, performing decoupling head processing on the first fusion feature to obtain a first prediction result, including: carrying out category judgment on the first fusion characteristics to obtain a category prediction result; performing foreground and background judgment on the first fusion characteristic to obtain a foreground and background prediction result; determining location information for the first fused feature; and determining a first prediction result according to the category prediction result, the foreground and background prediction results and the position information.
In the embodiment of the application, the decoupling head comprises three branches, and for the first branch: the method is characterized in that the category of the first fusion feature is predicted to obtain a prediction score, namely a category prediction result, and the prediction score is changed into prediction information with the size of 20-80 after being processed by a Sigmoid function. For the second branch: the method mainly judges whether a target frame is a foreground or a background, and after being processed by a Sigmoid function, the target frame becomes prediction information with the size of 20 × 1. For the third branch: the method is used for predicting the position information of a prediction frame, comprises two-dimensional coordinate information of the prediction frame and width and height information of the prediction frame, and becomes prediction information with the size of 20 × 4 after being processed by a Sigmoid function, so that the size of the prediction information of a first prediction result is 20 × 85.
It should be noted that the decoupling head processing is performed on the second fusion feature to obtain a second prediction result, the decoupling head processing is performed on the third fusion feature to obtain a third prediction result, and the execution process is the same as that of the decoupling head processing performed on the first fusion feature to obtain the first prediction result. The difference is that the prediction information size of the first prediction result is 20 × 85, the prediction information size of the second prediction result is 40 × 85, and the prediction information size of the third prediction result is 80 × 85.
And S250, carrying out merging processing, reshaping processing and transposition processing on the first prediction result, the second prediction result and the third prediction result to obtain a plurality of prediction frames and attribute information of each prediction frame.
In the examples of this application, this step is Anchor Free. After obtaining the three prediction results in step S240, first, merging the three prediction results, where each prediction result is used to determine a regression parameter of each feature point, and the regression parameters are adjusted to obtain a prediction frame; and the prediction method is used for judging whether each feature point contains an object or not and for judging the type of the object contained in each feature point, so as to obtain a prediction result after combination. And then, reshaping the combined prediction result to enable the information with different sizes to be unified and further processed, thereby obtaining the reshaped prediction result. And finally, transposing the reconstructed prediction result into a two-dimensional vector containing the number of the prediction frames and the information of each prediction frame, thereby obtaining a plurality of prediction frames and the attribute information of each prediction frame.
In the embodiment of the present application, the size of the prediction information of the first prediction result is 20 × 85, there are 400 frames, and the size of the corresponding anchor frame is 32 × 32. The prediction information size of the second prediction result is 40 × 85, wherein 1600 prediction frames exist, and the size of the corresponding anchor frame is 16 × 16. The prediction information size of the third prediction result is 80 × 85, which includes 6400 prediction frames, the size of the corresponding anchor frame is 8 × 8, and a total of 8400 prediction frames.
S260, screening a positive sample prediction frame from the multiple prediction frames according to the position relation between the anchor frame for labeling the image to be recognized and the image to be recognized.
In the embodiment of the present application, this step is label assignment. Only a small part of attribute information of the prediction frame is positive samples, and most of attribute information of the prediction frame is negative samples, so that a part of suitable positive sample anchor frame, namely the positive sample prediction frame, needs to be selected by utilizing the position relation between the anchor frame and the image to be recognized. For example: the 3 rd, 10 th and 15 th anchor frames are positive sample anchor frames, and the 3 rd, 10 th and 15 th prediction frames are corresponding positive sample prediction frames in 8400 prediction frames output by the network. In the training process, on the basis of the anchor frame, continuous prediction is carried out, and then continuous iteration is carried out, so that network parameters are updated, and the network prediction is more and more accurate.
Further, after the positive sample prediction box is selected through the initial screening, further fine screening through the SimOTA is required. The whole fine screening process is mainly divided into four stages: extracting the information of the primary screening positive sample, calculating a loss function, calculating loss cost and solving the SimOTA. The target box and the positive sample prediction box can be corresponded by label allocation.
S270, performing loss function calculation on the attribute information of the positive sample prediction frame to obtain a loss result, and determining whether the image to be identified has flames or not according to the loss result.
In the embodiment of the present application, this step is Loss calculation. After the positive sample prediction frames are screened out from the plurality of prediction frames in the step S260, the loss function calculation is performed on the attribute information of the positive sample prediction frames, that is, the loss is calculated for each single category, so as to obtain a loss result, and whether the image to be identified has flames or not is determined according to the loss result.
According to the technical scheme provided by the embodiment, the target image is obtained by acquiring the image to be identified and performing data enhancement processing on the image to be identified; inputting the target image into a feature extraction model for feature extraction to obtain a first feature, a second feature and a first fusion feature of the target image; the first feature is subjected to transmission fusion through the first fusion feature to obtain a second fusion feature, and the second feature is subjected to transmission fusion through the second fusion feature to obtain a third fusion feature; performing decoupling head processing on the first fusion characteristic to obtain a first prediction result, performing decoupling head processing on the second fusion characteristic to obtain a second prediction result, and performing decoupling head processing on the third fusion characteristic to obtain a third prediction result; merging, reshaping and transposing the first prediction result, the second prediction result and the third prediction result to obtain a plurality of prediction frames and attribute information of each prediction frame; screening a positive sample prediction frame from a plurality of prediction frames according to the position relation between an anchor frame for labeling the image to be recognized and the image to be recognized; and performing loss function calculation on the attribute information of the positive sample prediction frame to obtain a loss result, and determining whether the image to be identified has flames or not according to the loss result. Whether this application confirms to treat the discernment image and have flame through adopting to treat the discernment image and carry out image enhancement, feature extraction, feature fusion and feature prediction, has solved prior art and need adopt artifical whenever to carry out the problem that air gun flame monitored, can satisfy the demand to the quick discernment of flame and detection, can also improve the discernment rate of accuracy, the cost of the material resources of using manpower sparingly.
EXAMPLE III
Fig. 4 is a schematic structural diagram of a flame identification apparatus according to an embodiment of the present disclosure, and as shown in fig. 4, the apparatus 400 may include:
and the target image determining module 410 is configured to obtain an image to be identified, and perform data enhancement processing on the image to be identified to obtain a target image.
The first feature determining module 420 is configured to input the target image into a feature extraction model for feature extraction, so as to obtain a first feature, a second feature, and a first fusion feature of the target image.
The second feature determining module 430 is configured to perform transfer fusion on the first feature through the first fusion feature to obtain a second fusion feature, and perform transfer fusion on the second feature through the second fusion feature to obtain a third fusion feature.
A prediction box determining module 440, configured to perform feature prediction processing on the first fusion feature, the second fusion feature, and the third fusion feature respectively to obtain a plurality of prediction boxes and attribute information of each prediction box.
And a flame identification module 450, configured to determine whether there is a flame in the image to be identified according to the multiple prediction boxes and the attribute information of each prediction box.
Further, the target image determining module 410 may be specifically configured to: splicing the images to be identified for multiple times to obtain a first intermediate image and a second intermediate image; the splicing comprises at least one of random scaling, random clipping and random arrangement; filling left and right edges of the first intermediate image to obtain a first candidate image; filling upper and lower edges of the second intermediate image to obtain a second candidate image; and performing weighted fusion on the first candidate image and the second candidate image to obtain the target image.
Further, the first characteristic determining module 420 may be specifically configured to: performing feature extraction on the target image through a first convolution neural network in the feature extraction model to obtain a first feature of the target image; performing feature extraction on the target image through a second convolutional neural network in the feature extraction model to obtain a second feature of the target image; and performing feature fusion on the second features through a feature fusion network in the feature extraction model to obtain first fusion features of the target image.
Further, the second characteristic determining module 430 may be specifically configured to: performing upsampling processing on the first fusion feature to obtain first intermediate information, and fusing the first feature and the first intermediate information to obtain a second fusion feature; and performing upsampling processing on the second fusion characteristic to obtain second intermediate information, and fusing the second characteristic and the second intermediate information to obtain a third fusion characteristic.
Further, the prediction block determining module 440 may be specifically configured to: performing decoupling head processing on the first fusion characteristic to obtain a first prediction result, performing decoupling head processing on the second fusion characteristic to obtain a second prediction result, and performing decoupling head processing on the third fusion characteristic to obtain a third prediction result; and performing merging processing, reshaping processing and transposing processing on the first prediction result, the second prediction result and the third prediction result to obtain the plurality of prediction frames and the attribute information of each prediction frame.
Further, the prediction block determining module 440 may be further specifically configured to: carrying out category judgment on the first fusion characteristics to obtain a category prediction result; performing foreground and background judgment on the first fusion characteristic to obtain a foreground and background prediction result; determining location information for the first fused feature; and determining the first prediction result according to the category prediction result, the foreground and background prediction results and the position information.
Further, the flame identification module 450 may be specifically configured to: screening a positive sample prediction frame from the multiple prediction frames according to the position relation between an anchor frame for labeling the image to be recognized and the image to be recognized; and performing loss function calculation on the attribute information of the positive sample prediction frame to obtain a loss result, and determining whether the image to be identified has flames or not according to the loss result.
The flame identification device provided by the embodiment can be applied to the flame identification method provided by any embodiment, and has corresponding functions and beneficial effects.
Example four
FIG. 5 is a block diagram of an electronic device suitable for implementing a method of flame identification of embodiments of the present application, and FIG. 5 shows a block diagram of an exemplary electronic device suitable for implementing embodiments of the present application. The electronic device shown in fig. 5 is only an example, and should not bring any limitation to the functions and applicable scope of the embodiments of the present application. The electronic device can be a smart phone, a tablet computer, a notebook computer, a vehicle-mounted terminal, a wearable device and the like.
As shown in fig. 5, the electronic device 500 is embodied in the form of a general purpose computing device. The components of the electronic device 500 may include, but are not limited to: one or more processors or processing units 516, a memory 528, and a bus 518 that couples the various system components including the memory 528 and the processing unit 516.
A program/utility 540 having a set (at least one) of program modules 542, such program modules 542 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, may be stored in, for example, the memory 528, each of which examples or some combination thereof may include an implementation of a network environment. The program modules 542 generally perform the functions and/or methods described in embodiments herein.
The electronic device 500 may also communicate with one or more external devices 514 (e.g., keyboard, pointing device, display 524, etc.), with one or more devices that enable a user to interact with the electronic device 500, and/or with any devices (e.g., network card, modem, etc.) that enable the electronic device 500 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interfaces 522. Also, the electronic device 500 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 520. As shown in FIG. 5, the network adapter 520 communicates with the other modules of the electronic device 500 via the bus 518. It should be appreciated that although not shown in FIG. 5, other hardware and/or software modules may be used in conjunction with the electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processing unit 516 executes various functional applications and data processing by executing programs stored in the memory 528, for example, implementing the flame identification method provided in any embodiment of the present application.
EXAMPLE five
Embodiments of the present application also provide a computer-readable storage medium, on which a computer program (or referred to as computer-executable instructions) is stored, and when the program is executed by a processor, the program can be used to execute the flame identification method provided in any of the above embodiments of the present application.
The computer storage media of the embodiments of the present application may take any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for embodiments of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
Claims (10)
1. A method of flame identification, the method comprising:
acquiring an image to be identified, and performing data enhancement processing on the image to be identified to obtain a target image;
inputting the target image into a feature extraction model for feature extraction to obtain a first feature, a second feature and a first fusion feature of the target image;
performing transfer fusion on the first feature through the first fusion feature to obtain a second fusion feature, and performing transfer fusion on the second feature through the second fusion feature to obtain a third fusion feature;
respectively performing feature prediction processing on the first fusion feature, the second fusion feature and the third fusion feature to obtain a plurality of prediction frames and attribute information of each prediction frame;
and determining whether the image to be identified has flames or not according to the plurality of prediction boxes and the attribute information of each prediction box.
2. The flame identification method according to claim 1, wherein the performing data enhancement processing on the image to be identified to obtain a target image comprises:
splicing the images to be identified for multiple times to obtain a first intermediate image and a second intermediate image; the splicing comprises at least one of random scaling, random clipping and random arrangement;
filling left and right edges of the first intermediate image to obtain a first candidate image; filling upper and lower edges of the second intermediate image to obtain a second candidate image; and performing weighted fusion on the first candidate image and the second candidate image to obtain the target image.
3. The flame identification method according to claim 1, wherein the inputting the target image into a feature extraction model for feature extraction to obtain a first feature, a second feature and a first fused feature of the target image comprises:
performing feature extraction on the target image through a first convolution neural network in the feature extraction model to obtain a first feature of the target image;
performing feature extraction on the target image through a second convolutional neural network in the feature extraction model to obtain a second feature of the target image;
and performing feature fusion on the second features through a feature fusion network in the feature extraction model to obtain first fusion features of the target image.
4. The flame identification method according to claim 1, wherein the transfer-fusing the first feature by the first fused feature to obtain a second fused feature, and the transfer-fusing the second feature by the second fused feature to obtain a third fused feature comprises:
performing upsampling processing on the first fusion feature to obtain first intermediate information, and fusing the first feature and the first intermediate information to obtain a second fusion feature;
and performing upsampling processing on the second fusion characteristic to obtain second intermediate information, and fusing the second characteristic and the second intermediate information to obtain a third fusion characteristic.
5. The flame identification method according to claim 1, wherein the performing of the feature prediction processing on the first fused feature, the second fused feature, and the third fused feature to obtain a plurality of prediction boxes and attribute information of each prediction box comprises:
performing decoupling head processing on the first fusion characteristic to obtain a first prediction result, performing decoupling head processing on the second fusion characteristic to obtain a second prediction result, and performing decoupling head processing on the third fusion characteristic to obtain a third prediction result;
and performing merging processing, reshaping processing and transposing processing on the first prediction result, the second prediction result and the third prediction result to obtain the plurality of prediction frames and the attribute information of each prediction frame.
6. The flame identification method of claim 5, wherein the decoupling head processing the first fused feature to obtain a first prediction result comprises:
carrying out category judgment on the first fusion characteristics to obtain a category prediction result;
performing foreground and background judgment on the first fusion characteristic to obtain a foreground and background prediction result;
determining location information for the first fused feature;
and determining the first prediction result according to the category prediction result, the foreground and background prediction results and the position information.
7. The flame identification method according to claim 1, wherein the determining whether the image to be identified has flames according to the plurality of prediction boxes and the attribute information of each prediction box comprises:
screening a positive sample prediction frame from the multiple prediction frames according to the position relation between an anchor frame for labeling the image to be recognized and the image to be recognized;
and performing loss function calculation on the attribute information of the positive sample prediction frame to obtain a loss result, and determining whether the image to be identified has flames or not according to the loss result.
8. A flame identification device, the device comprising:
the target image determining module is used for acquiring an image to be identified and performing data enhancement processing on the image to be identified to obtain a target image;
the first feature determination module is used for inputting the target image into a feature extraction model for feature extraction to obtain a first feature, a second feature and a first fusion feature of the target image;
the second feature determination module is used for performing transfer fusion on the first feature through the first fusion feature to obtain a second fusion feature, and performing transfer fusion on the second feature through the second fusion feature to obtain a third fusion feature;
the prediction frame determining module is used for respectively performing feature prediction processing on the first fusion feature, the second fusion feature and the third fusion feature to obtain a plurality of prediction frames and attribute information of each prediction frame;
and the flame identification module is used for determining whether the image to be identified has flames or not according to the plurality of prediction boxes and the attribute information of each prediction box.
9. An electronic device, characterized in that the electronic device comprises:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the flame identification method of any of claims 1-7.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a flame identification method as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210126854.9A CN114170575A (en) | 2022-02-11 | 2022-02-11 | Flame identification method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210126854.9A CN114170575A (en) | 2022-02-11 | 2022-02-11 | Flame identification method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114170575A true CN114170575A (en) | 2022-03-11 |
Family
ID=80489680
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210126854.9A Pending CN114170575A (en) | 2022-02-11 | 2022-02-11 | Flame identification method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114170575A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114998337A (en) * | 2022-08-03 | 2022-09-02 | 联宝(合肥)电子科技有限公司 | Scratch detection method, device, equipment and storage medium |
| CN117315551A (en) * | 2023-11-29 | 2023-12-29 | 深圳鲲云信息科技有限公司 | Method and computing device for flame alerting |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190050942A1 (en) * | 2017-08-11 | 2019-02-14 | American International Group, Inc. | Systems and methods for dynamic real-time analysis from multi-modal data fusion for contextual risk identification |
| CN111598164A (en) * | 2020-05-15 | 2020-08-28 | 北京百度网讯科技有限公司 | Method and device for identifying attribute of target object, electronic equipment and storage medium |
| WO2021129691A1 (en) * | 2019-12-23 | 2021-07-01 | 长沙智能驾驶研究院有限公司 | Target detection method and corresponding device |
| CN113935364A (en) * | 2021-09-22 | 2022-01-14 | 常熟理工学院 | Unmanned aerial vehicle flame detection method and system based on YOLOv4 network |
-
2022
- 2022-02-11 CN CN202210126854.9A patent/CN114170575A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20190050942A1 (en) * | 2017-08-11 | 2019-02-14 | American International Group, Inc. | Systems and methods for dynamic real-time analysis from multi-modal data fusion for contextual risk identification |
| WO2021129691A1 (en) * | 2019-12-23 | 2021-07-01 | 长沙智能驾驶研究院有限公司 | Target detection method and corresponding device |
| CN111598164A (en) * | 2020-05-15 | 2020-08-28 | 北京百度网讯科技有限公司 | Method and device for identifying attribute of target object, electronic equipment and storage medium |
| CN113935364A (en) * | 2021-09-22 | 2022-01-14 | 常熟理工学院 | Unmanned aerial vehicle flame detection method and system based on YOLOv4 network |
Non-Patent Citations (3)
| Title |
|---|
| ZHENG GE 等: "YOLOX: Exceeding YOLO Series in 2021", 《ARXIV》 * |
| 李小飞: "基于图像的火焰检测算法", 《电子技术与软件工程》 * |
| 腾讯新闻: "深入浅出Yolo系列之Yolox核心基础完整讲解", 《HTTPS://VIEW.INEWS.QQ.COM/A/20210816A042O800》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114998337A (en) * | 2022-08-03 | 2022-09-02 | 联宝(合肥)电子科技有限公司 | Scratch detection method, device, equipment and storage medium |
| CN114998337B (en) * | 2022-08-03 | 2022-11-04 | 联宝(合肥)电子科技有限公司 | Scratch detection method, device, equipment and storage medium |
| CN117315551A (en) * | 2023-11-29 | 2023-12-29 | 深圳鲲云信息科技有限公司 | Method and computing device for flame alerting |
| CN117315551B (en) * | 2023-11-29 | 2024-03-19 | 深圳鲲云信息科技有限公司 | Method and computing device for flame alerting |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109918513B (en) | Image processing method, device, server and storage medium | |
| EP3637310A1 (en) | Method and apparatus for generating vehicle damage information | |
| CN114170575A (en) | Flame identification method and device, electronic equipment and storage medium | |
| KR20220122566A (en) | Text recognition model training method, text recognition method, and apparatus | |
| CN115861462B (en) | Training method and device for image generation model, electronic equipment and storage medium | |
| CN114863437B (en) | Text recognition method and device, electronic equipment and storage medium | |
| CN115063875A (en) | Model training method, image processing method, device and electronic equipment | |
| CN114299366A (en) | Image detection method and device, electronic equipment and storage medium | |
| CN115392937A (en) | User fraud risk identification method and device, electronic equipment and storage medium | |
| US20230245429A1 (en) | Method and apparatus for training lane line detection model, electronic device and storage medium | |
| CN110781849A (en) | Image processing method, device, equipment and storage medium | |
| CN114373460A (en) | Instruction determination method, device, equipment and medium for vehicle-mounted voice assistant | |
| CN113936232A (en) | Screen fragmentation identification method, device, equipment and storage medium | |
| CN114266860A (en) | Three-dimensional face model establishing method and device, electronic equipment and storage medium | |
| CN113643260A (en) | Method, apparatus, device, medium and product for detecting image quality | |
| CN112967248A (en) | Method, apparatus, medium, and program product for generating defect image sample | |
| CN113762303A (en) | Image classification method and device, electronic equipment and storage medium | |
| CN116468970A (en) | Model training method, image processing method, device, equipment and medium | |
| CN114741697B (en) | Malicious code classification method and device, electronic equipment and medium | |
| KR20230133808A (en) | Method and apparatus for training roi detection model, method and apparatus for detecting roi, device, and medium | |
| CN114241411B (en) | Counting model processing method and device based on target detection and computer equipment | |
| CN115116080A (en) | Table analysis method and device, electronic equipment and storage medium | |
| CN114882557A (en) | Face recognition method and device | |
| CN111049988A (en) | Intimacy prediction method, system, equipment and storage medium for mobile equipment | |
| CN112784189A (en) | Method and device for identifying page image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20220311 |