CN116206221B - Water flare detection method and system - Google Patents
Water flare detection method and system Download PDFInfo
- Publication number
- CN116206221B CN116206221B CN202310077797.4A CN202310077797A CN116206221B CN 116206221 B CN116206221 B CN 116206221B CN 202310077797 A CN202310077797 A CN 202310077797A CN 116206221 B CN116206221 B CN 116206221B
- Authority
- CN
- China
- Prior art keywords
- output
- decoding
- layer
- coding
- flare
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/776—Validation; Performance evaluation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/30—Assessment of water resources
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Remote Sensing (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method and a system for detecting water flare. The method comprises the following steps: collecting a water body image graph, and marking the water body image graph to generate a training set, a verification set and a test set; designing a deep neural network model for detecting water flare; training and adjusting parameters of the deep neural network model by applying the training set and the verification set; verifying the accuracy of the trained deep neural network model by using the test set and the evaluation index; and carrying out water flare detection on the acquired single water body image by applying the trained deep neural network model. According to the scheme provided by the invention, the false detection phenomenon of other ground objects is reduced; the over-fitting phenomenon is reduced, and the detected water flare is more accurate. The invention detects the water flare with Precision reaching 0.835, recall reaching 0.887, F1 reaching 0.836, IOU reaching 0.747.
Description
Technical Field
The invention belongs to the field of water flare detection, and particularly relates to a water flare detection method and system.
Background
The unmanned aerial vehicle remote sensing technology has the characteristics of strong timeliness, low cost, good portability and the like, overcomes the defects of low spatial resolution, long revisit period, easiness in being influenced by cloud and fog and the like of the traditional satellite remote sensing technology, and is widely applied to the observation of an earth surface ecosystem. The color of the water body mainly depends on the absorption and scattering characteristics of substances such as chlorophyll, suspended matters, colored soluble organic matters and the like in the water to light, and the occurrence of solar flare seriously interferes with the spectrum signal of the water body, so that great difficulty is caused to remote sensing ground object classification, water-leaving reflectivity estimation, water environment remote sensing monitoring and the like.
Under strong light irradiation, the surface of a non-homogeneous object is easy to generate mirror surface emission, so that the phenomenon of spectrum abnormality occurs. When the target object is a body of water, the surface of the water specularly reflects solar radiation to form a flare. In the process of acquiring the ground image by using the unmanned aerial vehicle, when the water body, the sun and the lens form a specular reflection condition, the water body part in the image can contain a large amount of flare. In addition, according to different viewpoints and light source positions, the positions of flare spots are different, the flare spots possibly appear in various areas of the water surface in the process of shooting the aircraft along the movement of the route, and a large number of spectrum saturation areas can be formed after the single images are spliced. However, due to the complex illumination environment and various ground object categories of image shooting, the existing flare detection and removal algorithm is difficult to be directly applied to unmanned aerial vehicle images.
The first prior art is:
the method for automatically detecting and compensating the highlight of the image water body of the single unmanned aerial vehicle comprises the following steps: and (3) approximating the high optical path by using the high light component in the unmanned aerial vehicle image, and taking the minimum value of red, green and blue as the high light component. And establishing a multi-scale Gaussian pyramid for the original image, and calculating a highlight component of each scale and a saturation component of the HSV space. And then performing threshold segmentation by using the pyramid images of the two components.
And the second prior art is as follows:
patent CN111932651a provides a method and a device for extracting water flare, wherein the method comprises the following steps: acquiring a remote sensing image to be extracted; the remote sensing image to be extracted is a sentinel 2 remote sensing image; determining the remote sensing reflectivity of each wave band in the remote sensing image to be extracted to obtain a remote sensing reflectivity image; extracting a water body region containing water surface flare from the remote sensing reflectivity image to obtain an extracted remote sensing reflectivity image; determining that a pixel with the remote sensing reflectivity of a third wave band greater than a preset threshold value in the extracted remote sensing reflectivity image is a water surface flare pixel; and generating a water surface flare distribution map according to the water surface flare pixel. The method can extract the water surface flare from the remote sensing image to be extracted.
Defects of the prior art
The first prior art is: the method not only separates out the actual highlight region, but also contains false detection of more non-water body ground objects.
And the second prior art is as follows: the method provides a water flare extraction method for a sentinel No. 2 satellite image, however, the research object aimed at by the research institute is an unmanned plane visible light image. There is a large difference between the sensors and the lack of near infrared band information for the visible image on the band set-up creates greater uncertainty for flare extraction.
Disclosure of Invention
In order to solve the technical problems, the invention provides a technical scheme of a water body flare detection method, so as to solve the technical problems. The invention discloses a water flare detection method, which comprises the following steps: s1, acquiring a water body image graph, and labeling the water body image graph to generate a training set, a verification set and a test set;
s2, designing a deep neural network model for detecting water flare;
s3, training and parameter adjustment are carried out on the deep neural network model by applying the training set and the verification set;
s4, verifying the accuracy of the trained deep neural network model by using the test set and the evaluation index;
and S5, carrying out water flare detection on the acquired single water body image by applying the trained deep neural network model.
According to the method of the first aspect of the present invention, in the step S2, the deep neural network model includes: an encoding unit and a decoding unit;
the coding unit extracts water flare characteristics of the water body image map to obtain a water flare characteristic coding output map;
and inputting the water flare characteristic coding output diagram into the decoding unit to obtain a flare detection result diagram.
According to the method of the first aspect of the present invention, in the step S2, the method for extracting the water flare feature from the water body image map by the encoding unit to obtain the water flare feature encoding output map includes:
the coding unit includes 5 coding layers: coding level1 layer, coding level2 layer, coding level3 layer, coding level4 layer and coding level5 layer;
inputting the water body image map into the coding level1 layer, and obtaining a first coding output through two continuous convolutions;
after the first code output is subjected to maximum pooling operation, inputting the code level2 layer, and carrying out continuous convolution twice to obtain a second code output;
after the second code output is subjected to maximum pooling operation, inputting the code level3 layer, and obtaining a third code output through two continuous convolutions;
after the third code output is subjected to maximum pooling operation, inputting the code level4 layer, and obtaining a fourth code output through two continuous convolutions;
and after carrying out maximum pooling operation on the fourth code output, inputting the code level5 layer, and carrying out continuous convolution twice to obtain a water flare characteristic code output graph.
According to the method of the first aspect of the present invention, in the step S2, the method of continuous convolution is:
Firstly, performing first 3×3 two-dimensional convolution, then performing first batch standardization on the output of the first 3×3 two-dimensional convolution, then inputting the output of the first batch standardization into a first ReLU activation function, then performing second 3×3 two-dimensional convolution on the output of the first ReLU activation function, then performing second batch standardization on the output of the second 3×3 two-dimensional convolution, then inputting the output of the second batch standardization and the output of the first 3×3 two-dimensional convolution together into a residual error module, performing addition operation of a feature map, obtaining residual error output, and finally inputting the residual error output into a second ReLU activation function, thereby obtaining a result of two continuous convolutions.
According to the method of the first aspect of the present invention, in the step S2, the method for inputting the water flare characteristic code output map to the decoding unit to obtain a flare detection result map includes:
the decoding unit includes 5 decoding layers: decoding level1 layer, decoding level2 layer, decoding level3 layer, decoding level4 layer, and decoding level5 layer;
inputting the water flare characteristic coding output image into a decoding level5 layer, and obtaining a first decoding output through up-sampling convolution operation;
after the first decoding output and the fourth coding output are connected in a jumping mode in the decoding level4 layer, a CBAM attention mechanism of the decoding level4 layer is input, and after two continuous convolution and up-sampling convolution operations, the output of the CBAM attention mechanism of the decoding level4 layer is subjected to a second decoding output;
After the second decoding output is connected with the third coding output in a jumping way in the decoding level3 layer, the CBAM attention mechanism of the decoding level3 layer is input, and the output of the CBAM attention mechanism of the decoding level3 layer is subjected to two continuous convolution and up-sampling convolution operations to obtain the third decoding output;
after the third decoding output is connected with the second coding output in a jumping way in the decoding level2 layer, the CBAM attention mechanism of the decoding level2 layer is input, and after the output of the CBAM attention mechanism of the decoding level2 layer is subjected to two continuous convolution and up-sampling convolution operations, a fourth decoding output is obtained;
and after the fourth decoding output is connected with the first coding output in a jumping way in the decoding level1 layer, inputting a CBAM attention mechanism of the decoding level1 layer, and after the output of the CBAM attention mechanism of the decoding level1 layer is subjected to two continuous convolutions, inputting 1X 1 convolutions to generate a flare detection result graph with the channel number of 2.
According to the method of the first aspect of the present invention, in the step S2, the method of up-sampling convolution operation is:
the up-sampling is performed first, followed by a 3 x 3 two-dimensional convolution of the up-sampled output.
According to the method of the first aspect of the present invention, in the step S2, there are 32 convolution kernels of the first 3×3 two-dimensional convolution and the second 3×3 two-dimensional convolution of the coding level1 layer;
the first 3×3 two-dimensional convolution of the coding level2 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 64;
the first 3×3 two-dimensional convolution of the coding level3 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 128;
the first 3×3 two-dimensional convolution of the coding level4 layer and the convolution kernels of the second 3×3 two-dimensional convolution have 256;
the first 3 x 3 two-dimensional convolution and the second 3 x 3 two-dimensional convolution of the coding level5 layers have 512 convolution kernels.
The second aspect of the invention discloses a water flare detection system, comprising:
the first processing module is configured to acquire a water body image and label the water body image to generate a training set, a verification set and a test set;
the second processing module is configured to design a deep neural network model for water flare detection;
a third processing module configured to train and tune the deep neural network model using the training set and the validation set;
a fourth processing module configured to verify the accuracy of the trained deep neural network model using the test set and the evaluation index;
And the fifth processing module is configured to apply the trained deep neural network model to detect water flare of the acquired single water body image.
According to the system of the second aspect of the present invention, the second processing module is configured such that the deep neural network model includes: an encoding unit and a decoding unit;
the coding unit extracts water flare characteristics of the water body image map to obtain a water flare characteristic coding output map;
and inputting the water flare characteristic coding output diagram into the decoding unit to obtain a flare detection result diagram.
By adopting the water flare detection method, the Precision for detecting the water flare reaches 0.835, the recall reaches 0.887, the F1 reaches 0.836, and the IOU reaches 0.747
According to the system of the second aspect of the present invention, the second processing module is configured to perform water flare feature extraction on the water body image map by the encoding unit, and the obtaining a water flare feature encoding output map includes:
the coding unit includes 5 coding layers: coding level1 layer, coding level2 layer, coding level3 layer, coding level4 layer and coding level5 layer;
inputting the water body image map into the coding level1 layer, and obtaining a first coding output through two continuous convolutions;
After the first code output is subjected to maximum pooling operation, inputting the code level2 layer, and carrying out continuous convolution twice to obtain a second code output;
after the second code output is subjected to maximum pooling operation, inputting the code level3 layer, and obtaining a third code output through two continuous convolutions;
after the third code output is subjected to maximum pooling operation, inputting the code level4 layer, and obtaining a fourth code output through two continuous convolutions;
and after carrying out maximum pooling operation on the fourth code output, inputting the code level5 layer, and carrying out continuous convolution twice to obtain a water flare characteristic code output graph.
According to the system of the second aspect of the present invention, the second processing module is configured to perform the continuous convolution as:
firstly, performing first 3×3 two-dimensional convolution, then performing first batch standardization on the output of the first 3×3 two-dimensional convolution, then inputting the output of the first batch standardization into a first ReLU activation function, then performing second 3×3 two-dimensional convolution on the output of the first ReLU activation function, then performing second batch standardization on the output of the second 3×3 two-dimensional convolution, then inputting the output of the second batch standardization and the output of the first 3×3 two-dimensional convolution together into a residual error module, performing addition operation of a feature map, obtaining residual error output, and finally inputting the residual error output into a second ReLU activation function, thereby obtaining a result of two continuous convolutions.
According to the system of the second aspect of the present invention, the second processing module is configured to input the water flare feature encoding output map to the decoding unit, and the obtaining of the flare detection result map includes:
the decoding unit includes 5 decoding layers: decoding level1 layer, decoding level2 layer, decoding level3 layer, decoding level4 layer, and decoding level5 layer;
inputting the water flare characteristic coding output image into a decoding level5 layer, and obtaining a first decoding output through up-sampling convolution operation;
after the first decoding output and the fourth coding output are connected in a jumping mode in the decoding level4 layer, a CBAM attention mechanism of the decoding level4 layer is input, and after two continuous convolution and up-sampling convolution operations, the output of the CBAM attention mechanism of the decoding level4 layer is subjected to a second decoding output;
after the second decoding output is connected with the third coding output in a jumping way in the decoding level3 layer, the CBAM attention mechanism of the decoding level3 layer is input, and the output of the CBAM attention mechanism of the decoding level3 layer is subjected to two continuous convolution and up-sampling convolution operations to obtain the third decoding output;
after the third decoding output is connected with the second coding output in a jumping way in the decoding level2 layer, the CBAM attention mechanism of the decoding level2 layer is input, and after the output of the CBAM attention mechanism of the decoding level2 layer is subjected to two continuous convolution and up-sampling convolution operations, a fourth decoding output is obtained;
And after the fourth decoding output is connected with the first coding output in a jumping way in the decoding level1 layer, inputting a CBAM attention mechanism of the decoding level1 layer, and after the output of the CBAM attention mechanism of the decoding level1 layer is subjected to two continuous convolutions, inputting 1X 1 convolutions to generate a flare detection result graph with the channel number of 2.
According to the system of the second aspect of the present invention, the second processing module is configured to perform the upsampling convolution operation as:
the up-sampling is performed first, followed by a 3 x 3 two-dimensional convolution of the up-sampled output.
According to the system of the second aspect of the present invention, the second processing module is configured to 32 convolution kernels of the first 3×3 two-dimensional convolution and the second 3×3 two-dimensional convolution of the coding level1 layer;
the first 3×3 two-dimensional convolution of the coding level2 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 64;
the first 3×3 two-dimensional convolution of the coding level3 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 128;
the first 3×3 two-dimensional convolution of the coding level4 layer and the convolution kernels of the second 3×3 two-dimensional convolution have 256;
the first 3 x 3 two-dimensional convolution and the second 3 x 3 two-dimensional convolution of the coding level5 layers have 512 convolution kernels.
A third aspect of the invention discloses an electronic device. The electronic device comprises a memory storing a computer program and a processor implementing the steps in a method of water flare detection of any one of the first aspects of the present disclosure when the processor executes the computer program.
A fourth aspect of the invention discloses a computer-readable storage medium. A computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps in a method of water flare detection of any one of the first aspects of the present disclosure.
According to the scheme provided by the invention, the Res_AUnet convolutional neural network is provided for flare existing in the visible light image of the unmanned aerial vehicle based on the water body, and the flare is accurately extracted. The model is focused on the extraction of the flare of the water body by adding the attention mechanism, so that the false detection phenomenon of other ground objects is reduced. The number of convolution kernels of each layer is properly reduced, and a residual error module is added, so that the overfitting phenomenon is reduced, and the detected water flare is more accurate.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for detecting water flare in accordance with an embodiment of the present invention;
FIG. 2 is a flow chart of an experimental method according to an embodiment of the invention;
FIG. 3 is an exemplary diagram of a dataset according to an embodiment of the invention;
fig. 4 is a diagram of a res_aunet network structure according to an embodiment of the present invention;
FIG. 5 is a diagram of a continuous convolution architecture in accordance with an embodiment of the present disclosure;
FIG. 6 is a block diagram of a CBAM attention mechanism according to an embodiment of the invention;
FIG. 7 is a diagram of a channel attention mechanism architecture according to an embodiment of the present invention;
FIG. 8 is a block diagram of a spatial attention mechanism according to an embodiment of the present invention;
fig. 9 is a diagram showing water flare detection results of res_aunet network, a AUnet network model and a threshold segmentation method on three randomly selected unmanned aerial vehicle photos according to an embodiment of the present invention;
FIG. 10 is a comparison of water flare detection results on an image of an unmanned aerial vehicle according to an embodiment of the present invention;
FIG. 11 is a block diagram of a water flare detection system according to an embodiment of the present invention;
fig. 12 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The first aspect of the invention discloses a method for detecting water flare. Fig. 1 is a flowchart of a method for detecting water flare according to an embodiment of the present invention, as shown in fig. 1 and fig. 2, the method includes:
s1, acquiring a water body image graph, and labeling the water body image graph to generate a training set, a verification set and a test set;
s2, designing a deep neural network model for detecting water flare;
s3, training and parameter adjustment are carried out on the deep neural network model by applying the training set and the verification set;
s4, verifying the accuracy of the trained deep neural network model by using the test set and the evaluation index;
and S5, carrying out water flare detection on the acquired single water body image by applying the trained deep neural network model.
In step S1, a water body image graph is acquired, and the water body image graph is marked to generate a training set, a verification set and a test set.
Specifically, a visible light camera mounted on an unmanned aerial vehicle is used for shooting a visible light image of a water body, and the flight height of the unmanned aerial vehicle is set to 300m. A large number of water body image pictures are obtained through photographing the river, and the size of the image pictures is 5472 multiplied by 3648.
In some embodiments, in particular, the water body image map is annotated using ENVI software, and the original unmanned aerial vehicle photograph (5472×3648) and the annotation file (5472×3648) are cropped to a uniform 256×256 size using Python programming language, resulting in 1764 sets of images.
The method comprises the following specific steps:
1) Labeling the water body image map (5472 x 3648) with ENVI;
2) Python cuts image photographs and markup files (both 5472x 3648), with a size of 256x256 after cutting
3) Together, 1764 sets of images are generated after cropping.
Then, according to 7:2:1, dividing a training set, a testing set and a verification set, wherein 1232 groups of images are used for training a newly constructed deep neural network model, 176 groups of images are used for verifying model precision in the training process, preventing an overfitting phenomenon, and 356 groups of images are used for testing network model effects after training is finished. An example of a dataset is shown in fig. 3. In the picture right side labeling file, white represents a flare region of the labeling, and black represents a non-flare region.
In step S2, a deep neural network model for water flare detection is designed.
In some embodiments, in the step S2, the deep neural network model includes: an encoding unit and a decoding unit;
The coding unit extracts water flare characteristics of the water body image map to obtain a water flare characteristic coding output map;
and inputting the water flare characteristic coding output diagram into the decoding unit to obtain a flare detection result diagram.
The method for extracting the water flare characteristics of the water body image map by the coding unit to obtain the water body flare characteristic coding output map comprises the following steps:
the coding unit includes 5 coding layers: coding level1 layer, coding level2 layer, coding level3 layer, coding level4 layer and coding level5 layer;
inputting the water body image map into the coding level1 layer, and obtaining a first coding output through two continuous convolutions;
after the first code output is subjected to maximum pooling operation, inputting the code level2 layer, and carrying out continuous convolution twice to obtain a second code output;
after the second code output is subjected to maximum pooling operation, inputting the code level3 layer, and obtaining a third code output through two continuous convolutions;
after the third code output is subjected to maximum pooling operation, inputting the code level4 layer, and obtaining a fourth code output through two continuous convolutions;
and after carrying out maximum pooling operation on the fourth code output, inputting the code level5 layer, and carrying out continuous convolution twice to obtain a water flare characteristic code output graph.
The method of continuous convolution is as follows:
firstly, performing first 3×3 two-dimensional convolution, then performing first batch standardization on the output of the first 3×3 two-dimensional convolution, then inputting the output of the first batch standardization into a first ReLU activation function, then performing second 3×3 two-dimensional convolution on the output of the first ReLU activation function, then performing second batch standardization on the output of the second 3×3 two-dimensional convolution, then inputting the output of the second batch standardization and the output of the first 3×3 two-dimensional convolution together into a residual error module, performing addition operation of a feature map, obtaining residual error output, and finally inputting the residual error output into a second ReLU activation function, thereby obtaining a result of two continuous convolutions.
The method for obtaining the flare detection result graph by inputting the water flare characteristic coding output graph into the decoding unit comprises the following steps:
the decoding unit includes 5 decoding layers: decoding level1 layer, decoding level2 layer, decoding level3 layer, decoding level4 layer, and decoding level5 layer;
inputting the water flare characteristic coding output image into a decoding level5 layer, and obtaining a first decoding output through up-sampling convolution operation;
after the first decoding output and the fourth coding output are connected in a jumping mode in the decoding level4 layer, a CBAM attention mechanism of the decoding level4 layer is input, and after two continuous convolution and up-sampling convolution operations, the output of the CBAM attention mechanism of the decoding level4 layer is subjected to a second decoding output;
After the second decoding output is connected with the third coding output in a jumping way in the decoding level3 layer, the CBAM attention mechanism of the decoding level3 layer is input, and the output of the CBAM attention mechanism of the decoding level3 layer is subjected to two continuous convolution and up-sampling convolution operations to obtain the third decoding output;
after the third decoding output is connected with the second coding output in a jumping way in the decoding level2 layer, the CBAM attention mechanism of the decoding level2 layer is input, and after the output of the CBAM attention mechanism of the decoding level2 layer is subjected to two continuous convolution and up-sampling convolution operations, a fourth decoding output is obtained;
and after the fourth decoding output is connected with the first coding output in a jumping way in the decoding level1 layer, inputting a CBAM attention mechanism of the decoding level1 layer, and after the output of the CBAM attention mechanism of the decoding level1 layer is subjected to two continuous convolutions, inputting 1X 1 convolutions to generate a flare detection result graph with the channel number of 2.
The up-sampling convolution operation method comprises the following steps:
the up-sampling is performed first, followed by a 3 x 3 two-dimensional convolution of the up-sampled output.
The first 3×3 two-dimensional convolution of the coding level1 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 32;
The first 3×3 two-dimensional convolution of the coding level2 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 64;
the first 3×3 two-dimensional convolution of the coding level3 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 128;
the first 3×3 two-dimensional convolution of the coding level4 layer and the convolution kernels of the second 3×3 two-dimensional convolution have 256;
the first 3 x 3 two-dimensional convolution and the second 3 x 3 two-dimensional convolution of the coding level5 layers have 512 convolution kernels.
Specifically, a brand new convolutional neural network is proposed based on the Unet: res_AUnet, as shown in FIG. 4. Because the characteristic of the flare of the water body is relatively simple, the overfitting phenomenon can be caused by the overmany convolution kernels and the continuous convolution in the Unet network, and the detection precision is reduced, so that after multiple experimental verification, the number of the convolution kernels is reduced to half of that of the Unet network, and a residual error module is added between the continuous convolution, as shown in fig. 5. The UNet network adds excessive invalid features during the jump connection of the features, so the application adds CBAM (Convolutional Block Attention Module) attention mechanism (figure 6) after each jump connection of the encoding stage, concentrates on the extraction of flare features in the two directions of a channel and a space, inhibits other invalid non-flare features and improves the detection precision of the network.
The deep neural network model, res_aunet, includes: an encoding unit and a decoding unit;
the coding unit extracts water flare characteristics of the water body image map to obtain a water flare characteristic coding output map;
and inputting the water flare characteristic coding output diagram into the decoding unit to obtain a flare detection result diagram.
The method for extracting the water flare characteristics of the water body image map by the coding unit to obtain the water body flare characteristic coding output map comprises the following steps:
as shown in fig. 4, the coding unit includes 5 coding layers: coding level1 layer, coding level2 layer, coding level3 layer, coding level4 layer and coding level5 layer;
inputting a water body image map with the size of 256 multiplied by 3 into the coding level1 layer, and obtaining a first coding output with the size of 256 multiplied by 32 through two continuous convolutions;
after the first code output is subjected to maximum pooling operation, the first code output is input into the code level2 layer, and a second code output with the size of 128 multiplied by 64 is obtained through two continuous convolutions; the maximum pooling operation reduces the parameter quantity of the network, and after the first coding output is subjected to the maximum pooling operation, the size of the characteristic diagram is changed into half of the first coding output to be 128 multiplied by 32;
After the second coding output is subjected to maximum pooling operation, inputting the coding level3 layer, and carrying out continuous convolution twice to obtain a third coding output with the size of 64 multiplied by 128;
after the third coding output is subjected to maximum pooling operation, inputting the coding level4 layer, and carrying out continuous convolution twice to obtain a fourth coding output with the size of 32 multiplied by 256;
and after the fourth code output is subjected to maximum pooling operation, inputting the code level5 layer, and carrying out continuous convolution twice to obtain a water flare characteristic code output graph, wherein the size of the water flare characteristic code output graph is 16 multiplied by 512.
As shown in fig. 5, the method of continuous convolution is:
firstly, performing first 3×3 two-dimensional convolution, then performing first batch standardization on the output of the first 3×3 two-dimensional convolution, then inputting the output of the first batch standardization into a first ReLU activation function, then performing second 3×3 two-dimensional convolution on the output of the first ReLU activation function, then performing second batch standardization on the output of the second 3×3 two-dimensional convolution, then inputting the output of the second batch standardization and the output of the first 3×3 two-dimensional convolution together into a residual error module, performing addition operation of a feature map, obtaining residual error output, and finally inputting the residual error output into a second ReLU activation function, thereby obtaining a result of two continuous convolutions.
The first 3×3 two-dimensional convolution of the coding level1 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 32;
the first 3×3 two-dimensional convolution of the coding level2 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 64;
the first 3×3 two-dimensional convolution of the coding level3 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 128;
the first 3×3 two-dimensional convolution of the coding level4 layer and the convolution kernels of the second 3×3 two-dimensional convolution have 256;
the first 3 x 3 two-dimensional convolution and the second 3 x 3 two-dimensional convolution of the coding level5 layers have 512 convolution kernels.
The method for obtaining the flare detection result graph by inputting the water flare characteristic coding output graph into the decoding unit comprises the following steps:
as shown in fig. 4, the decoding unit includes 5 decoding layers: decoding level1 layer, decoding level2 layer, decoding level3 layer, decoding level4 layer, and decoding level5 layer;
inputting the water flare characteristic coding output image into a decoding level5 layer, and performing up-sampling convolution operation to obtain a first decoding output with the size of 32 multiplied by 256;
after the first decoding output and the fourth coding output are connected in a jumping mode in the decoding level4 layer, the CBAM attention mechanism of the decoding level4 layer is input, and after the output of the CBAM attention mechanism of the decoding level4 layer is subjected to two continuous convolution and up-sampling convolution operations, a second decoding output with the size of 64 multiplied by 128 is obtained; the jump connection can combine the two feature graphs in channel dimension to obtain the water flare feature of the initial level, so that the detection precision is improved;
After the second decoding output and the third coding output are connected in a jumping manner in the decoding level3 layer, the CBAM attention mechanism of the decoding level3 layer is input, and the output of the CBAM attention mechanism of the decoding level3 layer is subjected to two continuous convolution and up-sampling convolution operations to obtain a third decoding output with the size of 128 multiplied by 64;
after the third decoding output is connected with the second coding output in a jumping way in the decoding level2 layer, the CBAM attention mechanism of the decoding level2 layer is input, and after the output of the CBAM attention mechanism of the decoding level2 layer is subjected to two continuous convolution and up-sampling convolution operations, a fourth decoding output with the size of 256 multiplied by 32 is obtained;
and after the fourth decoding output is connected with the first coding output in a jumping way in the decoding level11 layer, inputting a CBAM attention mechanism of the decoding level1 layer, and after the output of the CBAM attention mechanism of the decoding level1 layer is subjected to two continuous convolutions, inputting 1X 1 convolutions to generate a flare detection result graph with the channel number of 2. In the result graph, one channel represents the probability of predicted flare, the other is the probability of non-flare, the sizes of two channel values of one pixel point are compared through maximum value operation, whether the current pixel point is flare is judged, and the detection of the network model on the flare of the water body is completed.
The up-sampling convolution operation method comprises the following steps:
the up-sampling is performed first, followed by a 3 x 3 two-dimensional convolution of the up-sampled output.
And the CBAM attention mechanism is used for carrying out flare characteristic recombination of channel and space dimensions as shown in fig. 6, acquiring important water flare characteristics and removing irrelevant characteristics except flare so as to further improve detection accuracy. The CBAM attention mechanism consists of a channel attention mechanism and a spatial attention mechanism, and after the features are input into the CBAM attention mechanism, important feature extraction of the channel dimension is performed first, as shown in fig. 7. Important feature extraction of the space dimension is then performed, as shown in fig. 8. In the channel attention mechanism, firstly, global average pooling operation and maximum pooling operation are carried out, two characteristic weight coefficients of channel dimension are obtained, and then multi-layer perceptron operation is carried out, so that the weight coefficients have nonlinear characteristics. And adding the two characteristic weight coefficients, performing Sigmoid activation function operation to obtain weight coefficients of different channels, and multiplying the weight coefficients by the original characteristic map to finally obtain the characteristic map with important information of different channels. In the space attention mechanism, global average pooling operation and maximum pooling operation are carried out, two characteristic weight coefficients of space dimension are obtained, the two weight coefficients are connected in series, and convolution operation is carried out once, so that the space dimension has nonlinear characteristics. And then carrying out Sigmoid activation function operation to obtain weight coefficients of different spatial features, and multiplying the weight coefficients by the original feature map to obtain the feature map with different spatial important information.
And in step S3, training and parameter tuning are performed on the deep neural network model by applying the training set and the verification set.
Specifically, res_aunet is built based on a PyTorch deep learning library, the programming environment is PyCharm, and the programming language is Python. In the model training process, the training period is set to 100 generations, the data input batch size is 10, and the initial learning rate is 0.001. The learning rate is adjusted by adopting an equal interval adjustment strategy (StepLR), and the learning rate is reduced to half of the original learning rate by 10 generations of each iteration.
And S4, verifying the accuracy of the trained deep neural network model by using the test set and the evaluation index.
Specifically, four indices of Precision (P), recall (R), F1 score (F1), and IOU (Intersection over Union) were chosen to verify the accuracy of the proposed model. The calculation methods of the four indices are as follows.
Where TP represents a pixel that is actually flare and the model detects flare, TN represents a pixel that is not flare and the model detects not flare, FP represents a pixel that is not flare but the model detects flare, and FN represents a pixel that is actually flare but the model detects not flare.
In summary, the scheme provided by the invention can provide a Res_AUnet convolutional neural network for flare existing in the visible light image of the unmanned aerial vehicle based on the water body, and can accurately extract the flare. The model is focused on the extraction of the flare of the water body by adding the attention mechanism, so that the false detection phenomenon of other ground objects is reduced. The number of convolution kernels of each layer is properly reduced, and a residual error module is added, so that the overfitting phenomenon is reduced, and the detected water flare is more accurate. Fig. 9 shows water flare detection results of the trained res_aunet network, the AUnet network model and the threshold segmentation method on three randomly selected unmanned aerial vehicle photos. Compared with other methods, the water flare detection method provided by the invention can accurately detect the water flare rapidly and with high precision, avoid false detection and has strong universality. The white area in the figure represents the water flare detected by the neural network model, so that the water flare detected by the Res_AUnet is more complete and accurate, and meanwhile, the false detection phenomenon is less. Table 1 shows the results of comparison of the respective indexes.
TABLE 1
As can be taken from the data in table 1, the res_aunet deep neural network model proposed herein achieves better results, precision improves by 1.07%, recall improves by 1.17%, F1 improves by 2.07%, and IOU improves by 2.06%.
In fig. 10, white represents detected water flare, and it can be seen that the trained res_aunet network model also shows a good detection effect on other unmanned aerial vehicle images, accurately detects water flare, and greatly reduces false detection compared with the Unet network and the threshold segmentation method.
The second aspect of the invention discloses a water flare detection system. FIG. 11 is a block diagram of a water flare detection system according to an embodiment of the present invention; as shown in fig. 11, the system 100 includes:
the first processing module 101 is configured to collect a water body image graph, mark the water body image graph and generate a training set, a verification set and a test set;
a second processing module 102 configured to design a deep neural network model of water flare detection;
a third processing module 103 configured to train and tune the deep neural network model using the training set and the validation set;
a fourth processing module 104 configured to verify the accuracy of the trained deep neural network model using the test set and the evaluation index;
a fifth processing module 105 is configured to apply the trained deep neural network model to perform water flare detection on the acquired single water image.
According to the system of the second aspect of the present invention, the second processing module 102 is configured such that the deep neural network model includes: an encoding unit and a decoding unit;
the coding unit extracts water flare characteristics of the water body image map to obtain a water flare characteristic coding output map;
and inputting the water flare characteristic coding output diagram into the decoding unit to obtain a flare detection result diagram.
According to the system of the second aspect of the present invention, the second processing module 102 is configured to perform, by the encoding unit, water flare feature extraction on the water body image map, and obtaining a water flare feature encoding output map includes:
the coding unit includes 5 coding layers: coding level1 layer, coding level2 layer, coding level3 layer, coding level4 layer and coding level5 layer;
inputting the water body image map into the coding level1 layer, and obtaining a first coding output through two continuous convolutions;
after the first code output is subjected to maximum pooling operation, inputting the code level2 layer, and carrying out continuous convolution twice to obtain a second code output;
after the second code output is subjected to maximum pooling operation, inputting the code level3 layer, and obtaining a third code output through two continuous convolutions;
After the third code output is subjected to maximum pooling operation, inputting the code level4 layer, and obtaining a fourth code output through two continuous convolutions;
and after carrying out maximum pooling operation on the fourth code output, inputting the code level5 layer, and carrying out continuous convolution twice to obtain a water flare characteristic code output graph.
According to the system of the second aspect of the present invention, the second processing module 102 is configured to perform the continuous convolution as:
firstly, performing first 3×3 two-dimensional convolution, then performing first batch standardization on the output of the first 3×3 two-dimensional convolution, then inputting the output of the first batch standardization into a first ReLU activation function, then performing second 3×3 two-dimensional convolution on the output of the first ReLU activation function, then performing second batch standardization on the output of the second 3×3 two-dimensional convolution, then inputting the output of the second batch standardization and the output of the first 3×3 two-dimensional convolution together into a residual error module, performing addition operation of a feature map, obtaining residual error output, and finally inputting the residual error output into a second ReLU activation function, thereby obtaining a result of two continuous convolutions.
According to the system of the second aspect of the present invention, the second processing module 102 is configured to input the water flare feature encoding output map to the decoding unit, and obtaining a flare detection result map includes:
The decoding unit includes 5 decoding layers: decoding level1 layer, decoding level2 layer, decoding level3 layer, decoding level4 layer, and decoding level5 layer;
inputting the water flare characteristic coding output image into a decoding level5 layer, and obtaining a first decoding output through up-sampling convolution operation;
after the first decoding output and the fourth coding output are connected in a jumping mode in the decoding level4 layer, a CBAM attention mechanism of the decoding level4 layer is input, and after two continuous convolution and up-sampling convolution operations, the output of the CBAM attention mechanism of the decoding level4 layer is subjected to a second decoding output;
after the second decoding output is connected with the third coding output in a jumping way in the decoding level3 layer, the CBAM attention mechanism of the decoding level3 layer is input, and the output of the CBAM attention mechanism of the decoding level3 layer is subjected to two continuous convolution and up-sampling convolution operations to obtain the third decoding output;
after the third decoding output is connected with the second coding output in a jumping way in the decoding level2 layer, the CBAM attention mechanism of the decoding level2 layer is input, and after the output of the CBAM attention mechanism of the decoding level2 layer is subjected to two continuous convolution and up-sampling convolution operations, a fourth decoding output is obtained;
And after the fourth decoding output is connected with the first coding output in a jumping way in the decoding level1 layer, inputting a CBAM attention mechanism of the decoding level1 layer, and after the output of the CBAM attention mechanism of the decoding level1 layer is subjected to two continuous convolutions, inputting 1X 1 convolutions to generate a flare detection result graph with the channel number of 2.
According to the system of the second aspect of the present invention, the second processing module 102 is configured to perform the upsampling convolution operation as follows:
the up-sampling is performed first, followed by a 3 x 3 two-dimensional convolution of the up-sampled output.
According to the system of the second aspect of the present invention, the second processing module is configured to 32 convolution kernels of the first 3×3 two-dimensional convolution and the second 3×3 two-dimensional convolution of the coding level1 layer;
the first 3×3 two-dimensional convolution of the coding level2 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 64;
the first 3×3 two-dimensional convolution of the coding level3 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 128;
the first 3×3 two-dimensional convolution of the coding level4 layer and the convolution kernels of the second 3×3 two-dimensional convolution have 256;
the first 3 x 3 two-dimensional convolution and the second 3 x 3 two-dimensional convolution of the coding level5 layers have 512 convolution kernels.
A third aspect of the invention discloses an electronic device. The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the steps in a method for detecting flare of water body according to any one of the first aspect of the disclosure.
Fig. 12 is a block diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 12, the electronic device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface of the electronic device is used for conducting wired or wireless communication with an external terminal, and the wireless communication can be achieved through WIFI, an operator network, near Field Communication (NFC) or other technologies. The display screen of the electronic equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the electronic equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in fig. 12 is merely a block diagram of a portion related to the technical solution of the present disclosure, and does not constitute a limitation of the electronic device to which the technical solution of the present disclosure is applied, and a specific electronic device may include more or less components than those shown in the drawings, or may combine some components, or have different component arrangements.
A fourth aspect of the application discloses a computer-readable storage medium. A computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of a method for detecting water flare in any one of the first aspects of the present disclosure.
Note that the technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be regarded as the scope of the description. The foregoing examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.
Claims (7)
1. A method for detecting water flare, the method comprising:
s1, acquiring a water body image graph, and labeling the water body image graph to generate a training set, a verification set and a test set;
s2, designing a deep neural network model for detecting water flare;
s3, training and parameter adjustment are carried out on the deep neural network model by applying the training set and the verification set;
s4, verifying the accuracy of the trained deep neural network model by using the test set and the evaluation index;
s5, performing water flare detection on the acquired single water body image by applying the trained deep neural network model;
in the step S2, the deep neural network model includes: an encoding unit and a decoding unit;
the coding unit extracts water flare characteristics of the water body image map to obtain a water flare characteristic coding output map;
inputting the water flare characteristic coding output image into the decoding unit to obtain a flare detection result image;
in the step S2, the method for extracting the water flare features from the water body image map by the encoding unit to obtain the water flare feature encoding output map includes:
The coding unit includes 5 coding layers: coding level1 layer, coding level2 layer, coding level3 layer, coding level4 layer and coding level5 layer;
inputting the water body image map into the coding level1 layer, and obtaining a first coding output through two continuous convolutions;
after the first code output is subjected to maximum pooling operation, inputting the code level2 layer, and carrying out continuous convolution twice to obtain a second code output;
after the second code output is subjected to maximum pooling operation, inputting the code level3 layer, and obtaining a third code output through two continuous convolutions;
after the third code output is subjected to maximum pooling operation, inputting the code level4 layer, and obtaining a fourth code output through two continuous convolutions;
after the fourth code output is subjected to maximum pooling operation, inputting the code level5 layer, and carrying out continuous convolution twice to obtain a water flare characteristic code output graph;
in the step S2, the method for obtaining the flare detection result map by inputting the flare feature code output map of the water body into the decoding unit includes:
the decoding unit includes 5 decoding layers: decoding level1 layer, decoding level2 layer, decoding level3 layer, decoding level4 layer, and decoding level5 layer;
Inputting the water flare characteristic coding output image into a decoding level5 layer, and obtaining a first decoding output through up-sampling convolution operation;
after the first decoding output and the fourth coding output are connected in a jumping mode in the decoding level4 layer, a CBAM attention mechanism of the decoding level4 layer is input, and after two continuous convolution and up-sampling convolution operations, the output of the CBAM attention mechanism of the decoding level4 layer is subjected to a second decoding output;
after the second decoding output is connected with the third coding output in a jumping way in the decoding level3 layer, the CBAM attention mechanism of the decoding level3 layer is input, and the output of the CBAM attention mechanism of the decoding level3 layer is subjected to two continuous convolution and up-sampling convolution operations to obtain the third decoding output;
after the third decoding output is connected with the second coding output in a jumping way in the decoding level2 layer, the CBAM attention mechanism of the decoding level2 layer is input, and after the output of the CBAM attention mechanism of the decoding level2 layer is subjected to two continuous convolution and up-sampling convolution operations, a fourth decoding output is obtained;
and after the fourth decoding output is connected with the first coding output in a jumping way in the decoding level1 layer, inputting a CBAM attention mechanism of the decoding level1 layer, and after the output of the CBAM attention mechanism of the decoding level1 layer is subjected to two continuous convolutions, inputting 1X 1 convolutions to generate a flare detection result graph with the channel number of 2.
2. A method of water flare detection according to claim 1, wherein in step S2, the method of continuous convolution is:
first performing a first 3 x 3 two-dimensional convolution, and then performing a first batch normalization on the output of the first 3 x 3 two-dimensional convolution; inputting the first batch of normalized outputs into a first ReLU activation function, performing a second 3×3 two-dimensional convolution on the output of the first ReLU activation function, and performing a second batch of normalization on the output of the second 3×3 two-dimensional convolution; inputting the second batch of standardized output and the first 3×3 two-dimensional convolution output into a residual error module together, and performing addition operation of the feature graphs to obtain residual error output; and finally, inputting the residual output into a second ReLU activation function to obtain a result of two continuous convolutions.
3. A method of water flare detection according to claim 1, wherein in step S2, the method of up-sampling convolution operation is:
the up-sampling is performed first, followed by a 3 x 3 two-dimensional convolution of the up-sampled output.
4. A method of water flare detection as in claim 2, wherein in step S2, the first 3 x 3 two-dimensional convolution of the encoded level1 layer and the convolution kernels of the second 3 x 3 two-dimensional convolution are 32;
The first 3×3 two-dimensional convolution of the coding level2 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 64;
the first 3×3 two-dimensional convolution of the coding level3 layer and the convolution kernels of the second 3×3 two-dimensional convolution are 128;
the first 3×3 two-dimensional convolution of the coding level4 layer and the convolution kernels of the second 3×3 two-dimensional convolution have 256;
the first 3 x 3 two-dimensional convolution and the second 3 x 3 two-dimensional convolution of the coding level5 layers have 512 convolution kernels.
5. A system for water flare detection, the system comprising:
the first processing module is configured to acquire a water body image and label the water body image to generate a training set, a verification set and a test set;
the second processing module is configured to design a deep neural network model for water flare detection;
the deep neural network model includes: an encoding unit and a decoding unit;
the coding unit extracts water flare characteristics of the water body image map to obtain a water flare characteristic coding output map;
inputting the water flare characteristic coding output image into the decoding unit to obtain a flare detection result image;
the encoding unit performs water flare feature extraction on the water body image map, and the obtaining of the water body flare feature encoding output map comprises the following steps:
The coding unit includes 5 coding layers: coding level1 layer, coding level2 layer, coding level3 layer, coding level4 layer and coding level5 layer;
inputting the water body image map into the coding level1 layer, and obtaining a first coding output through two continuous convolutions;
after the first code output is subjected to maximum pooling operation, inputting the code level2 layer, and carrying out continuous convolution twice to obtain a second code output;
after the second code output is subjected to maximum pooling operation, inputting the code level3 layer, and obtaining a third code output through two continuous convolutions;
after the third code output is subjected to maximum pooling operation, inputting the code level4 layer, and obtaining a fourth code output through two continuous convolutions;
after the fourth code output is subjected to maximum pooling operation, inputting the code level5 layer, and carrying out continuous convolution twice to obtain a water flare characteristic code output graph;
inputting the water flare characteristic coding output diagram into the decoding unit, and obtaining a flare detection result diagram comprises the following steps:
the decoding unit includes 5 decoding layers: decoding level1 layer, decoding level2 layer, decoding level3 layer, decoding level4 layer, and decoding level5 layer;
Inputting the water flare characteristic coding output image into a decoding level5 layer, and obtaining a first decoding output through up-sampling convolution operation;
after the first decoding output and the fourth coding output are connected in a jumping mode in the decoding level4 layer, a CBAM attention mechanism of the decoding level4 layer is input, and after two continuous convolution and up-sampling convolution operations, the output of the CBAM attention mechanism of the decoding level4 layer is subjected to a second decoding output;
after the second decoding output is connected with the third coding output in a jumping way in the decoding level3 layer, the CBAM attention mechanism of the decoding level3 layer is input, and the output of the CBAM attention mechanism of the decoding level3 layer is subjected to two continuous convolution and up-sampling convolution operations to obtain the third decoding output;
after the third decoding output is connected with the second coding output in a jumping way in the decoding level2 layer, the CBAM attention mechanism of the decoding level2 layer is input, and after the output of the CBAM attention mechanism of the decoding level2 layer is subjected to two continuous convolution and up-sampling convolution operations, a fourth decoding output is obtained;
after the fourth decoding output is connected with the first coding output in a jumping way in the decoding level1 layer, the CBAM attention mechanism of the decoding level1 layer is input, and after the output of the CBAM attention mechanism of the decoding level1 layer is subjected to two continuous convolutions, the output of the CBAM attention mechanism of the decoding level1 layer is input into a 1X 1 convolution to generate a flare detection result graph with the channel number of 2;
A third processing module configured to train and tune the deep neural network model using the training set and the validation set;
a fourth processing module configured to verify the accuracy of the trained deep neural network model using the test set and the evaluation index;
and the fifth processing module is configured to apply the trained deep neural network model to detect water flare of the acquired single water body image.
6. An electronic device comprising a memory storing a computer program and a processor implementing the steps of a method of water flare detection as claimed in any one of claims 1 to 4 when the computer program is executed by the processor.
7. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of a method for detecting water flare according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310077797.4A CN116206221B (en) | 2023-01-16 | 2023-01-16 | Water flare detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310077797.4A CN116206221B (en) | 2023-01-16 | 2023-01-16 | Water flare detection method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116206221A CN116206221A (en) | 2023-06-02 |
| CN116206221B true CN116206221B (en) | 2023-09-05 |
Family
ID=86508900
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310077797.4A Active CN116206221B (en) | 2023-01-16 | 2023-01-16 | Water flare detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116206221B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117789056B (en) * | 2024-02-27 | 2024-05-07 | 杭州蚁联传感科技有限公司 | Remote sensing data processing method and device with solar flare and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109919123A (en) * | 2019-03-19 | 2019-06-21 | 自然资源部第一海洋研究所 | Offshore spilled oil detection method based on Analysis On Multi-scale Features depth convolutional neural networks |
| CN111563420A (en) * | 2020-04-16 | 2020-08-21 | 自然资源部第一海洋研究所 | Sea surface solar flare area oil spilling multispectral detection method based on convolutional neural network |
| CN112200750A (en) * | 2020-10-21 | 2021-01-08 | 华中科技大学 | Ultrasonic image denoising model establishing method and ultrasonic image denoising method |
| CN113763327A (en) * | 2021-08-10 | 2021-12-07 | 上海电力大学 | CBAM-Res _ Unet-based power plant pipeline high-pressure steam leakage detection method |
| CN113807238A (en) * | 2021-09-15 | 2021-12-17 | 河海大学 | Visual measurement method for area of river surface floater |
-
2023
- 2023-01-16 CN CN202310077797.4A patent/CN116206221B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109919123A (en) * | 2019-03-19 | 2019-06-21 | 自然资源部第一海洋研究所 | Offshore spilled oil detection method based on Analysis On Multi-scale Features depth convolutional neural networks |
| CN111563420A (en) * | 2020-04-16 | 2020-08-21 | 自然资源部第一海洋研究所 | Sea surface solar flare area oil spilling multispectral detection method based on convolutional neural network |
| CN112200750A (en) * | 2020-10-21 | 2021-01-08 | 华中科技大学 | Ultrasonic image denoising model establishing method and ultrasonic image denoising method |
| CN113763327A (en) * | 2021-08-10 | 2021-12-07 | 上海电力大学 | CBAM-Res _ Unet-based power plant pipeline high-pressure steam leakage detection method |
| CN113807238A (en) * | 2021-09-15 | 2021-12-17 | 河海大学 | Visual measurement method for area of river surface floater |
Non-Patent Citations (1)
| Title |
|---|
| CBAM: Convolutional Block Attention Module;Sanghyun Woo等;https://arxiv.org/pdf/1807.06521.pdf;正文第1-17页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116206221A (en) | 2023-06-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111738124B (en) | Remote sensing image cloud detection method based on Gabor transformation and attention | |
| CN111709416B (en) | License plate positioning method, device, system and storage medium | |
| CN109815997A (en) | The method and relevant apparatus of identification vehicle damage based on deep learning | |
| US11308714B1 (en) | Artificial intelligence system for identifying and assessing attributes of a property shown in aerial imagery | |
| CN110059579B (en) | Method and apparatus for in vivo testing, electronic device, and storage medium | |
| CN111626295B (en) | Training method and device for license plate detection model | |
| CN116206221B (en) | Water flare detection method and system | |
| CN108776777A (en) | The recognition methods of spatial relationship between a kind of remote sensing image object based on Faster RCNN | |
| CN110517261A (en) | Seat belt status detection method, device, computer equipment and storage medium | |
| CN109086765B (en) | Licence plate recognition method, device, medium, server and automobile data recorder | |
| CN111144372A (en) | Vehicle detection method, device, computer equipment and storage medium | |
| CN116824413A (en) | Aerial image target detection method based on multi-scale cavity convolution | |
| CN110321867B (en) | Shielded target detection method based on component constraint network | |
| CN116052026A (en) | Unmanned aerial vehicle aerial image target detection method, system and storage medium | |
| CN117422711B (en) | Ocean vortex hyperspectral change detection method, device, equipment and medium | |
| Bhamare et al. | Quality Inspection of Tire using Deep Learning based Computer Vision | |
| CN112990107B (en) | Hyperspectral remote sensing image underwater target detection method and device and computer equipment | |
| CN116665153A (en) | Road scene segmentation method based on improved deep bv3+ network model | |
| CN114511781A (en) | Method, device and medium for identifying disguise personnel by carrying multispectral camera by unmanned aerial vehicle | |
| CN116415019A (en) | Virtual reality VR image recognition method and device, electronic equipment and storage medium | |
| CN113326891A (en) | Method, system and device for detecting small target object | |
| CN117576109B (en) | Defect detection method, device, equipment and storage medium | |
| CN117765680B (en) | Forest fire hazard monitoring and early warning method, device, equipment and storage medium | |
| CN117372789B (en) | Image classification method and image classification device | |
| CN117935087A (en) | Unmanned aerial vehicle radar image target recognition method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |