CN113592786A - Deep learning-based detection method for mesoscale vortices in ocean - Google Patents
Deep learning-based detection method for mesoscale vortices in ocean Download PDFInfo
- Publication number
- CN113592786A CN113592786A CN202110783889.5A CN202110783889A CN113592786A CN 113592786 A CN113592786 A CN 113592786A CN 202110783889 A CN202110783889 A CN 202110783889A CN 113592786 A CN113592786 A CN 113592786A
- Authority
- CN
- China
- Prior art keywords
- mesoscale
- ocean
- deep learning
- convolution
- automatic detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 46
- 238000013135 deep learning Methods 0.000 title claims abstract description 24
- 238000000605 extraction Methods 0.000 claims abstract description 15
- 238000000034 method Methods 0.000 claims description 37
- 230000002159 abnormal effect Effects 0.000 claims description 24
- 230000006870 function Effects 0.000 claims description 23
- 238000005070 sampling Methods 0.000 claims description 19
- 238000012549 training Methods 0.000 claims description 17
- 238000011176 pooling Methods 0.000 claims description 12
- 238000010586 diagram Methods 0.000 claims description 11
- 230000004913 activation Effects 0.000 claims description 6
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 230000008569 process Effects 0.000 description 14
- 230000011218 segmentation Effects 0.000 description 14
- 238000004364 calculation method Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 9
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 238000012795 verification Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000015556 catabolic process Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 230000004931 aggregating effect Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 235000015097 nutrients Nutrition 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 150000003839 salts Chemical class 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30181—Earth observation
- G06T2207/30184—Infrastructure
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses an automatic detection method for marine mesoscale vortexes based on deep learning. Based on the thought of deep learning, the invention constructs an automatic detection network model suitable for detecting the marine mesoscale vortex, abstracts the high-level essential characteristics of the marine mesoscale vortex step by step, emphasizes on solving the key scientific problem of weak characteristic marine remote sensing image information extraction, realizes the automatic and accurate detection of the marine mesoscale vortex without manual interference, saves the manpower and time consumed by detecting the marine mesoscale vortex, and provides reliable technical support for a oceanologist to detect the marine mesoscale vortex by using the marine remote sensing image.
Description
Technical Field
The invention relates to the field of target extraction in deep learning, in particular to a deep learning-based detection method for mesoscale vortexes in ocean.
Background
The mesoscale ocean vortex is an important ocean phenomenon characterized by closed circular current, widely exists in global oceans, has an irregular spiral structure in oceans, has a spatial scale of 10-100 kilometers, and has an influence depth of 1 kilometer in the vertical direction. The mesoscale ocean vortex carries huge energy, can obviously change the vertical distribution of nutrient substances and thermocline, and has very important functions on the distribution of plankton and the transportation of energy and salt. The global climate and ecosystem are deeply influenced by the change of marine substances and energy, so that the realization of the automatic monitoring of the mesoscale vortexes in the sea is not only beneficial to researching the change of the marine climate, but also has important effects on effectively developing and managing marine resources and ensuring the safety of marine environment.
The automatic detection of the mesoscale vortices in the ocean is an important method for monitoring and analyzing the spatiotemporal change characteristics of the mesoscale vortices. The ocean remote sensing satellite has the characteristics of all weather, large area, long distance, non-contact, rapidness, high efficiency and the like when observing ocean phenomena, and the observation method provides abundant data resources for researching ocean mesoscale vortexes. The traditional method for detecting mesoscale vortices in the ocean has the defects. In the traditional physical characteristic-based detection method of the mesoscale vortices in the ocean, such as an Okubo-Weiss (OW) algorithm, in the process of manually designing the physical characteristics of the mesoscale vortices, such as vorticity, amplitude, speed and the like, the detection accuracy of the mesoscale vortices in the ocean is low due to the introduction of a large number of artificial subjective factors; a traditional ocean mesoscale vortex detection method based on flow field geometric characteristics, such as a wining-angle (WA) algorithm, selects closed streamlines and clusters the streamlines to achieve the purpose of detecting vortex, but the method is complex in calculation process, large in calculation amount and lack of generalization capability.
The marine reflection signals are very weak compared with land remote sensing signals, and the spectral characteristic difference of marine phenomena is much smaller than that of land surface objects, so that the characteristic and distinguishability of the marine phenomena on the marine remote sensing images are weak. In addition, many factors influence the ocean remote sensing images, and the same ocean phenomenon has obvious space-time heterogeneity under different sea surface conditions and different water body turbidities, which directly causes the inconsistency of the characteristics of the ocean phenomenon presented on the ocean remote sensing images. The weak characteristics of the ocean remote sensing images exacerbate the limitations of the traditional identification method.
Disclosure of Invention
The invention aims to overcome the problems in the prior art and provides a deep learning-based detection model of mesoscale vortices in the ocean. According to the invention, the high-level essential characteristics of the marine mesoscale vortexes are abstracted step by step, the key scientific problem of weak characteristic marine remote sensing image information extraction is emphatically solved, and the automatic and accurate detection of the marine mesoscale vortexes without manual interference is realized.
The purpose of the invention is realized by the following technical scheme:
a deep learning-based automatic detection method for mesoscale vortices in the ocean comprises the following steps:
s1: acquiring a sea level abnormal image dataset with a marine mesoscale vortex label;
s2: constructing an automatic detection model of the marine mesoscale vortex based on the improved U-Net network; the improved U-Net network includes an encoder portion on the left side and a decoder portion on the right side;
the encoder part is formed by connecting 4 coding blocks; the first coding block takes a sea level abnormal image as input; after the coding block input in each coding block passes through 3 multiplied by 3 convolution layers in sequence, down-sampling is carried out by utilizing the maximum pooling operation with the step length of 2, the down-sampling result is output to the next coding block as input, and simultaneously the down-sampling result and the output of the second 3 multiplied by 3 convolution layer calculate residual error so as to realize residual error learning; the characteristic diagram output by the last coding block is used as the output of the encoder part;
the decoder part is formed by connecting 4 decoding blocks, wherein the decoding block input in each decoding block sequentially passes through 3 multiplied by 3 convolutional layers, then 12 multiplied by 2 transposed convolution operation is utilized to carry out up-sampling to reduce the number of characteristic channels by half, and meanwhile, residual errors are calculated for the output of the two latter 3 multiplied by 3 convolutional layers to realize residual error learning;
the first decoding block takes the output of the encoder section directly as input; for each decoding block except for the first decoding block, respectively acquiring a feature map before downsampling of the corresponding hierarchical coding block from an encoder part by adding a convolution attention module and a jump connection structure, wherein the size of the feature map is the same as that of an upsampling result output by the last decoding block, inputting the feature map into the convolution attention module to perform attention enhancement of two dimensions of a channel and a space, splicing an attention enhancement result and the upsampling result output by the last decoding block through the jump connection structure, and taking the spliced result as the input of the current hierarchical decoding block; taking the up-sampling result of the last decoding block as the output of the decoder part, and outputting an ocean mesoscale vortex extraction result graph with the same size as the input sea level abnormal image after passing through 1 3 × 3 convolutional layer and 1 × 1 convolutional layer;
s3: and training the constructed automatic detection model of the mesoscale vortexes in the sea by using the sea level abnormal image data set, and using the trained automatic detection model of the mesoscale vortexes in the sea to detect the mesoscale vortexes in the sea level abnormal image.
Preferably, the activation function in each of the 3 × 3 convolutional layers is a ReLU activation function.
Preferably, the sea level abnormal image data sets are input into the automatic detection model of the mesoscale eddy in the sea for training in batches as training data.
Preferably, when the automatic detection model for the mesoscale vortices in the ocean is trained, an Adam optimizer is adopted to optimize model parameters by taking a minimum loss function as a target.
Further, the loss function is a cross entropy loss.
Preferably, the convolution attention module is composed of a channel attention module and a spatial attention module.
Further, in the convolution attention module, the number of channels of the channel attention module is 32, and the size of a convolution kernel is 3 × 3; the convolution kernel size in the spatial attention module is 7 × 7.
Preferably, in the automatic detection model for mesoscale ocean eddy, the size of each sea level anomaly image input is 256 pixels × 256 pixels.
Compared with the prior art, the invention has the beneficial effects that:
the invention solves the problems that the traditional detection method for the mesoscale ocean eddy is lack of general ability and sea level abnormal image characteristics are weak, avoids the problems of manually designed characteristics and threshold values based on a deep learning thought, constructs a model suitable for detecting the mesoscale ocean eddy, gradually abstracts and expresses the high-level essence of the mesoscale ocean eddy, introduces a depth characteristic selection method of a convolution attention module into the model, enhances the characteristics of relevant areas, inhibits the characteristics of irrelevant areas, improves the detection capability of a network model on the mesoscale ocean eddy target, and realizes the automatic and accurate detection of the mesoscale ocean eddy. Therefore, the method provided by the invention can save the manpower and time consumed for detecting the mesoscale ocean eddy and provide reliable technical support for oceanologists to detect the mesoscale ocean eddy by using the ocean remote sensing image.
Drawings
FIG. 1 is a flow chart of steps of a method for automatically detecting mesoscale vortices in the ocean based on deep learning;
FIG. 2 is a schematic structural diagram of an automatic detection model of the marine mesoscale vortex based on deep learning according to the present invention;
FIG. 3 is a block diagram of the encoder portion of FIG. 2;
FIG. 4 is a block diagram of a portion of the decoder of FIG. 2;
FIG. 5 is a diagram of a convolution attention module;
FIG. 6 is a process of operation of the channel attention module;
FIG. 7 is a process of operation of the spatial attention module;
FIG. 8 is a schematic diagram of a residual learning mechanism;
FIG. 9 is a segmentation result of the deep learning-based automatic detection model of mesoscale vortices in the ocean of the present invention on a data set.
Detailed Description
The invention is described in further detail below with reference to the figures and specific examples.
The core of the invention is to construct an automatic detection model suitable for the mesoscale ocean vortex, and the model can abstract and express the high-level essential characteristics of the mesoscale ocean vortex step by step based on the deep learning idea so as to realize automatic and accurate detection of the mesoscale ocean vortex. The present invention will be described in detail below.
As shown in fig. 1, in a preferred implementation manner of the present invention, there is provided an automatic detection method for mesoscale vortices in ocean based on deep learning, the method including the following steps S1-S3:
s1: and acquiring a sea level abnormal image data set with the marine mesoscale vortex label. The data set contains a large number of sea level abnormal image samples, and each image sample is manually marked in advance to form corresponding mesoscale ocean eddy marking information. Each image sample is sized to match the subsequent model input. The Sea Level abnormal image may be obtained by preprocessing an msla (maps Sea Level Anomaly large) image, or may be replaced by other image data.
S2: and constructing an automatic detection model of the mesoscale vortex in the ocean based on the improved U-Net network.
As shown in fig. 2, the overall form of the improved U-Net network is substantially similar to that of the conventional U-Net network, and is a U-shaped network structure, the left side of the network is an encoder part, the right side of the network is a decoder part, and a hopping connection structure exists between the encoder part and the decoder part.
Wherein, as shown in fig. 3, the encoder part is formed by connecting 4 encoding blocks. The structure of each coding block is basically similar, the current coding block input in each coding block is sequentially passed through 3 × 3 convolution layers, then the down sampling is carried out by utilizing the maximum pooling operation with the step length of 2, the down sampling result is output to the next coding block as input, and meanwhile, the down sampling result in each coding block and the output of the second 3 × 3 convolution layer need to calculate the residual error so as to realize residual error learning. But the input of 4 encoding blocks has difference, the first encoding block takes the sea level abnormal image transferred by the input layer of the whole improved U-Net network as the input directly, and the rest 3 encoding blocks take the down sampling result output by the above encoding block as the input of the first encoding block. The last coding block is not followed by any more coding blocks, so the feature map of its output, i.e. the down-sampling result, is taken as the output of the whole encoder part, which is subsequently input into the decoder part.
In the encoder part, 4 coding blocks are arranged, and the group number of 3 multiplied by 3 convolution layers in each coding block is set to be 3, so that the detection accuracy is prevented from being reduced due to the over-depth of a network after experimental verification. The convolution layer is used for extracting image characteristics, and the down-sampling layer is used for filtering some unimportant high-frequency information, reducing the number of channels and increasing the receptive field. The repeated convolution and pooling operations can fully extract the high-level features of the image, and the feature map size is reduced by half and the number of channels is doubled every time the feature map passes through one coding block.
Similarly, as shown in fig. 4, the decoder portion is also formed by connecting 4 decoding blocks, the basic structures of the 4 decoding blocks are similar, the current decoding block is input into each decoding block and sequentially passes through 3 × 3 convolutional layers, then 12 × 2 transposed convolution operation is used for up-sampling to reduce the number of characteristic channels by half, and meanwhile, the output of the last two 3 × 3 convolutional layers in each decoding block needs to calculate a residual so as to realize residual learning. But there is also a difference in the input of the 4 decoding blocks, the first decoding block having as input the output of the previous encoder section directly. And for each decoding block except for the first decoding block, respectively acquiring a feature map (marked as a feature map T) before downsampling of the corresponding hierarchical coding block from an encoder part by adding a convolution attention module and a jump connection structure, inputting the feature map T acquired from the encoder part into the convolution attention module to perform attention enhancement of two dimensions of a channel and a space, performing channel splicing on an attention enhancement result and an upsampling result output by the last decoding block (at the moment, the image size and the number of the channels of the two decoding blocks are the same) through the jump connection structure, and taking a splicing result as the input of the current hierarchical decoding block. It should be noted that, for each decoding block, the coding block level of the feature map T obtained by the corresponding convolution attention module needs to be determined according to the size of the feature map before downsampling in the coding block (i.e. the output result of the 3 rd 3 × 3 convolutional layer), and the obtained feature map should have the same size as the upsampling result output by the last decoding block of the current decoding block, so that the subsequent concatenation can be performed.
Since there are no more decoded blocks after the last decoded block, the upsampled result is output as the entire decoder section. Finally, the feature map output by the decoder part is subjected to 1 3 × 3 convolutional layer and 1 × 1 convolutional layer in sequence, the size of the image is restored again, and an ocean mesoscale vortex extraction result map with the same size as the sea level abnormal image input by the model is output.
In the invention, the identification of the mesoscale ocean vortexes is a two-classification process, so that the finally output mesoscale ocean vortexes extraction result graph is also a 2-channel feature graph, and whether each pixel point in the image belongs to the mesoscale ocean vortexes or not can be judged according to the feature graph.
It should be noted that, in the invention, a convolution attention module is added between the encoder and the decoder except the first layer, the salient content and the position feature of the marine mesoscale vortex target are adaptively selected and enhanced from the channel direction and the space direction to the two dimensions, the feature of the relevant region is enhanced, the feature of the irrelevant region is suppressed, and the detection and identification capability of the network model on the target is improved. Except the first decoding block, each of the rest decoding blocks and the coding blocks acquire a characteristic diagram from the corresponding level coding block in the encoder part through a convolution attention module and a jump connection structure to be used as the input of a subsequent convolution layer, and a shallow network is used for storing better detail position information to assist in segmentation so as to reduce the spatial information loss caused by the reduction of resolution ratio during down-sampling. The reason that the jump connection structure and the convolution attention module of the first layer decoding block are eliminated is that the first layer up-sampling does not represent input data in a high-dimensional space, so that the jump connection structure added in the first layer decoding block does not bring obvious effect, but reduces the performance of the model.
It follows that the core in the feature extraction module is the convolution attention module. The specific structure thereof is described in detail below.
(1) Convolution attention module
The convolution attention module is a light and general attention module, can be simply and effectively integrated into most convolution neural network models, can not obviously increase the parameter quantity and the computational complexity of the network while improving the network feature extraction capability, and better improves the performance of the models. The convolution attention module is composed of a channel attention module and a space attention module, and the specific operation process is shown in fig. 5. Given an input feature map F ∈ RC×H×WThe number of channels is C, and the width and height of each channel feature map are W and H, respectively. The CBAM firstly calculates an input feature map F to obtain a one-dimensional channel attention map AC∈RC×1×1Then F and A are addedCObtaining a channel-direction significant feature map F after pixel-level multiplicationC∈R1×H×WThe calculation process is shown as the formula (1.1); then FCCalculating to obtain a two-dimensional space attention diagram AS∈RC×1×1. Finally FCAnd ASOutputting final significant characteristic graph F after pixel-level multiplicationR∈RC×H×WThe calculation process is shown as the formula (1.2),
representing the operation of pixel level multiplication.
The CBAM adaptively selects and enhances the salient content and location features of the target from the channel direction and the space direction of the input feature map to both dimensions. And then multiplying the input feature map by the attention map to obtain a significant feature map subjected to feature selection, enhancing important features and suppressing useless features, so that the network model can pay more attention to content information and position information of the target, and the detection capability of the target is improved.
The channel attention module in the CBAM is interested in the meaningful content information in the input feature map, wherein the feature map on each channel can be regarded as a feature detector, and the operation process is shown in fig. 6.
The channel attention module first generates feature maps for two different spatial contents using maximum pooling and average pooling, respectively, on the input feature map F:
and
maximum pooling is used to aggregate features unique to different targets; the average pooling operation is used for aggregating spatial information and feeding back global information, which are complementary to each other. Then, the two feature maps are simultaneously sent into a shared network containing a Multi-layer Perceptron (MLP), and a channel attention map a is obtained through calculationC. And finally, fusing output vectors of the shared network in a point pixel bitwise addition mode, wherein a specific calculation process is shown as a formula (1.3). The channel attention module effectively reduces information loss in a parallel connection mode, reduces the parameter number by sharing the multilayer perceptron model, and reduces the calculation burden of the network model.
Wherein σ is sigmoid function, MLP stands for multilayer perceptron, and AvgPool and MaxPool are average pooling and maximum pooling operations, respectively. W0∈RC/r×CAnd W1∈RC×C/rRepresenting parameters in a multi-layered perceptron model.
The spatial attention module in the CBAM is primarily concerned with the location information of the target, enabling the network to adaptively select features in the input feature map that characterize the target's content information and its location information. The operation of the spatial attention module is shown in fig. 7.
The spatial attention Module begins with feature map FC∈RC×H×WRespectively using maximum pooling and average pooling to generate feature maps of two different spatial contents
And
then the two feature maps are spliced so as to highlight the target area; finally, the spliced characteristic diagram is converted into a space attention diagram A by utilizing convolution operationS∈R1 ×H×WAnd thus feed back areas that need to be highlighted or suppressed. The specific calculation process is shown as formula (1.4), wherein Conv7×7Representing a 7 x 7 convolution operation.
(2) Residual learning mechanism
In the field of deep learning, increasing the depth of a convolutional neural network can improve the feature extraction capability of the network. However, the deepening network is increasingly difficult to train, and even causes the network performance to be degraded. The improved U-Net network model also has the problem that the model is difficult to train due to the fact that the network is too deep. In order to solve the problem, a residual learning (residual learning) idea is creatively proposed in the prior art, and after a residual learning module is introduced into the ResNet, a deep network can extract complex features and simultaneously prevent gradient from disappearing, so that the overall performance is improved. Therefore, the invention also introduces a residual error learning module in the improved U-Net network, solves the problem of degradation caused by too deep network and prevents the network gradient from disappearing.
The residual learning module is composed of a residual path and an identity connection path, and is one of the basic units constituting the ResNet, as shown in fig. 8. Wherein the residual path includes two sets of convolutional layers and a ReLU (rectified Linear units) activation function. The output result of the residual error learning module is obtained by adding the output results of the residual error path and the identity connection path together. When the residual F (x) is 0, the output of the residual learning module is x, and the model can not be degraded at the moment; in fact, the residual f (x) will not be 0 in most cases, i.e. the residual learning module can learn new features without losing the original features. The Skip Connection structure in the residual learning module does not increase the number of parameters and the computational complexity, and enables the deep gradient to be directly transmitted to the shallow layer through the Skip Connection structure during reverse transmission, so that the network gradient is prevented from disappearing, and the training process of the network is optimized.
S3: the constructed automatic detection model of the mesoscale eddy in the sea needs to be trained by using an abnormal image data set of the sea level. Before training, the loss function in the model training process needs to be defined, and then the model parameters are optimized by adopting an optimizer with the minimum loss function as a target. After the precision of the trained automatic detection model for the mesoscale vortexes in the ocean meets the requirement, the model can be used for detecting the mesoscale vortexes in the ocean from the abnormal image of the sea level.
The Cross-entropy loss function (Cross-entropy loss or Softmax loss) is the most commonly used loss function in classification models. The invention selects a cross entropy function as a loss function of the improved U-Net network model, and a calculation formula of the function is shown as a formula (1.5). The cross entropy loss function ensures that the weight gradient of the last layer in the network model is not related to the derivative of the activation function any more, but is in direct proportion to the difference value between the output value and the true value, thereby accelerating the convergence speed of the network model and accelerating the updating of the whole weight matrix.
Llogistic(y′,y)＝-ylogy′-(1-y)log(1-y′) (1.5)
Where y' and y are the actual and predicted results, respectively.
The following is a description of the effects of the method based on the methods shown in the above steps S1 to S3. The specific process is as described above, and is not described again, and the specific parameter setting and implementation effect are mainly shown below.
Example 1
In this embodiment, a part of the sea area is used as a research area, and the detection of the mesoscale vortices in the sea is performed. The following mainly shows some specific implementation details and effects of each step.
1. Data set production
The spatial resolution of the sea level abnormal image dataset is 0.25 degrees, the time period of the selected data is from 1/1993 to 12/2012/31/20 years in total, the time resolution is 7 days, and 1043 MSLA image samples in total are obtained. The data set takes 18 years of data (i.e. 939 MSLA images) from 1993 to 2010 as a training set, and 104 MSLA images from 2011 and 2012 as a verification set. Each single channel image of a sample in the data set has a size of 256 pixels by 256 pixels, and each image exists as one piece of sample data. The data set comprises a picture set and a tag set, wherein the picture set is used as an input source of the model and is an MSLA picture containing the mesoscale vortex characteristics in the ocean; the label set is the position information and the category information of the mesoscale vortexes in the ocean contained in each picture.
2. Experimental Environment
In the embodiment, an experimental environment is built according to the current mainstream deep learning environment configuration, and the basic system platform configuration is shown in table 1. In addition, the important software configuration of the present invention is shown in table 2.
TABLE 1 basic System platform configuration
TABLE 2 important software configuration
3. Parameter setting
By means of a random search strategy, 3 aspects of model calculation efficiency, result precision and hardware are comprehensively considered, the weight initialization strategy of the training network is set to be random initialization, a cross entropy function is selected as a loss function, an Adam optimizer is adopted to optimize the loss function, and the initial learning rate is set to be 1 e-3. The batch size during neural network training is set to 16, and the number of iterations is set to 500. The number of channels of the channel attention module in the convolution attention module is 32, and the size of a convolution kernel is 3 multiplied by 3; the convolution kernel size in the spatial attention module is 7 × 7. After iterative training, the network finally converges.
4. Evaluation index
And sequentially comparing pixel points in the marine mesoscale vortex image detected based on the MSLA remote sensing image with pixel points in the marine mesoscale vortex data set label image, wherein if the network segmentation result is consistent with the comparison and segmentation standard result, the image is True Positive (TP), and if the network result is wrong with the comparison and segmentation standard result, the image is False Positive (FP). For non-mesoscale vortex pixels in the image, if the network segmentation result contrast is correct, the network segmentation result contrast is True Negative (TN), and if the network segmentation result contrast is wrong, the network segmentation result contrast is False Negative (FN).
TABLE 3 confusion matrix
In order to accurately and effectively evaluate the experimental result, the invention adopts four common indexes in the segmentation field based on the pixel level to evaluate the experimental result, wherein the four indexes comprise Accuracy (Accuracy), Precision (Precision), Recall (Recall) and F1-Score (Dice coefficient), and the four indexes are specifically defined as follows:
(1) accuracy (Accuracy): represents the weight of the classifier judging correctly for the whole sample, i.e. the ratio of the number of correctly judged pixels to the total number of pixels.
(2) Precision (Precision): the specific gravity of the positive sample in the positive case is determined by the classifier.
(3) Recall (Recall): refers to the proportion of total positive examples that are predicted to be positive examples.
(4)F1-Score：
5. Analysis of Experimental results
In order to prove that the model provided by the invention has advancement and robustness in the aspect of extracting the mesoscale vortices in the ocean, the model (Our Method) is compared with the EddyNet model provided by Lguensat and the like, the U-Net model provided by Ronneberger and the like and the extraction effect of different structures of the model provided by the invention on the mesoscale vortices in the ocean, and the segmentation performance of the network is comprehensively compared and analyzed in the aspects of Accuracy (Accuracy), Precision (Precision), Recall (Recall), F1-Score, training time consumption, segmentation time consumption and the like. From the segmentation effect comparison table 4 below it can be seen that: the algorithm of the invention obtains better results than the EddyNet algorithm in the aspects of accuracy, precision, recall ratio and F1-Score. In addition, compared with the classical U-Net + residual error module and the classical U-Net + CBAM attention module, the two modules provided by the invention respectively improve the identification performance of the model to different degrees, which shows the effectiveness of the attention mechanism and the residual error learning on improving the extraction precision of the mesoscale vortex in the ocean. Through iterative training, the model of the invention obtains 96.33% of classification precision on a training set and 93.88% of classification precision on a verification set. The improved U-Net model introduces a CBAM attention mechanism and a residual error learning module, so that a network can extract more complex high-level essential features, the model degradation is avoided, the gradient disappearance is prevented, and the detection precision is improved.
TABLE 4 comparison of segmentation effects of five networks on datasets
In addition, the two sets of experimental results of fig. 9(a) and (b) correspond to the mesoscale vortex extraction cases at week 18 and week 27 in the validation set, respectively. The picture in the 1 st column is a sea level height anomaly picture, and the pictures in the 2 nd, the 3 rd and the 4 th columns respectively correspond to a vortex extraction reference true value, an EddyNet method and a mesoscale vortex detection result picture obtained by the method. Observing the area outlined by the black square frame in the two groups of experimental results of fig. 9(a) and (b), compared with the sea level height abnormal contour map of the area, the method of the invention obtains vortex extraction results which are more consistent with the sea level height abnormal contour map, and the EddyNet method detects a plurality of false vortices, which are caused by the fact that the network structure of the EddyNet method is too simple, and the method of the invention can more cautiously utilize more context information to capture vortex details. Therefore, the method has good mesoscale vortex detection effect and has more potential in detecting mesoscale vortices.
The above-described embodiments are merely preferred embodiments of the present invention, which should not be construed as limiting the invention. Various changes and modifications may be made by one of ordinary skill in the pertinent art without departing from the spirit and scope of the present invention. Therefore, the technical scheme obtained by adopting the mode of equivalent replacement or equivalent transformation is within the protection scope of the invention.
Claims (8)
1. A deep learning-based automatic detection method for mesoscale vortices in the ocean is characterized by comprising the following steps:
s1: acquiring a sea level abnormal image dataset with a marine mesoscale vortex label;
s2: constructing an automatic detection model of the marine mesoscale vortex based on the improved U-Net network; the improved U-Net network includes an encoder portion on the left side and a decoder portion on the right side;
the encoder part is formed by connecting 4 coding blocks; the first coding block takes a sea level abnormal image as input; after the coding block input in each coding block passes through 3 multiplied by 3 convolution layers in sequence, down-sampling is carried out by utilizing the maximum pooling operation with the step length of 2, the down-sampling result is output to the next coding block as input, and simultaneously the down-sampling result and the output of the second 3 multiplied by 3 convolution layer calculate residual error so as to realize residual error learning; the characteristic diagram output by the last coding block is used as the output of the encoder part;
the decoder part is formed by connecting 4 decoding blocks, wherein the decoding block input in each decoding block sequentially passes through 3 multiplied by 3 convolutional layers, then 12 multiplied by 2 transposed convolution operation is utilized to carry out up-sampling to reduce the number of characteristic channels by half, and meanwhile, residual errors are calculated for the output of the two latter 3 multiplied by 3 convolutional layers to realize residual error learning;
the first decoding block takes the output of the encoder section directly as input; for each decoding block except for the first decoding block, respectively acquiring a feature map before downsampling of the corresponding hierarchical coding block from an encoder part by adding a convolution attention module and a jump connection structure, wherein the size of the feature map is the same as that of an upsampling result output by the last decoding block, inputting the feature map into the convolution attention module to perform attention enhancement of two dimensions of a channel and a space, splicing an attention enhancement result and the upsampling result output by the last decoding block through the jump connection structure, and taking the spliced result as the input of the current hierarchical decoding block; taking the up-sampling result of the last decoding block as the output of the decoder part, and outputting an ocean mesoscale vortex extraction result graph with the same size as the input sea level abnormal image after passing through 1 3 × 3 convolutional layer and 1 × 1 convolutional layer;
s3: and training the constructed automatic detection model of the mesoscale vortexes in the sea by using the sea level abnormal image data set, and using the trained automatic detection model of the mesoscale vortexes in the sea to detect the mesoscale vortexes in the sea level abnormal image.
2. The method for automatically detecting mesoscale vortices in the ocean based on deep learning of claim 1 wherein the activation function in each of the 3 x 3 convolutional layers is a ReLU activation function.
3. The deep learning-based automatic detection method for mesoscale vortices in ocean according to claim 1, wherein the sea level abnormal image dataset is input into an automatic detection model for mesoscale vortices in ocean for training in batches as training data.
4. The deep learning-based automatic detection method for mesoscale vortices in sea as claimed in claim 1, wherein during training of the automatic detection model for mesoscale vortices in sea, an Adam optimizer is used to optimize model parameters with a goal of minimizing a loss function.
5. The method of automatic detection of mesoscale vortices in the ocean based on deep learning of claim 4 wherein the loss function is cross entropy loss.
6. The method for automatically detecting mesoscale vortices in the ocean based on deep learning of claim 1 wherein the convolution attention module is composed of a channel attention module and a spatial attention module.
7. The method for automatically detecting the mesoscale vortices in the ocean based on the deep learning of claim 6, wherein in the convolution attention module, the number of channels of the channel attention module is 32, and the size of a convolution kernel is 3 x 3; the convolution kernel size in the spatial attention module is 7 × 7.
8. The method for automatically detecting mesoscale ocean vortex based on deep learning of claim 1, wherein in the model for automatically detecting mesoscale ocean vortex, the size of each abnormal image of the sea level is 256 pixels by 256 pixels.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110783889.5A CN113592786B (en) | 2021-07-12 | 2021-07-12 | Deep learning-based ocean mesoscale vortex detection method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110783889.5A CN113592786B (en) | 2021-07-12 | 2021-07-12 | Deep learning-based ocean mesoscale vortex detection method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113592786A true CN113592786A (en) | 2021-11-02 |
| CN113592786B CN113592786B (en) | 2023-12-05 |
Family
ID=78246784
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110783889.5A Active CN113592786B (en) | 2021-07-12 | 2021-07-12 | Deep learning-based ocean mesoscale vortex detection method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113592786B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114022774A (en) * | 2022-01-10 | 2022-02-08 | 航天宏图信息技术股份有限公司 | Radar image-based marine mesoscale vortex monitoring method and device |
| CN114299377A (en) * | 2021-12-31 | 2022-04-08 | 中山大学 | Vortex identification method and device based on width learning |
| CN115953394A (en) * | 2023-03-10 | 2023-04-11 | 中国石油大学(华东) | Target segmentation-based detection method and system for mesoscale ocean vortexes |
| CN117422711A (en) * | 2023-12-14 | 2024-01-19 | 武汉理工大学三亚科教创新园 | Ocean vortex hyperspectral change detection method, device, equipment and medium |
| CN117935023B (en) * | 2024-03-22 | 2024-05-28 | 青岛哈尔滨工程大学创新发展中心 | Method and system for identifying mesoscale vortex and analyzing features based on attention mechanism |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111767827A (en) * | 2020-06-28 | 2020-10-13 | 中国人民解放军国防科技大学 | Mesoscale vortex identification method based on deep learning |
| CN112102325A (en) * | 2020-09-17 | 2020-12-18 | 中国科学院海洋研究所 | Ocean abnormal mesoscale vortex identification method based on deep learning and multi-source remote sensing data |
| AU2020103905A4 (en) * | 2020-12-04 | 2021-02-11 | Chongqing Normal University | Unsupervised cross-domain self-adaptive medical image segmentation method based on deep adversarial learning |
| WO2021119366A1 (en) * | 2019-12-12 | 2021-06-17 | FLIR Belgium BVBA | Deep learning-based marine object classification using 360-degree images |
-
2021
- 2021-07-12 CN CN202110783889.5A patent/CN113592786B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021119366A1 (en) * | 2019-12-12 | 2021-06-17 | FLIR Belgium BVBA | Deep learning-based marine object classification using 360-degree images |
| CN111767827A (en) * | 2020-06-28 | 2020-10-13 | 中国人民解放军国防科技大学 | Mesoscale vortex identification method based on deep learning |
| CN112102325A (en) * | 2020-09-17 | 2020-12-18 | 中国科学院海洋研究所 | Ocean abnormal mesoscale vortex identification method based on deep learning and multi-source remote sensing data |
| AU2020103905A4 (en) * | 2020-12-04 | 2021-02-11 | Chongqing Normal University | Unsupervised cross-domain self-adaptive medical image segmentation method based on deep adversarial learning |
Non-Patent Citations (1)
| Title |
|---|
| 黎安舟;周为峰;范秀梅: "遥感图像中尺度海洋锋及涡旋提取方法研究进展", 中国图象图形学报, vol. 22, no. 6 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114299377A (en) * | 2021-12-31 | 2022-04-08 | 中山大学 | Vortex identification method and device based on width learning |
| CN114299377B (en) * | 2021-12-31 | 2024-03-22 | 中山大学 | Vortex identification method and device based on width learning |
| CN114022774A (en) * | 2022-01-10 | 2022-02-08 | 航天宏图信息技术股份有限公司 | Radar image-based marine mesoscale vortex monitoring method and device |
| CN115953394A (en) * | 2023-03-10 | 2023-04-11 | 中国石油大学(华东) | Target segmentation-based detection method and system for mesoscale ocean vortexes |
| CN117422711A (en) * | 2023-12-14 | 2024-01-19 | 武汉理工大学三亚科教创新园 | Ocean vortex hyperspectral change detection method, device, equipment and medium |
| CN117422711B (en) * | 2023-12-14 | 2024-03-26 | 武汉理工大学三亚科教创新园 | Ocean vortex hyperspectral change detection method, device, equipment and medium |
| CN117935023B (en) * | 2024-03-22 | 2024-05-28 | 青岛哈尔滨工程大学创新发展中心 | Method and system for identifying mesoscale vortex and analyzing features based on attention mechanism |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113592786B (en) | 2023-12-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113592786A (en) | Deep learning-based detection method for mesoscale vortices in ocean | |
| CN112668494A (en) | Small sample change detection method based on multi-scale feature extraction | |
| CN112017192B (en) | Glandular cell image segmentation method and glandular cell image segmentation system based on improved U-Net network | |
| CN112102324A (en) | Remote sensing image sea ice identification method based on depth U-Net model | |
| CN112132012B (en) | High-resolution SAR ship image generation method based on generation countermeasure network | |
| CN113408398B (en) | Remote sensing image cloud detection method based on channel attention and probability up-sampling | |
| CN112561876A (en) | Image-based pond and reservoir water quality detection method and system | |
| CN115565056A (en) | Underwater image enhancement method and system based on condition generation countermeasure network | |
| CN112765556A (en) | Method for evaluating tree age of new-born mangrove forest based on intensive time remote sensing data | |
| CN115131313A (en) | Hyperspectral image change detection method and device based on Transformer | |
| Wang et al. | An unsupervised heterogeneous change detection method based on image translation network and post-processing algorithm | |
| Zhou et al. | Water–land segmentation via structure-aware CNN–transformer network on large-scale SAR data | |
| CN117217368A (en) | Training method, device, equipment, medium and program product of prediction model | |
| CN116580243A (en) | Cross-domain remote sensing scene classification method for mask image modeling guide domain adaptation | |
| CN114937154B (en) | Significance detection method based on recursive decoder | |
| CN113313668B (en) | Subway tunnel surface disease feature extraction method | |
| CN113343861A (en) | Neural network model-based remote sensing image water body region extraction method | |
| Chen et al. | A novel split-frequency feature fusion framework for processing the dual-optical images of offshore oil spills | |
| Zheng et al. | Utilizing residual networks for remote sensing estimation of total nitrogen concentration in Shandong offshore areas | |
| CN115409842B (en) | Remote sensing image unsupervised change detection method based on convolution self-encoder-decoder | |
| CN114373118B (en) | Underwater target detection method based on improved YOLOV4 | |
| Feng et al. | An underwater image enhancement strategy based on pyramid attention mechanism | |
| CN111784180B (en) | Analysis and interpretation method for evaluation results of criminal reconstruction quality evaluation system | |
| CN116879192B (en) | Water bloom prediction method, device, equipment and medium based on satellite remote sensing data | |
| CN117409307A (en) | Hidden signal detection method under sea clutter background |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |