CN117292144A - Sonar image simulation method based on generation countermeasure network - Google Patents
Sonar image simulation method based on generation countermeasure network Download PDFInfo
- Publication number
- CN117292144A CN117292144A CN202311374902.7A CN202311374902A CN117292144A CN 117292144 A CN117292144 A CN 117292144A CN 202311374902 A CN202311374902 A CN 202311374902A CN 117292144 A CN117292144 A CN 117292144A
- Authority
- CN
- China
- Prior art keywords
- module
- image
- feature map
- layer
- resolution image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 37
- 238000004088 simulation Methods 0.000 title claims abstract description 34
- 238000012549 training Methods 0.000 claims abstract description 36
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 90
- 230000004913 activation Effects 0.000 claims description 43
- 238000010586 diagram Methods 0.000 claims description 33
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 23
- 238000000605 extraction Methods 0.000 claims description 18
- 238000005070 sampling Methods 0.000 claims description 14
- 239000013598 vector Substances 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 10
- 238000010606 normalization Methods 0.000 claims description 7
- 238000009826 distribution Methods 0.000 claims description 4
- 230000008485 antagonism Effects 0.000 claims description 3
- 230000003213 activating effect Effects 0.000 claims 1
- 230000015572 biosynthetic process Effects 0.000 description 5
- 238000003786 synthesis reaction Methods 0.000 description 5
- 238000010276 construction Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000003321 amplification Effects 0.000 description 2
- 230000003042 antagnostic effect Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/20—Drawing from basic elements, e.g. lines or circles
- G06T11/203—Drawing of straight lines or curves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2218/00—Aspects of pattern recognition specially adapted for signal processing
- G06F2218/08—Feature extraction
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a sonar image simulation method based on a generation countermeasure network, which comprises the following specific steps: s1, taking an original sonar image as a high-resolution image, preprocessing the original sonar image to obtain a low-resolution image, and forming a data set by the high-resolution image and the low-resolution image; s2: creating a generated countermeasure network model, wherein the generated countermeasure network model comprises a sonar image generation network module and a sonar image discrimination network module, the sonar image generation network module generates a super-resolution image based on a low-resolution image in the data set, and the sonar image discrimination network module is used for outputting a discrimination result based on the generated super-resolution image and the high-resolution image in the data set; s3: training the generated countermeasure network model based on the data set to obtain a trained generated countermeasure network model; s4: and simulating a sonar image based on the generated countermeasure network model after training. The invention can improve the definition of the existing sonar image through the constructed generation countermeasure network model and generate the sonar image with higher resolution.
Description
Technical Field
The invention relates to the technical field of sonar image simulation, in particular to a sonar image simulation method based on a generation countermeasure network.
Background
The study of image simulation has undergone various stages from simple line segment, regular shape synthesis, to regular image synthesis, such as texture image and face image synthesis, to complex natural image synthesis, such as picture synthesis in dataset ImageNet, from the 60 th century. Along with the gradual increase of the image data volume, the computing capability of modern computers is improved, the natural image simulation technology is mature, the quality of the simulated natural image is improved, but the sonar image is rare because of the complexity of the underwater environment, and the simulation technology of the sonar image is still deficient.
The purpose of the submarine sonar image simulation is to research a method capable of generating submarine sonar images with high quality. Image generation is in fact a probabilistic modeling of the image, which can be generalized to the application of generating models, most of which are based on maximum likelihood estimation, with appropriate correct parameters calculated for the selected model so that the data likelihood function value in the training data set is maximized. The traditional sonar image simulation method utilizes a computer simulation technology to carry out modeling simulation according to an imaging mechanism and image characteristics of the sonar, so that the two simulation methods are established on the basis of model construction, professional field knowledge is required, when a scene where a simulation object is located lacks an available geometric model, the two traditional methods are difficult to generate a sonar image, the simulation effect has great dependence on the accuracy of model construction and the selection of parameters, and the simulation object is difficult to adjust parameters due to the factors such as various structures, complex imaging process, clutter and the like, so that the quality of the generated sonar image is low.
Disclosure of Invention
The invention provides a sonar image simulation method based on a generation countermeasure network, which aims to solve the problems that when image simulation is carried out in the prior art, the simulation result has larger dependence on the accuracy of model construction and the selection of parameters, and the parameters of a simulation model are influenced by a simulation object and are difficult to adjust, so that the quality of the generated sonar image is lower.
In order to achieve the above object, the technical scheme of the present invention is as follows:
a sonar image simulation method based on a generated countermeasure network comprises the following specific steps:
s1, taking an original sonar image as a high-resolution image, preprocessing the original sonar image to obtain a low-resolution image, and forming a data set by the high-resolution image and the low-resolution image;
s2: creating a generated countermeasure network model, wherein the generated countermeasure network model comprises a sonar image generation network module and a sonar image discrimination network module, the sonar image generation network module generates a super-resolution image based on a low-resolution image in the data set, and the sonar image discrimination network module is used for outputting a discrimination result based on the generated super-resolution image and the high-resolution image in the data set;
s3: training the generated countermeasure network model based on the data set to obtain a trained generated countermeasure network model;
s4: and simulating a sonar image based on the generated countermeasure network model after training.
Specifically, in S2, the sonar image generation network module includes a first input module, a feature extraction module, a feature learning module, and a first output module;
the first input module is used for extracting the edge and texture characteristics of the input low-resolution image to output a first feature map, and transmitting the first feature map to the feature extraction module;
the feature extraction module is used for extracting the edge and texture features of the first feature map to output a second feature map, and transmitting the second feature map to the feature learning module;
the feature learning module is used for extracting the shape of the second feature map and the overall structural feature of the object to output a third feature map, and transmitting the third feature map to the first output module;
the first output module is used for up-sampling the third feature map to output a super-resolution image, and transmitting the generated super-resolution image to the sonar image discrimination network module;
the sonar image discrimination network module comprises a second input module, a plurality of convolution modules and a second output module;
the second input module is used for extracting the edges and texture features of the super-resolution image and the high-resolution image in the data set, respectively outputting a fourth feature map and a fifth feature map, and transmitting the fourth feature map and the fifth feature map to the convolution module;
the convolution module is used for extracting the shape of the fourth characteristic diagram and the fifth characteristic diagram and the integral structure characteristic of the object, respectively outputting a sixth characteristic diagram and a seventh characteristic diagram, and transmitting the sixth characteristic diagram and the seventh characteristic diagram to the second output module;
the second output module is used for calculating Wasserstein distance between the sixth characteristic diagram and the seventh characteristic diagram.
Specifically, the first input module comprises a first convolution layer and a first PReLU activation function layer;
the feature extraction module comprises a multi-scale convolution module, wherein the multi-scale convolution module comprises a first branch, a second branch, a third branch, a fourth branch and a fifth branch;
the first branch comprises a 1 multiplied by 1 first branch convolution layer and a first branch cavity convolution layer, and is used for extracting the edge and texture characteristics of the first feature map, wherein the size of the receptive field of the edge and texture characteristics is in a range of 3 multiplied by 3;
the second branch comprises a 1 multiplied by 1 second branch convolution layer, a second branch asymmetric convolution layer and a second branch cavity convolution layer, and is used for extracting the edge and texture characteristics of the first feature map, wherein the size of the edge and texture characteristics is 9 multiplied by 9;
the third branch comprises a 1 multiplied by 1 third branch convolution layer, a third branch asymmetric convolution layer and a third branch cavity convolution layer, and is used for extracting the edge and texture characteristics of the first feature map, wherein the size of the edge and texture characteristics is 9 multiplied by 9;
the fourth branch comprises a 1 multiplied by 1 fourth branch convolution layer, a fourth branch asymmetric convolution layer and a fourth branch cavity convolution layer, and is used for extracting the edge and texture characteristics of the first feature map, wherein the size of the edge and texture characteristics is 15 multiplied by 15;
the fifth branch performs splicing operation on the feature graphs output by the four branches along the channel dimension to obtain feature graphs with the number of 2 times of original channels, then adjusts the feature graphs with the number of 2 times of original channels into feature graphs with the number of 1 times of original channels by using a 1X 1 fifth branch convolution layer, sums the feature graphs with the first feature graphs along the channel dimension through jump connection, and outputs a second feature graph through a ReLU activation function layer;
the feature learning module comprises L densely connected residual blocks, L is more than or equal to 2, each residual block comprises a second convolution layer and a second PReLU activation function layer, a first residual block takes the second feature map as input, an L-th residual block takes a feature map which is output after summation operation is carried out on the second feature map and feature maps which are respectively output by the L-1 residual blocks, and a third feature map is output;
the first output module comprises a second convolution block, a plurality of up-sampling modules and a third convolution block;
the second convolution block comprises a third convolution layer and a first BN layer, the third convolution layer maps the third feature map to the high-resolution image in the data set according to a rule that texture, edge and context information of the third feature map correspond to the high-resolution image, and the first BN layer is used for carrying out batch normalization processing on output of the third convolution layer, outputting a fourth feature map and transmitting the fourth feature map to the up-sampling module;
each up-sampling module comprises a fourth convolution layer, a sub-pixel convolution layer and a third PReLU activation function layer, wherein the fourth convolution layer is used for splitting and outputting r in a fourth characteristic diagram 2 A low resolution feature map, the subpixel convolution layer being based on r 2 Sub-pixel convolution of a low-resolution feature map to generate a feature map with a size r 2 R is the magnification factor, and then the characteristic diagram is transmitted to the third convolution block after passing through the third PReLU activation function layer;
the third convolution block comprises a fifth convolution layer and is used for converting abstract high-level features contained in the high-resolution feature map output by the up-sampling module into pixel representations of final output images and outputting super-resolution images, wherein the abstract high-level features comprise shape and appearance features related to object types;
the second input module comprises a sixth convolution layer and a fourth PReLU activation function layer, wherein the sixth convolution layer is used for extracting the edges and texture features of the super-resolution image and the high-resolution image in the data set, and then outputting a fourth feature map and a fifth feature map through the fourth PReLU activation function layer;
each convolution module comprises a seventh convolution layer, a second BN layer and a first leak ReLU activation function layer, wherein the seventh convolution layer is used for extracting the shapes and the integral structure characteristics of objects of the fourth characteristic image and the fifth characteristic image, carrying out batch normalization processing through the second BN layer, and outputting the sixth characteristic image and the seventh characteristic image after the second BN layer passes through the first leak ReLU activation function layer;
the second output module comprises a first full-connection layer, a second leak ReLU activation function layer and a second full-connection layer which are sequentially connected, the first full-connection layer is used for outputting feature vectors of a sixth feature map and a seventh feature map, and the second leak ReLU activation function layer adds the feature vectors of the sixth feature map and the seventh feature map pixel by pixel and then outputs the feature vectors; and the second full connection layer calculates a Wasserstein distance based on the feature vector output by the second Leaky ReLU activation function layer, namely a real value and outputs the real value.
Specifically, in S3, the specific training step of generating the countermeasure network model includes:
s31: initializing the sonar image generation network module and the sonar image discrimination network module, inputting the low-resolution image in the data set into the sonar image generation network module, generating a super-resolution image through the sonar image generation network module, training the sonar image discrimination network module based on the super-resolution image and the high-resolution image in the data set, minimizing an antagonism loss function of the sonar image discrimination network module, and updating parameters of the sonar image discrimination network module;
s32: training the sonar image generation network module based on the updated parameters of the sonar image discrimination network module, minimizing a content loss function of the sonar image generation network module, and updating the parameters of the sonar image generation network module;
s33: and repeating the training process, and when the loss function formed by the countermeasure loss function and the content loss function converges, ending the training to obtain a generated countermeasure network model after the training is finished.
Specifically, in S31, when the sonar image discrimination network module performs training based on the super-resolution image and the high-resolution image in the dataset, the difference between the high-resolution image in the dataset and the generated super-resolution image distribution is determined by determining the wasperstein distance between the two, and a counterloss function is set, where the counterloss function is expressed as:
in the method, in the process of the invention,for Wasserstein distance, +.>For gradient penalty, D (I HR ) A discrimination network score representing a high resolution sonar image; d (G (I) LR ) A discrimination network score, lambda, representing the generated super-resolution image GP Weight coefficient representing gradient penalty, E is the expected value operator, ++>Is to judge the high resolution image I of the network module HR Output of (1) is related to I HR 1 is used to reference the target value of the gradient norm.
Specifically, in S32, based on the updated parameters of the sonar image discrimination network module, the sonar image generation network module is trained, feature extraction is performed on the high-resolution image in the data set and the generated super-resolution image through the pretrained VGG19 network, and content loss of the generated super-resolution image and the high-resolution image in the data set is calculated, where a content loss function is set as follows:
wherein G (I) LR ) Is to generate super-resolution image generated by network, I LR And I HR Representing a low resolution image and a high resolution image, respectively, W i,j And H i,j For the size of the feature map of the jth convolutional layer before the i-th layer is maximally pooled in the VGG19 network, Φ represents the feature extraction function in the VGG19 network, and x and y represent the coordinate positions on the feature map.
Specifically, in S33, when the loss function composed of the countermeasure loss function and the content loss function converges, the training ends, the loss function is expressed as:
the beneficial effects are that: according to the invention, the countermeasure network model is constructed and generated for sonar image simulation, compared with the traditional method, the method is independent of a data set, and a high-quality image can be generated even if the data set is absent, so that the original sonar image data set is expanded, and compared with the traditional method, the method of deep learning is used, and the dependence on manual adjustment parameters is reduced. Meanwhile, the definition of the existing sonar image is improved, and a sonar image with higher resolution can be generated. The problem that the quality of a generated sonar image is low due to the fact that when image simulation is carried out through the prior art, the simulation result is high in selection dependence on accuracy and parameters of model construction, and parameters of a simulation model are influenced by a simulation object and are difficult to adjust is solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it will be obvious that the drawings in the following description are some embodiments of the present invention, and that other drawings can be obtained according to these drawings without inventive effort to a person skilled in the art.
FIG. 1 is a schematic diagram of a structure for generating an countermeasure network model in the present invention;
FIG. 2 is a schematic diagram of a low resolution image in a dataset according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a super-resolution image generated by generating a super-resolution image after processing an antagonistic network model in an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
GAN is a powerful data generator, with the improvement of future network architecture and algorithm, GAN will be expected to generate higher quality images, music, video and text, will inject new vitality into the fields of natural language processing and computer vision, GAN is still a continuous development direction, how to increase the diversity of the generated samples, and how to explore more effective evaluation methods is an important challenge for GAN, so how to use deep learning generation algorithm to realize the generation of clearer sonar image samples which can be directly used by the original sonar image is a problem to be solved.
The embodiment provides a sonar image simulation method based on a generation countermeasure network, which comprises the following specific steps:
s1, taking an original sonar image as a high-resolution image, preprocessing the original sonar image to obtain a low-resolution image, and forming a data set by the high-resolution image and the low-resolution image;
s11: acquiring a sample image dataset, processing the dataset into a mat file format by adopting an acoustic match dataset of an underwater robot large race, reading a bmp image and storing the bmp image into an x variable, respectively storing tag information and position information into a y variable and a digitsruct variable, performing bicubic interpolation on the sample image in the dataset as a high-resolution image to obtain a low-resolution image, and forming a dataset of the low-resolution image and the high-resolution image, wherein in the embodiment, the resolution of the high-resolution image is 1024×1024, and the resolution of the low-resolution image is 500×500;
s2: creating a generated countermeasure network model, wherein the generated countermeasure network model comprises a sonar image generation network module and a sonar image discrimination network module, the sonar image generation network module generates a super-resolution image based on a low-resolution image in the data set, and the sonar image discrimination network module is used for outputting a discrimination result based on the generated super-resolution image and the high-resolution image in the data set;
specifically, in S2, as shown in fig. 1, the sonar image generation network module includes a first input module, a feature extraction module, a feature learning module, and a first output module;
the first input module is used for extracting the edge and texture characteristics of the input low-resolution image to output a first feature map, and transmitting the first feature map to the feature extraction module;
specifically, the first input module includes a 9×9 first convolution layer and a first pralu activation function layer;
the feature extraction module is used for extracting the edge and texture features of the first feature map to output a second feature map, and transmitting the second feature map to the feature learning module;
specifically, the feature extraction module comprises a multi-scale convolution module, wherein the multi-scale convolution module comprises a first branch, a second branch, a third branch, a fourth branch and a fifth branch, the first branch comprises a 1×1 first branch convolution layer and a 3×3 first branch cavity convolution layer, the expansion rate of the cavity convolution layer is 1, and the multi-scale convolution module is used for extracting the edge and texture features of the first feature map, wherein the receptive field size of the edge and texture features is in a 3×3 range;
the second branch comprises a 1 multiplied by 1 second branch convolution layer, a 1 multiplied by 3 second branch asymmetric convolution layer and a 3 multiplied by 3 second branch cavity convolution layer, the expansion rate of the cavity convolution layer is 3, and the second branch convolution layer is used for extracting the edge and texture characteristics of the first feature map, wherein the size of the receptive field of the edge and texture characteristics is in the range of 9 multiplied by 9;
the third branch comprises a third branch convolution layer with the size of 1 multiplied by 1, a third branch asymmetric convolution layer with the size of 3 multiplied by 1 and a third branch cavity convolution layer with the size of 3 multiplied by 3, wherein the expansion rate of the cavity convolution layer is 3 and the third branch convolution layer is used for extracting the edge and texture characteristics of the first characteristic diagram with the receptive field size of 9 multiplied by 9;
the fourth branch comprises a 1 multiplied by 1 fourth branch convolution layer, two 1 multiplied by 3 fourth branch asymmetric convolution layers and a 3 multiplied by 3 fourth branch cavity convolution layer, the expansion rate of the cavity convolution layer is 5, and the expansion rate is used for extracting the edge and texture characteristics of the first feature map with the receptive field size of 15 multiplied by 15;
the fifth branch performs splicing operation on the feature graphs output by the four branches along the channel dimension to obtain feature graphs with the number of 2 times of original channels, then adjusts the feature graphs with the number of 2 times of original channels into feature graphs with the number of 1 times of original channels by using a 1X 1 convolution layer in the fifth branch, sums the feature graphs with the first feature graphs along the channel dimension through jump connection, and outputs a second feature graph through a ReLU activation function layer;
specifically, by introducing the multi-scale convolution module, the embodiment can better reserve the detailed information of the small object in the large-size picture, and ensure the quality of the generated image.
The feature learning module is used for extracting the shape of the second feature map and the overall structural feature of the object to output a third feature map, and transmitting the third feature map to the first output module;
specifically, the feature learning module comprises L densely connected residual blocks, L is more than or equal to 2, each residual block comprises a second convolution layer and a second PReLU activation function layer, a first residual block takes the second feature map as input, a second residual block takes a feature map output after summation operation of the second feature map and an output feature map of the first residual block as input, and an L residual block takes a feature map output after summation operation of the second feature map and feature maps respectively output by the L-1 residual blocks as input and outputs a third feature map;
specifically, the feature learning module in this embodiment includes 16 densely connected residual blocks, each residual block includes a 3×3 convolution layer, a second pralu activation function layer, and a 3×3 convolution layer that are sequentially connected, in this embodiment, the residual block structure is modified, a BN layer in the residual block is removed, noise is reduced, the repeated utilization rate of features is increased, and resource consumption of a single residual block is reduced, and the output of the L-th residual block is subjected to batch normalization processing through the first BN layer after the convolution layer processing, so that the contrast of an image can be stretched, and model overfitting is prevented.
The first output module is used for up-sampling the third feature map to output a super-resolution image, and transmitting the generated super-resolution image to the sonar image discrimination network module;
specifically, the first output module comprises a second convolution block, a plurality of up-sampling modules and a third convolution block;
the second convolution block comprises a 3×3 third convolution layer and a first BN layer, the third convolution layer maps the third feature map to the high-resolution image in the dataset according to a rule that enables texture, edge and context information of the third feature map to correspond to the high-resolution image, so as to ensure that the texture, edge and context information of the third feature map correspond to the high-resolution image, and the first BN layer is used for carrying out batch normalization processing on the output of the third convolution layer and then outputting a fourth feature map and transmitting the fourth feature map to the up-sampling module;
each up-sampling module comprises a 3×3 fourth convolution layer, two sub-pixel convolution layers and a third PReLU activation function layer, wherein the fourth convolution layer is used for splitting and outputting a fourth characteristic image 2 A low resolution feature map, the subpixel convolution layer being based on r 2 Sub-pixel convolution of a low-resolution feature map to generate a feature map with a size r 2 R is the amplification factor, then the amplification factor is transmitted to the third convolution block after passing through the third PReLU activation function layer, the third PReLU activation function layer processes the input pixel by pixel, and a negative slope is introduced when the input is smaller than zero so as to avoid the generation of dead neurons; in this embodiment, preferably, two upsampling modules are included;
the third convolution block comprises 3 fifth convolution layers of 9×9, and is configured to convert abstract high-level features contained in the high-resolution feature map into pixel representations of final output images and output 3-channel super-resolution images, where the abstract high-level features include shape and appearance features related to object types;
in this embodiment, preferably r=2, and 2 is generated by the fourth convolution layer 2 A low-resolution characteristic diagram of the number of channels, which divides the characteristic channel of one pixel into 2 2 Sub-channels are sequentially rearranged, so that a high-resolution characteristic image is obtained, the high-resolution characteristic image is processed through 3 fifth convolution layers of 9 multiplied by 9, a super-resolution image of the 3 channels is output, and the generated image size is 4 times that of the original image.
The sonar image discrimination network module comprises a second input module, a plurality of convolution modules and a second output module;
the second input module is used for extracting the edges and texture features of the super-resolution image and the high-resolution image in the data set, respectively outputting a fourth feature map and a fifth feature map, and transmitting the fourth feature map and the fifth feature map to the convolution module;
specifically, the second input module includes a 3×3 sixth convolution layer and a fourth pralu activation function layer, where the sixth convolution layer is configured to extract edges and texture features of the super-resolution image and the high-resolution image in the dataset, and output a fourth feature map and a fifth feature map through the fourth pralu activation function layer, where the fourth pralu activation function layer is configured to introduce a negative slope, so that the network can adapt to different types of features more flexibly; the method comprises the steps of carrying out a first treatment on the surface of the
The convolution module is used for extracting the shape of the fourth characteristic diagram and the fifth characteristic diagram and the integral structure characteristic of the object, respectively outputting a sixth characteristic diagram and a seventh characteristic diagram, and transmitting the sixth characteristic diagram and the seventh characteristic diagram to the second output module;
specifically, each convolution module includes a 3×3 seventh convolution layer, a second BN layer, and a first leak ReLU activation function layer, where the seventh convolution layer is configured to extract shapes and overall structural features of objects of the fourth feature map and the fifth feature map, perform batch normalization processing through the second BN layer, and output the sixth feature map and the seventh feature map after passing through the first leak ReLU activation function layer, and preferably, the embodiment includes 7 convolution modules;
the second output module is used for calculating Wasserstein distance between the sixth characteristic diagram and the seventh characteristic diagram.
The second output module comprises a first full-connection layer, a second leak ReLU activation function layer and a second full-connection layer which are sequentially connected, the first full-connection layer is used for outputting the feature vectors of 1 multiplied by 1024 of a sixth feature map and a seventh feature map, the second leak ReLU activation function layer is used for adding the feature vectors of the sixth feature map and the seventh feature map pixel by pixel and then outputting a feature vector of 1 multiplied by 1024, and the second leak Relu activation function is used for introducing nonlinearity and simultaneously can perform back propagation even for a negative input value by utilizing good retention characteristics of the second leak ReLU activation function in a negative number region; the second full-connection layer is used for multiplying the feature vector output by the second leak ReLU activation function layer by the weight obtained in the training process to obtain a Wasserstein distance, namely a real value, and outputting the Wasserstein distance. In this embodiment, the weight is initialized to 0.01, and the subsequent weight is obtained through learning by network training.
The conventional discriminator network module for generating the countermeasure network inputs the output result into a Sigmoid activation function to obtain a probability value, which is used to determine whether the input image belongs to a real sample or a probability of a fake sample generated by the generator, and for some input values, a derivative of the Sigmoid activation function is close to 0, so that gradient disappearance can block gradient propagation, so that the Sigmoid activation function is not used in the embodiment, and a difference between the real image and the generated image, that is, a wasperstein distance between two image distributions is output through the discriminator.
S3: training the generated countermeasure network model based on the data set to obtain a trained generated countermeasure network model;
s4: and simulating a sonar image based on the generated countermeasure network model after training.
Specifically, in S3, the specific training step of generating the countermeasure network model includes:
s31: initializing the sonar image generation network module and the sonar image discrimination network module, inputting the low-resolution image in the data set into the sonar image generation network module, outputting a super-resolution image through the sonar image generation network module, training the sonar image discrimination network module based on the super-resolution image and the high-resolution image in the data set, minimizing the antagonism loss function of the sonar image discrimination network module, and updating the parameters of the sonar image discrimination network module;
specifically, in S31, when the sonar image discrimination network module performs training based on the super-resolution image and the high-resolution image in the dataset, the difference between the high-resolution image in the dataset and the generated super-resolution image distribution is determined by determining the wasperstein distance between the two, and a counterloss function is set, where the counterloss function is expressed as:
in the method, in the process of the invention,for Wasserstein distance, +.>For gradient penalty, D (I HR ) A discrimination network score representing a high resolution sonar image; d (G (I) LR ) A discrimination network score, lambda, representing the generated super-resolution image GP Weight coefficient representing gradient penalty, E is the expected value operator, ++>Is to judge the high resolution image I of the network module HR Output of (1) is related to I HR 1 is used to reference the target value of the gradient norm.
S32: training the sonar image generation network module based on the updated parameters of the sonar image discrimination network module, minimizing a content loss function of the sonar image generation network module, and updating the parameters of the sonar image generation network module;
specifically, in S32, based on the updated parameters of the sonar image discrimination network module, the sonar image generation network module is trained, feature extraction is performed on the high-resolution image in the data set and the generated super-resolution image through the pretrained VGG19 network, and content loss of the generated super-resolution image and the high-resolution image in the data set is calculated, where a content loss function is set as follows:
wherein G (I) LR ) Is to generate super-resolution image generated by network, I LR And I HR Representing a low resolution image and a high resolution image, respectively, W i,j And H i,j For the size of the feature map of the jth convolutional layer before the i-th layer is maximally pooled in the VGG19 network, Φ represents the feature extraction function in the VGG19 network, and x and y represent the coordinate positions on the feature map.
S33: and repeating the training process, and when the loss function formed by the countermeasure loss function and the content loss function converges, ending the training to obtain a generated countermeasure network model after the training is finished.
Specifically, in the embodiment, the Wasserstein distance and gradient penalty improvement loss function is used as the counterloss, so that the stability of training is ensured, and the problems of model collapse and vibration in the training process are reduced.
Specifically, in S33, when the loss function composed of the countermeasure loss function and the content loss function converges, the training ends, the loss function is expressed as:
minimizing the loss function is used to train the generating network module to generate high quality super-resolution images, wherein the counterloss function is used to encourage the generating network module to generate realistic images, while the content loss function ensures that the generated images retain important structural and semantic features. Balancing the demands of the generating network module in terms of image quality and visual details by weighted addition of the counterloss function and the content loss function, increasing the weight of the content loss if visual details of the generated image are more focused; if the reality and fidelity of the generated image are more of a concern, the weight against the loss is increased.
In this embodiment, the learning rate is set to r=0.002, the batch size is set to m=4, the iteration number is set to epochs=100, and the iteration number n of the network is determined once per iteration of the network critic =5, gradient penalty factor λ GP =10; as shown in fig. 2, which is a low-resolution image in the dataset in the present embodiment, fig. 3 is a high-resolution image generated by generating the antagonistic network model in the present embodiment, and it is seen from the image that the sonar image generated by the method is improved in detail, image quality and sharpness compared with the original image, and can be applied.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention.
Claims (7)
1. A sonar image simulation method based on a generation countermeasure network is characterized by comprising the following specific steps:
s1, taking an original sonar image as a high-resolution image, preprocessing the original sonar image to obtain a low-resolution image, and forming a data set by the high-resolution image and the low-resolution image;
s2: creating a generated countermeasure network model, wherein the generated countermeasure network model comprises a sonar image generation network module and a sonar image discrimination network module, the sonar image generation network module generates a super-resolution image based on a low-resolution image in the data set, and the sonar image discrimination network module is used for outputting a discrimination result based on the generated super-resolution image and the high-resolution image in the data set;
s3: training the generated countermeasure network model based on the data set to obtain a trained generated countermeasure network model;
s4: and simulating a sonar image based on the generated countermeasure network model after training.
2. The sonar image simulation method based on the generation countermeasure network according to claim 1, wherein in S2, the sonar image generation network module includes a first input module, a feature extraction module, a feature learning module and a first output module;
the first input module is used for extracting the edge and texture characteristics of the input low-resolution image to output a first feature map, and transmitting the first feature map to the feature extraction module;
the feature extraction module is used for extracting the edge and texture features of the first feature map to output a second feature map, and transmitting the second feature map to the feature learning module;
the feature learning module is used for extracting the shape of the second feature map and the overall structural feature of the object to output a third feature map, and transmitting the third feature map to the first output module;
the first output module is used for up-sampling the third feature map to output a super-resolution image, and transmitting the generated super-resolution image to the sonar image discrimination network module;
the sonar image discrimination network module comprises a second input module, a plurality of convolution modules and a second output module;
the second input module is used for extracting the edges and texture features of the super-resolution image and the high-resolution image in the data set, respectively outputting a fourth feature map and a fifth feature map, and transmitting the fourth feature map and the fifth feature map to the convolution module;
the convolution module is used for extracting the shape of the fourth characteristic diagram and the fifth characteristic diagram and the integral structure characteristic of the object, respectively outputting a sixth characteristic diagram and a seventh characteristic diagram, and transmitting the sixth characteristic diagram and the seventh characteristic diagram to the second output module;
the second output module is used for calculating Wasserstein distance between the sixth characteristic diagram and the seventh characteristic diagram.
3. The method for generating a sonar image simulation based on a countermeasure network according to claim 2, wherein the first input module includes a first convolution layer and a first prime activating function layer;
the feature extraction module comprises a multi-scale convolution module, wherein the multi-scale convolution module comprises a first branch, a second branch, a third branch, a fourth branch and a fifth branch;
the first branch comprises a 1 multiplied by 1 first branch convolution layer and a first branch cavity convolution layer, and is used for extracting the edge and texture characteristics of the first feature map, wherein the size of the receptive field of the edge and texture characteristics is in a range of 3 multiplied by 3;
the second branch comprises a 1 multiplied by 1 second branch convolution layer, a second branch asymmetric convolution layer and a second branch cavity convolution layer, and is used for extracting the edge and texture characteristics of the first feature map, wherein the size of the edge and texture characteristics is 9 multiplied by 9;
the third branch comprises a 1 multiplied by 1 third branch convolution layer, a third branch asymmetric convolution layer and a third branch cavity convolution layer, and is used for extracting the edge and texture characteristics of the first feature map, wherein the size of the edge and texture characteristics is 9 multiplied by 9;
the fourth branch comprises a 1 multiplied by 1 fourth branch convolution layer, a fourth branch asymmetric convolution layer and a fourth branch cavity convolution layer, and is used for extracting the edge and texture characteristics of the first feature map, wherein the size of the edge and texture characteristics is 15 multiplied by 15;
the fifth branch performs splicing operation on the feature graphs output by the four branches along the channel dimension to obtain feature graphs with the number of 2 times of original channels, then adjusts the feature graphs with the number of 2 times of original channels into feature graphs with the number of 1 times of original channels by using a 1X 1 fifth branch convolution layer, sums the feature graphs with the first feature graphs along the channel dimension through jump connection, and outputs a second feature graph through a ReLU activation function layer;
the feature learning module comprises L densely connected residual blocks, L is more than or equal to 2, each residual block comprises a second convolution layer and a second PReLU activation function layer, a first residual block takes the second feature map as input, an L-th residual block takes a feature map which is output after summation operation is carried out on the second feature map and feature maps which are respectively output by the L-1 residual blocks, and a third feature map is output;
the first output module comprises a second convolution block, a plurality of up-sampling modules and a third convolution block;
the second convolution block comprises a third convolution layer and a first BN layer, the third convolution layer maps the third feature map to the high-resolution image in the data set according to a rule that texture, edge and context information of the third feature map correspond to the high-resolution image, and the first BN layer is used for carrying out batch normalization processing on output of the third convolution layer, outputting a fourth feature map and transmitting the fourth feature map to the up-sampling module;
each up-sampling module comprises a fourth convolution layer, a sub-pixel convolution layer and a third PReLU activation function layer, wherein the fourth convolution layer is used for splitting and outputting r in a fourth characteristic diagram 2 A low resolution feature map, the subpixel convolution layer being based on r 2 Sub-pixel convolution of a low-resolution feature map to generate a feature map with a size r 2 R is the magnification factor, and then the characteristic diagram is transmitted to the third convolution block after passing through the third PReLU activation function layer;
the third convolution block comprises a fifth convolution layer and is used for converting abstract high-level features contained in the high-resolution feature map output by the up-sampling module into pixel representations of final output images and outputting super-resolution images, wherein the abstract high-level features comprise shape and appearance features related to object types;
the second input module comprises a sixth convolution layer and a fourth PReLU activation function layer, wherein the sixth convolution layer is used for extracting the edges and texture features of the super-resolution image and the high-resolution image in the data set, and then outputting a fourth feature map and a fifth feature map through the fourth PReLU activation function layer;
each convolution module comprises a seventh convolution layer, a second BN layer and a first LeakyReLU activation function layer, wherein the seventh convolution layer is used for extracting the shapes and the integral structure characteristics of objects of the fourth characteristic image and the fifth characteristic image, carrying out batch normalization processing through the second BN layer, and outputting the sixth characteristic image and the seventh characteristic image after the second BN layer passes through the first LeakyReLU activation function layer;
the second output module comprises a first full-connection layer, a second leak ReLU activation function layer and a second full-connection layer which are sequentially connected, the first full-connection layer is used for outputting feature vectors of a sixth feature map and a seventh feature map, and the second leak ReLU activation function layer adds the feature vectors of the sixth feature map and the seventh feature map pixel by pixel and then outputs the feature vectors; and the second full connection layer calculates a Wasserstein distance based on the feature vector output by the second Leaky ReLU activation function layer, namely a real value and outputs the real value.
4. A sonar image simulation method based on generating an countermeasure network according to claim 3, wherein in S3, the specific training step of generating the countermeasure network model includes:
s31: initializing the sonar image generation network module and the sonar image discrimination network module, inputting the low-resolution image in the data set into the sonar image generation network module, generating a super-resolution image through the sonar image generation network module, training the sonar image discrimination network module based on the super-resolution image and the high-resolution image in the data set, minimizing an antagonism loss function of the sonar image discrimination network module, and updating parameters of the sonar image discrimination network module;
s32: training the sonar image generation network module based on the updated parameters of the sonar image discrimination network module, minimizing a content loss function of the sonar image generation network module, and updating the parameters of the sonar image generation network module;
s33: and repeating the training process, and when the loss function formed by the countermeasure loss function and the content loss function converges, ending the training to obtain a generated countermeasure network model after the training is finished.
5. The sonar image simulation method based on the generated countermeasure network according to claim 4, wherein in S31, when the sonar image discrimination network module trains based on the super-resolution image and the high-resolution image in the dataset, the difference between the high-resolution image in the dataset and the generated super-resolution image distribution is judged by judging the waserstein distance between them, and a countermeasure loss function is set, and the countermeasure loss function is expressed as:
in the method, in the process of the invention,for Wasserstein distance, +.>For gradient penalty, D (I HR ) A discrimination network score representing a high resolution sonar image; d (G (I) LR ) A discrimination network score, lambda, representing the generated super-resolution image GP Weight coefficient representing gradient penalty, E is the expected value operator, ++>Is to judge the high resolution image I of the network module HR Output of (1) is related to I HR 1 is used to reference the target value of the gradient norm.
6. The sonar image simulation method based on the generation countermeasure network according to claim 5, wherein in S32, based on the updated parameters of the sonar image discrimination network module, the sonar image generation network module is trained, the feature extraction is performed on the high resolution image in the data set and the generated super resolution image through the pretrained VGG19 network, the content loss of the generated super resolution image and the high resolution image in the data set is calculated, and the content loss function is set as follows:
wherein G (I) LR ) Is to generate super-resolution image generated by network, I LR And I HR Representing a low resolution image and a high resolution image, respectively, W i,j And H i,j For the size of the feature map of the jth convolutional layer before the i-th layer is maximally pooled in the VGG19 network, Φ represents the feature extraction function in the VGG19 network, and x and y represent the coordinate positions on the feature map.
7. The sonar image simulation method based on generating an countermeasure network according to claim 6, wherein in S33, when a loss function composed of the countermeasure loss function and the content loss function converges, the training ends, the loss function is expressed as:
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311374902.7A CN117292144A (en) | 2023-10-23 | 2023-10-23 | Sonar image simulation method based on generation countermeasure network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311374902.7A CN117292144A (en) | 2023-10-23 | 2023-10-23 | Sonar image simulation method based on generation countermeasure network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117292144A true CN117292144A (en) | 2023-12-26 |
Family
ID=89248039
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311374902.7A Pending CN117292144A (en) | 2023-10-23 | 2023-10-23 | Sonar image simulation method based on generation countermeasure network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117292144A (en) |
-
2023
- 2023-10-23 CN CN202311374902.7A patent/CN117292144A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113240580B (en) | Lightweight image super-resolution reconstruction method based on multi-dimensional knowledge distillation | |
| CN113673307B (en) | Lightweight video action recognition method | |
| CN112052886B (en) | Intelligent human body action posture estimation method and device based on convolutional neural network | |
| CN109949255B (en) | Image reconstruction method and device | |
| CN112489164B (en) | Image coloring method based on improved depth separable convolutional neural network | |
| CN109447897B (en) | Real scene image synthesis method and system | |
| CN112184582B (en) | Attention mechanism-based image completion method and device | |
| CN109961397B (en) | Image reconstruction method and device | |
| CN114494003B (en) | Ancient character generation method combining shape transformation and texture transformation | |
| CN113379606B (en) | Face super-resolution method based on pre-training generation model | |
| CN113096015B (en) | Image super-resolution reconstruction method based on progressive perception and ultra-lightweight network | |
| Liu et al. | Facial image inpainting using multi-level generative network | |
| CN112686817B (en) | Image completion method based on uncertainty estimation | |
| CN117593187A (en) | Remote sensing image super-resolution reconstruction method based on meta-learning and transducer | |
| CN116797681A (en) | Text-to-image generation method and system for progressive multi-granularity semantic information fusion | |
| CN116342385A (en) | Training method and device for text image super-resolution network and storage medium | |
| CN113111906B (en) | Method for generating confrontation network model based on condition of single pair image training | |
| CN117292144A (en) | Sonar image simulation method based on generation countermeasure network | |
| CN114926591A (en) | Multi-branch deep learning 3D face reconstruction model training method, system and medium | |
| CN113269815A (en) | Deep learning-based medical image registration method and terminal | |
| CN114782961B (en) | Character image augmentation method based on shape transformation | |
| CN114881843B (en) | Fluid artistic control method based on deep learning | |
| CN114494007A (en) | Text-guided natural image super-resolution reconstruction method | |
| Shah | An efficient framework for single image super-resolution | |
| CN116452687A (en) | Image generation method for fine-grained text based on contrast learning and dynamic convolution |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |