CN115019107A - Method, system and medium for sonar simulation image generation based on style transfer - Google Patents

Abstract

The invention discloses a sonar simulation image generation method, system and medium based on style transfer, wherein the method includes: according to the target object category included in the real sonar image, taking the satellite remote sensing image as input, and recognizing the satellite remote sensing image, Identify the same type of object as the target object, and mark the range of the object; perform image segmentation on the identified and marked satellite remote sensing images, and use the target object category as the classification to construct a satellite sub-image dataset; construct a style transfer network , taking the real sonar image as the style image, and performing the style transfer on the segmented satellite sub-image to generate the sonar simulation image as the training sample image. The present invention is based on remote sensing satellite images, uses a style transfer network to obtain simulated sonar images, can effectively expand the number of sonar data sets, and provides a basis for subsequent related work based on sonar images. The invention can be widely used in the technical field of image processing.

Description

Method, system and medium for sonar simulation image generation based on style transfer

technical field

The invention relates to the technical field of image processing, in particular to a method, system and medium for generating a sonar simulation image based on style transfer.

Background technique

As a high-resolution, multi-purpose, low-cost underwater imaging device, sonar is widely used in oceans, rivers, lakes and other waters. Because it can obtain high-resolution, continuous seabed images, it is widely used in marine mapping, offshore exploration and underwater search and rescue and other fields. In underwater search and rescue activities, sonar can effectively detect underwater targets such as aircraft wreckage and sunken ships. During long underwater search and rescue missions, rescuers need to continuously and carefully check sonar images to confirm whether there is a target object. After working for a period of time, the staff will experience visual fatigue and easily miss the rescue target. In order to effectively reduce the workload of staff, reduce misjudgment caused by visual fatigue, and improve work efficiency, it is of practical significance to automatically classify sonar images. However, the prior art has the following defects:

In recent years, the high-accuracy underwater image automatic classification algorithms all use deep neural network algorithms, which require a large amount of sonar image data for training. However, because sonar images are often related to emergency rescue or national defense, it is difficult to obtain data sets, and there is still no public, unified large-scale sonar images. This makes the convolutional neural network unable to obtain a wide range of applications in the field of sonar.

SUMMARY OF THE INVENTION

In order to solve one of the technical problems existing in the prior art at least to a certain extent, the purpose of the present invention is to provide a method, system and medium for sonar simulation image generation based on style transfer.

The technical scheme adopted in the present invention is:

A method for generating sonar simulation images based on style transfer, comprising the following steps:

According to the target object category contained in the real sonar image, the satellite remote sensing image is used as the input, the satellite remote sensing image is identified, the identification object of the same category as the target object is identified, and the range of the identification object is marked;

Perform image segmentation on the identified and marked satellite remote sensing images, and construct satellite sub-image datasets based on the category of target objects;

A style transfer network is constructed, using real sonar images as style images, and performing style transfer on the segmented satellite sub-images to generate sonar simulation images as training sample images.

Further, identifying the satellite remote sensing image, identifying the identifier of the same type as the target object, and marking the range where the identifier is located, including:

The target recognition network trained by the preset data set is used to detect the satellite remote sensing images, and the detection results are divided into n+1 categories, where n is the number of target object categories;

Mark the area where the target is located, wherein the resolution of the marked area is adjusted according to different input images.

Further, image segmentation is performed on the identified and marked satellite remote sensing images, and the target object category is used as a classification to construct a satellite sub-image data set, including:

After marking the identifier, obtain the target area, read the parameters x, y, w, h of the target area, where x, y are the horizontal and vertical coordinates of the upper left corner of the target area, and w, h are the horizontal and vertical dimensions of the target area ;

According to the read parameters, adaptive resolution segmentation is performed on the target area, and the segmented satellite sub-image area is:

为由原始图片的横纵方向尺度所决定的自适应分辨率缩放因子；

为原始标记的分割范围，

为自适应分辨率放缩以后的分割范围。in,

is an adaptive resolution scaling factor determined by the horizontal and vertical dimensions of the original image;

is the segmentation range of the original mark,

Segmentation range after scaling for adaptive resolution.

Further, the adaptive resolution scaling factor λ is obtained in the following manner:

Among them, f(w,h) is the smaller side of w,h.

Further, the style transfer network uses an empty space pyramid structure to extract the style feature map.

Further, the style transfer network is constructed, and the real sonar image is used as the style image to perform style transfer on the segmented satellite sub-images to generate sonar simulation images as training sample images, including:

Construct a convolutional neural network, use the convolutional neural network to extract the features of real sonar images and satellite sub-images, and optimize the loss function with the Euclidean distance between the features of the two images and the Euclidean distance of the Gram matrix to obtain sonar. Simulation image.

Further, the described construction convolutional neural network, utilizes the described convolutional neural network to extract the features of real sonar images and satellite sub-images, and takes the Euclidean distance between the two picture features and the Euclidean distance of the Gram matrix as the loss function Optimized for sonar simulation images, including:

Based on the first 17 layers of the VGG16 network, shortcut connections are added to the first 17 layers of the VGG16 to build a convolutional neural network;

The calculation of the convolutional neural network after adding the shortcut is expressed as:

x _l+1 = x _l +F(x _l )

In the formula, x _l and x _l+1 represent the characteristics of the lth layer and the l+1th layer of the network respectively; F(x _l ) represents the convolution and activation operations of the network; the network backpropagation process after adding the shortcut is:

The first 17 layers of the trained VGG16 network are used as the backbone of the convolutional neural network, the real sonar image is used as the style image, and the satellite sub-image is used as the content image to input the VGG16 network with shortcut, and the style feature S and content feature are extracted respectively. C;

Taking the Euclidean distance between the style feature S and the content feature C as the content loss, the content loss is expressed as:

where i represents the i-th element in the feature map;

The style loss function is:

是风格特征S(也称风格图片S)在CNN第l层第(i,j,k)位置的输出，其中(i,j,k)对应高，宽，通道三个维度；in the formula

is the output of the style feature S (also known as the style image S) at the (i, j, k) position of the lth layer of the CNN, where (i, j, k) corresponds to the three dimensions of height, width and channel;

是风格图片在CNN第l层第(i,j,k′)位置的输出，其中(i,j,k′)对应高，宽，通道三个维度；

is the output of the style image at the (i, j, k') position of the lth layer of the CNN, where (i, j, k') corresponds to the three dimensions of height, width and channel;

为是网络生成图像G在CNN第l层第(i,j,k)位置的输出，其中(i,j,k)对应高，宽，通道三个维度；

It is the output of the image G generated by the network at the (i, j, k) position of the lth layer of the CNN, where (i, j, k) corresponds to the three dimensions of height, width and channel;

为CNN第l层高度维度上的总数；

is the total number on the height dimension of the lth layer of CNN;

为CNN第l层宽度维度上的总数；

is the total number on the width dimension of the lth layer of CNN;

为CNN第l层通道维度上的总数；

is the total number on the channel dimension of the first layer of CNN;

为CNN第l层的风格特征S在通道k^′和通道k的格拉姆矩阵；

is the Gram matrix of the style feature S of the lth layer of CNN in channel k ^′ and channel k;

为CNN第l层的内容特征C在通道k^′和通道k的格拉姆矩阵；

is the Gram matrix of the content feature C of the lth layer of CNN in channel k ^' and channel k;

The loss function for style transfer is:

L=αL _c +βL _s

where α and β are weight coefficients.

Another technical scheme adopted by the present invention is:

A sonar simulation image generation system based on style transfer, comprising:

The satellite image processing module is used to identify the satellite remote sensing image according to the target object category contained in the real sonar image, and use the satellite remote sensing image as input, identify the identification object of the same category as the target object, and identify the target object. mark the area where the object is located;

The satellite image segmentation module is used to perform image segmentation on the identified and marked satellite remote sensing images, and use the target object category as a classification to construct a satellite sub-image dataset;

The image style transfer module is used to build a style transfer network, using real sonar images as style images, and performing style transfer on the segmented satellite sub-images to generate sonar simulation images as training sample images.

Another technical scheme adopted by the present invention is:

A sonar simulation image generation system based on style transfer, comprising:

at least one processor;

at least one memory for storing at least one program;

When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

Another technical scheme adopted by the present invention is:

A computer-readable storage medium in which a processor-executable program is stored, the processor-executable program, when executed by the processor, is used to perform the method as described above.

The beneficial effects of the present invention are: the present invention uses remote sensing satellite images as input, takes real sonar images as targets, and uses a style transfer network to obtain simulated sonar images, which can effectively expand the number of sonar data sets, and is a follow-up correlation based on sonar images. work provides the foundation.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following descriptions are given to the accompanying drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings in the following introduction are only In order to facilitate and clearly express some embodiments of the technical solutions of the present invention, for those skilled in the art, other drawings can also be obtained from these drawings without creative work.

Fig. 1 is the step flow chart of a kind of sonar simulation image generation method based on style transfer in the embodiment of the present invention;

2 is a schematic flowchart of a method for generating a sonar simulation image based on style transfer in an embodiment of the present invention;

3 is a structural diagram of a convolutional neural network in an embodiment of the present invention;

Fig. 4 is the working flow chart of the style transfer network based on VGG16 network in the embodiment of the present invention;

5 is a schematic diagram of a satellite remote sensing image used in an embodiment of the present invention;

6 is a schematic diagram of a satellite remote sensing image after target identification in an embodiment of the present invention;

7 is a schematic diagram of a satellite remote sensing sub-image after image cropping in an embodiment of the present invention;

FIG. 8 is a schematic diagram of a simulated sonar image obtained through style transfer in an embodiment of the present invention.

Detailed ways

The following describes in detail the embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, only used to explain the present invention, and should not be construed as a limitation of the present invention. The numbers of the steps in the following embodiments are only set for the convenience of description, and the sequence between the steps is not limited in any way, and the execution sequence of each step in the embodiments can be adapted according to the understanding of those skilled in the art Sexual adjustment.

In the description of the present invention, it should be understood that the azimuth description, such as the azimuth or position relationship indicated by up, down, front, rear, left, right, etc., is based on the azimuth or position relationship shown in the drawings, only In order to facilitate the description of the present invention and simplify the description, it is not indicated or implied that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore should not be construed as limiting the present invention.

In the description of the present invention, the meaning of several is one or more, the meaning of multiple is two or more, greater than, less than, exceeding, etc. are understood as not including this number, above, below, within, etc. are understood as including this number. If it is described that the first and the second are only for the purpose of distinguishing technical features, it cannot be understood as indicating or implying relative importance, or indicating the number of the indicated technical features or the order of the indicated technical features. relation.

In the description of the present invention, unless otherwise clearly defined, words such as setting, installation, connection should be understood in a broad sense, and those skilled in the art can reasonably determine the specific meanings of the above words in the present invention in combination with the specific content of the technical solution.

As shown in FIG. 1 , this embodiment provides a method for generating a sonar simulation image based on style transfer, which includes the following steps:

S1. According to the target object category contained in the real sonar image, the satellite remote sensing image is used as the input to identify the satellite remote sensing image, identify the identified object with the same category as the target object, and mark the range where the identified object is located.

According to the existing target object category contained in the real sonar image and the target object category to be added, the satellite remote sensing image is used as the input to identify the satellite remote sensing image area with the same category as the target object.

(1) There are two types of real sonar image datasets: ship wreckage and aircraft wreckage. There are few types of targets, and the number distribution is uneven, which will make the follow-up work based on the real sonar image dataset impossible. In order to make the sonar image datasets diverse and sufficient, the present invention takes ship wrecks, aircraft wrecks, people in the water, and rocks as target object categories.

(2) Take the satellite remote sensing image as the original image as input, identify the above-mentioned target object in the image, obtain the identified object, and mark the image area where the identified object is located. The coordinates x, y of the corner, and the scale h, w of the area where the recognized object is located in the height and width directions of the image.

S2. Perform image segmentation on the identified and marked satellite remote sensing images, and construct a satellite sub-image dataset by classifying the target object category.

According to the identifier marking data in the above steps, the marked area is cropped, and the length, width and ratio of the cropped sub-image are determined, and the image is scaled and transformed with different length, width and aspect ratio.

The specific steps are:

(1) According to the x, y, h, w obtained in the above steps, the original image is cropped, and the cropping range is:

{(x, y), (x+w, y), (x, y+h), (x+w, y+h)}

The four coordinates in the above formula are the four-corner coordinates of the cropped image, and the image is cropped with the four-corner coordinates as the cropping range to obtain a sub-image.

(2) According to the sub-image obtained in the above steps, image scaling is performed with 416×416 pixels as a standard. The image scaling transformation processing formula is as follows:

S3, constructing a style transfer network, using the real sonar image as the style image, performing style transfer on the segmented satellite sub-images, and generating a sonar simulation image as a training sample image.

A convolutional neural network is constructed, and the features of real sonar images and sub-images are extracted by this network, and the Euclidean distance between the features of the two images and the Euclidean distance of the Gram matrix are used to optimize the loss function to obtain a simulated sonar image.

The specific steps are:

(1) Based on the first 17 layers of the VGG16 network, shortcut connections are added to the first 17 layers of the VGG16 to construct a convolutional neural network. As shown in Figure 3.

Shortcut connections are adding an input to the output of a function. The output after adding the shortcut can be explicitly expressed as a linear superposition of F(x) and x. Express the output as a linear superposition of the input and a nonlinear transformation of the input. In the case of deepening the network structure, the training effect of the network is guaranteed. The network calculation after adding shortcuts can be expressed as:

x _l+1 = x _l +F(x _l )

In the above formula, x _l and x _l+1 represent the characteristics of the lth layer and the l+1th layer of the network, respectively. F(x _l ) represents the convolution and activation operations of the network. The network backpropagation process after adding the shortcut is:

(2) Load the parameters of the above-mentioned convolutional neural network, and input the sub-image obtained in step S2 and the real sonar image into the network for style transfer. As shown in Figure 4.

The first 17 layers of the trained VGG16 network are used as the backbone of the above-mentioned convolutional neural network, the real sonar image is used as the style image, and the sub-image obtained in step S2 is used as the content image. Input the VGG16 network with shortcut to extract the style features respectively S and content features C.

Taking the Euclidean distance between the style feature S and the content feature C as the content loss, the content loss can be expressed as:

where i represents the i-th element in the feature map, and each element in the feature map is a value between [0, 1).

The style loss function is:

是风格图片在CNN第l层第(i,j,k)位置的输出，其中(i,j,k)对应高，宽，通道in the formula

is the output of the style image at the (i, j, k) position of the lth layer of CNN, where (i, j, k) corresponds to height, width, channel

The loss function for style transfer is:

L=αL _c +βL _s

where α and β are weight coefficients

(3) Set up a comparative experiment to verify the similarity between the simulated sonar image and the real sonar image.

Data setting of the experimental group: The real sonar image and the simulated sonar image are combined into a mixed dataset, and the dataset is divided into training set and validation set according to the ratio of 7:3. Control group data setting: only real sonar images were set as the dataset, and the training set and validation set were divided according to the ratio of 7:3. After being tested by the image classification network, the accuracy rates of the two sets of experimental data sets are similar.

The above method will be explained in detail below with reference to the accompanying drawings and specific embodiments.

Referring to FIG. 2, FIG. 2 shows a flowchart of a method for generating a sonar simulation image based on style transfer provided by an embodiment of the present invention, and the method includes the following steps:

Step S101 , according to the existing target object category and the target object category to be added contained in the real sonar image, using the satellite remote sensing image as input, identify the satellite remote sensing image area with the same target object category.

(1) There are two types of real sonar image data sets involved in this embodiment: ship wreckage and aircraft wreckage, and in order to make the simulated sonar image target types have practical application value in rescue and salvage and underwater detection, add drowning personnel And stones are also used as target types.

(2) Figure 5 is one of the satellite remote sensing images used in this example. Taking the satellite remote sensing image as input, the objects in the remote sensing image are identified according to the classification criteria, and the identified areas are marked. Figure 5 contains three planes, so after identification, the three planes and their corresponding areas should be marked. The labeled map after identification is shown in Figure 6. The marked data are:

{airplane,(642,187),120,120}

{airplane,(151,360),210,180}

{airplane,(329,552),174,147}

Step S102, according to the identifier mark data in the above steps, the marked area is cropped, and the length, width and ratio of the cropped sub-image are judged, and the image is stretched with different length, width and aspect ratio. transform.

(1) According to the {classes,(x,y),w,h} obtained by the above steps, the cropping range of the three images is:

{(642,187),(762,187),(642,370),(762,370)}

{(151,360),(361,360),(151,542),(361,542)}

{(329,552),(503,552),(329,699),(503,699)}

The image is cropped with the coordinates of the four corners as the cropping range, and the sub-image is obtained as shown in Figure 7.

(2) According to the sub-image obtained in the above steps, image scaling is performed with 416×416 pixels as a standard. The image scaling transformation processing formula is as follows:

The sizes of the three sub-images after scaling are: 416×416, 416×357, and 416×351.

S103: Construct a convolutional neural network, and use the network to extract the features of real sonar images and sub-images, and optimize the loss function with the Euclidean distance between the features of the two images and the Euclidean distance of the Gram matrix to obtain a simulated sonar image.

(1) Based on the first 17 layers of the VGG16 network, shortcut connections are added to the first 17 layers of the VGG16 to construct a convolutional neural network.

Shortcut structure: In addition to the necessary downsampling operation, the data input to the network is not processed, and it is directly fused with the feature map after convolution to improve the ability of the neural network to extract features.

(2) Load the parameters of the above-mentioned convolutional neural network, which are trained from the ImageNet dataset. The sub-image and the real sonar image obtained in step S102 are input into the network for style transfer.

The simulated sonar image after style transfer is shown in Figure 8.

(3) Set up a comparative experiment to verify the similarity between the simulated sonar image and the real sonar image.

Data setting of the experimental group: The real sonar image and the simulated sonar image are combined into a mixed data set and the data set is divided into training set and validation set according to the ratio of 7:3. The numbers of training and validation sets are 84 and 36, respectively. Control group data setting: only real sonar images were set as the dataset, and the training set and validation set were divided according to the ratio of 7:3. The numbers of training and validation sets are 44 and 18, respectively. After inference by the image classification network, the accuracy rates of the two sets of data are: 86.11% and 83.33%, respectively. The two sets of data are quite different, indicating that for the same image classification network, the real data set and the mixed data set have strong similarity.

This embodiment uses the remote sensing satellite aircraft image as the input, takes the real aircraft wreckage sonar image as the target, uses the style transfer network to obtain the simulated sonar image, and effectively expands the number of aircraft sonar image data sets.

To sum up, the method of this embodiment has the following advantages and beneficial effects compared to the prior art:

The invention takes the remote sensing satellite image as the input, takes the real sonar image as the target, and uses the style transfer network to obtain the simulated sonar image. In addition, in the process from inputting an image to finally obtaining a simulated sonar image, the present invention does not need to manually add image features, can quickly obtain simulated sonar images according to remote sensing images of different resolutions, and the obtained simulated sonar images have the style of real sonar images , which can effectively expand the number of sonar datasets and provide a basis for subsequent related work based on sonar images.

This embodiment also provides a sonar simulation image generation system based on style transfer, including:

The satellite image processing module is used to identify the satellite remote sensing image according to the target object category contained in the real sonar image, and use the satellite remote sensing image as input, identify the identification object of the same category as the target object, and identify the target object. mark the area where the object is located;

The satellite image segmentation module is used to perform image segmentation on the identified and marked satellite remote sensing images, and use the target object category as a classification to construct a satellite sub-image dataset;

The image style transfer module is used to build a style transfer network, using real sonar images as style images, and performing style transfer on the segmented satellite sub-images to generate sonar simulation images as training sample images.

The system for generating a sonar simulation image based on style transfer in this embodiment can execute the method for generating a sonar simulation image based on style transfer provided by the method embodiment of the present invention, and can perform any combination of implementation steps of the method embodiment. The corresponding functions and beneficial effects of the method.

This embodiment also provides a sonar simulation image generation system based on style transfer, including:

at least one processor;

at least one memory for storing at least one program;

When the at least one program is executed by the at least one processor, the at least one processor implements the method shown in FIG. 1 .

The system for generating a sonar simulation image based on style transfer in this embodiment can execute the method for generating a sonar simulation image based on style transfer provided by the method embodiment of the present invention, and can perform any combination of implementation steps of the method embodiment. The corresponding functions and beneficial effects of the method.

Embodiments of the present application further disclose a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of the computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method shown in FIG. 1 .

This embodiment also provides a storage medium, which stores an instruction or program for executing a style transfer-based sonar simulation image generation method provided by the method embodiment of the present invention. When the instruction or program is executed, the method can be executed. Any combination of the implementation steps of the embodiments has the corresponding functions and beneficial effects of the method.

In some alternative implementations, the functions/operations noted in the block diagrams may occur out of the order noted in the operational diagrams. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/operations involved. Furthermore, the embodiments presented and described in the flowcharts of the present invention are provided by way of example in order to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of the various operations are altered and in which sub-operations described as part of larger operations are performed independently.

Furthermore, while the invention is described in the context of functional modules, it is to be understood that, unless stated to the contrary, one or more of the described functions and/or features may be integrated in a single physical device and/or or software modules, or one or more functions and/or features may be implemented in separate physical devices or software modules. It will also be appreciated that a detailed discussion of the actual implementation of each module is not necessary to understand the present invention. Rather, given the attributes, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the modules will be within the routine skill of the engineer. Accordingly, those skilled in the art, using ordinary skill, can implement the invention as set forth in the claims without undue experimentation. It is also to be understood that the specific concepts disclosed are illustrative only and are not intended to limit the scope of the invention, which is to be determined by the appended claims along with their full scope of equivalents.

The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus.

More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of the present invention may be implemented in hardware, software, firmware or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following techniques known in the art: Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

In the above description of the present specification, reference to the description of the terms "one embodiment/example", "another embodiment/example" or "certain embodiments/examples" etc. means the description in conjunction with the embodiment or example. Particular features, structures, materials, or characteristics are included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, The scope of the invention is defined by the claims and their equivalents.

The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned embodiments, and those skilled in the art can also make various equivalent deformations or replacements on the premise of not violating the spirit of the present invention. Equivalent modifications or substitutions are included within the scope defined by the claims of the present application.

Claims (10)

1. a sonar simulation image generation method based on style transfer, is characterized in that, comprises the following steps: According to the target object category contained in the real sonar image, the satellite remote sensing image is used as the input, the satellite remote sensing image is identified, the identification object of the same category as the target object is identified, and the range of the identification object is marked; Perform image segmentation on the identified and marked satellite remote sensing images, and construct satellite sub-image datasets based on the category of target objects; A style transfer network is constructed, using real sonar images as style images, and performing style transfer on the segmented satellite sub-images to generate sonar simulation images as training sample images. 2. a kind of sonar simulation image generation method based on style transfer according to claim 1, is characterized in that, the described satellite remote sensing image is identified, the identification object that is identical with the described target object category is identified, and the Mark the range where the identifier is located, including: The target recognition network trained by the preset data set is used to detect the satellite remote sensing images, and the detection results are divided into n+1 categories, where n is the number of target object categories; Mark the area where the target is located. 3. a kind of sonar simulation image generation method based on style transfer according to claim 1, it is characterized in that, described carrying out image segmentation to the identified and marked satellite remote sensing image, taking the target object category as classification, constructing satellite sub-images. Image datasets, including: After marking the identifier, obtain the target area, read the parameters x, y, w, h of the target area, where x, y are the horizontal and vertical coordinates of the upper left corner of the target area, and w, h are the horizontal and vertical dimensions of the target area ; According to the read parameters, adaptive resolution segmentation is performed on the target area, and the segmented satellite sub-image area is:

in,

is an adaptive resolution scaling factor determined by the horizontal and vertical dimensions of the original image;

is the segmentation range of the original mark,

Segmentation range after scaling for adaptive resolution. 4. a kind of sonar simulation image generation method based on style transfer according to claim 3, is characterized in that, described adaptive resolution scaling factor λ is obtained by the following way:

Among them, f(w,h) is the smaller side of w,h. 5 . The method for generating sonar simulation images based on style transfer according to claim 1 , wherein the style transfer network uses a hollow space pyramid structure to extract style feature maps. 6 . 6. a kind of sonar simulation image generation method based on style transfer according to claim 1, is characterized in that, described constructing style transfer network, takes real sonar image as style image, carries out style to the satellite sub-image after segmentation. Migrate, generate sonar simulation images as training sample images, including: Construct a convolutional neural network, use the convolutional neural network to extract the features of real sonar images and satellite sub-images, and optimize the loss function with the Euclidean distance between the features of the two images and the Euclidean distance of the Gram matrix to obtain sonar. Simulation image. 7. a kind of sonar simulation image generation method based on style transfer according to claim 6, is characterized in that, described constructing convolutional neural network, utilizes described convolutional neural network to extract real sonar image and satellite sub-image. feature, optimize the loss function with the Euclidean distance between the two image features and the Euclidean distance of the Gram matrix to obtain the sonar simulation image, including: Based on the first 17 layers of the VGG16 network, shortcut connections are added to the first 17 layers of the VGG16 to build a convolutional neural network; The calculation of the convolutional neural network after adding the shortcut is expressed as: x _l+1 = x _l +F(x _l ) In the formula, x _l and x _l+1 represent the characteristics of the lth layer and the l+1th layer of the network respectively; F(x _l ) represents the convolution and activation operations of the network; the network backpropagation process after adding the shortcut is:

The first 17 layers of the trained VGG16 network are used as the backbone of the convolutional neural network, the real sonar image is used as the style image, and the satellite sub-image is used as the content image to input the VGG16 network with shortcut, and the style feature S and content feature are extracted respectively. C; Taking the Euclidean distance between the style feature S and the content feature C as the content loss, the content loss is expressed as:

where i represents the i-th element in the feature map; The style loss function is:

in the formula

is the output of the style feature S at the (i, j, k) position of the lth layer of CNN, where (i, j, k) corresponds to the three dimensions of height, width and channel;

is the output of the style image at the (i, j, k') position of the lth layer of the CNN, where (i, j, k') corresponds to the three dimensions of height, width and channel;

It is the output of the image G generated by the network at the (i, j, k) position of the lth layer of the CNN, where (i, j, k) corresponds to the three dimensions of height, width and channel;

is the total number on the height dimension of the lth layer of CNN;

is the total number on the width dimension of the lth layer of CNN;

is the total number on the channel dimension of the first layer of CNN;

is the Gram matrix of the style feature S of the lth layer of CNN in channel k' and channel k;

is the Gram matrix of the content feature C of the lth layer of CNN in channel k' and channel k; The loss function for style transfer is: L=αL _c +βL _s where α and β are weight coefficients. 8. A sonar simulation image generation system based on style transfer, is characterized in that, comprises: The satellite image processing module is used to identify the satellite remote sensing image according to the target object category contained in the real sonar image, and use the satellite remote sensing image as input, identify the identification object of the same category as the target object, and identify the target object. mark the area where the object is located; The satellite image segmentation module is used to perform image segmentation on the identified and marked satellite remote sensing images, and use the target object category as a classification to construct a satellite sub-image dataset; The image style transfer module is used to build a style transfer network, using real sonar images as style images, and performing style transfer on the segmented satellite sub-images to generate sonar simulation images as training sample images. 9. A sonar simulation image generation system based on style transfer, is characterized in that, comprises: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method of any one of claims 1-7. 10. A computer-readable storage medium, wherein a program executable by a processor is stored, wherein the program executable by the processor, when executed by the processor, is used to execute any one of claims 1-7 the method.

Images