CN118096922A - Method for generating map based on style migration and remote sensing image - Google Patents

Abstract

The invention discloses a method for generating a map based on style migration and remote sensing images, wherein a constructed map generation network model comprises an encoder, a style conversion module and a decoder; capturing long-range dependency relations among map features by fusing a multi-head self-attention mechanism and a residual error module as a style converter; the method combines the traditional transpose convolution and Carafe operators in the up-sampling stage of the decoder to better utilize the neighborhood information to improve the up-sampling quality; and optimizing and training the map generation network model, inputting the remote sensing image into the trained optimal model, and outputting a corresponding map image. The invention provides more accurate characteristic information for the up-sampling operation, and the up-sampling operation is carried out through up-sampling kernel prediction and characteristic recombination, so that the generated map has good visual improvement effect on the aspects of roads, buildings, edge details, map content color saturation and the like. The problems of detail loss and unclear content in map generation in the prior art are solved.

Description

A method for generating maps based on style transfer and remote sensing images

Technical Field

The present invention belongs to the technical field of map making and relates to a method for generating a map based on style migration and remote sensing images.

Background technique

Maps play an important and indispensable role in people's daily life and work. They not only provide spatial positioning and navigation functions, but also provide rich geographic information and spatial data resources. Traditional map production methods usually rely on manual surveys and vehicle-mounted GPS trajectory data. However, these methods have some inherent limitations in the map update process. First, traditional map production methods require a lot of human resources and time, resulting in slow map updates. Second, manual surveys may introduce human errors, which in turn lead to differences between maps and actual ground. In addition, vehicle-mounted GPS trajectory data may also be limited by the environment and equipment, and therefore cannot fully and accurately reflect the real situation. Considering the frequent renovation of ground buildings and roads and the occurrence of natural disasters, the actual situation on the ground does not match the existing maps. Therefore, a map generation method that is both fast and accurate is needed.

The emergence of the idea of style transfer has brought a solution to the above problems. In recent years, with the improvement of computing performance of hardware devices and the rapid development of deep learning, deep learning has been widely used in various fields of computer vision. Network models based on deep learning continue to emerge, various algorithms are constantly optimized and upgraded, and have achieved great results in practical applications. Thanks to this, many scholars have begun to use neural networks to transfer the style of images.

In recent years, many scholars have proposed map generation methods based on GAN networks, such as: the fully supervised model Pix2pix: based on the conditional generative adversarial network, through the supervised training of paired images, one-to-one image style transfer is achieved, but it needs to rely on paired data and even semantic labels. However, in many practical scenarios, it is very difficult to obtain accurate paired data and labels; the unsupervised model CycleGAN: through the cycle consistency loss function to ensure the consistency and accuracy of the conversion, and does not require paired training data, but due to the lack of supervised training of paired data, the actual generation effect is not very satisfactory; the semi-supervised model SmapGAN: based on the semi-supervised generative adversarial network to achieve the style transfer conversion between remote sensing images and maps of the same region. And design the image gradient L1 loss and image gradient structure loss to generate a styled map block with global topological relationships and detailed edge curves of objects. This model combines the advantages of supervised and unsupervised models. Although it has certain advantages, since the style converter based on Resblock mainly uses local receptive fields to capture the spatial local relationship in the input data, it still lacks the processing of long-distance dependencies. In addition, traditional transposed convolution is affected by the local receptive field and padding of the convolution kernel, which easily causes blurring and information loss during upsampling.

Based on the defects of the supervised model, unsupervised model and semi-supervised model mentioned above, the generation of remote sensing images to maps has problems such as loss of map details and unclear content. Therefore, a method of generating maps based on style transfer and remote sensing images is proposed.

Summary of the invention

The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a method for generating maps based on style transfer and remote sensing images.

The specific steps include:

Step 1: First, divide any public remote sensing image dataset into training set, validation set and test set according to the set ratio. The dataset includes multiple paired images, each of which contains a remote sensing image and a corresponding map image. At the same time, data enhancement is performed on the training set and validation set, while no data enhancement is performed on the test set. Data enhancement includes flipping and rotation operations. The purpose of the training set is to learn features, the validation set is used to assist model debugging, and the test set is used to evaluate the accuracy of the model;

Step 2: Build a map generation network model, which includes an encoder, a style transfer module, and a decoder;

The encoder includes a convolution layer with a convolution kernel size of 7×7, a batch normalization layer, a ReLU activation function, and two downsampling layers. The downsampling layer includes a convolution layer with a convolution kernel size of 3×3, a batch normalization layer, and a ReLU activation function.

The style conversion module uses a residual block combined with a multi-head self-attention mechanism as the main structure of the style converter; the feature map Feature _R obtained after being processed by the encoder is sequentially passed through 9 residual blocks consisting of 3×3 convolution layers, ReLU activation functions, and 3×3 convolution layers for preliminary style conversion. In the last layer of the style converter, a 1×1 convolution layer, a ReLU activation function, a 4-head self-attention mechanism module, a ReLU activation function, a 1×1 convolution layer, and a ReLU activation function are used in sequence to output Feature _M.

The decoder includes an upsampling module and a convolution layer with a convolution kernel size of 7×7. Feature _M is upsampled by a traditional 3×3 transposed convolution as the first layer, which is expressed by the formula: F ₁ =TC(Feature _M ); where TC represents the transposed convolution operation and F ₁ represents the result after the 3×3 transposed convolution. The Carafe upsampling operator is used in the second layer. The Carafe module consists of three convolution layers and is divided into two parts: an upsampling kernel prediction module and a feature reorganization module.

The Carafe module processing process is as follows:

First, a 1×1 convolutional layer is used to compress the number of channels of the input feature map from C to C _n . This is expressed by the formula: F ₂ =C ₁ (F ₁ ); where C ₁ represents the channel compression operation, and F ₂ represents the result of channel compression of F ₁ .

Secondly, a 3×3 convolution kernel is used to perform upsampling kernel prediction, where the input is H×W×C _n and the output is The parameters are set as σ = 2, k _up = 3, where σ represents the upsampling factor and k _up represents the size of the reorganization kernel. This operation can increase the encoder's receptive field and make full use of context information in a larger area. At the same time, the channel dimension is expanded in the spatial dimension to obtain a shape of/> The upsampling kernel is then used, and each recombined kernel of size k _up ×k _up is spatially normalized. It is expressed as:/> In the formula, KPM represents the upsampling kernel prediction operation, Unfolding represents the spatial expansion, softmax represents the normalization operation, and F ₃ represents the result of the upsampling kernel prediction.

In the feature recombination module, the up-sampled prediction kernel is dot-producted with the k _up ×k _up region centered on a feature point of the input feature map to achieve feature recombination. Finally, a convolution layer with a convolution kernel of 1×1 is used to compress the channel and output Feature _CM . It is expressed as: Where CARM represents the feature recombination operation, N(F _1i ,k _up ) represents the area of size k _up ×k _up centered on the feature point F _1i in the input feature map, F _3i represents the upsampling kernel of the point predicted by the upsampling kernel prediction module, C ₂ represents the channel compression operation, and Feature _CM represents the result output by TC-Carafe.

Finally, Feature _CM is fed into a 7×7 convolutional layer and Tanh activation is performed to output the generated map image.

Step 3: Input the training set and validation set in step 1 into the network model built in step 2 for optimization training to obtain the optimal model;

The discriminator is used to discriminate the map output in step 2. The discriminator uses PatchGAN, which decomposes the judgment task of the whole image into the judgment task of multiple local areas (patches) in the image. This method helps the network understand the local structure and details of the image more carefully.

The loss function used in the training process is as follows:

1) Topological consistency loss: is the image gradient L1 loss, and L _grastr is the image gradient structure loss.

In the formula, Indicates sampling from remote sensing image samples; C ₁ and C ₂ are constant terms; M and N indicate that the input image has an M×N scale of N columns; /> is the covariance of G _j (y) and G _j (G _X→Y (x));/> is the standard deviation of G _j (y); is the standard deviation of G _j (G _X→Y (x));/> is the covariance of _Gi (y) and _Gi (G _X→Y (x));/> is the standard deviation of _Gi (y);/> is the standard deviation of _Gi (G _X→Y (x)). For the 255×255 gradient images of the real map and the generated map, their pixel matrices G(y) and G(G _X→Y (x)) have 255 rows and 255 columns. For the jth column (ith row), the pixel value of the point on this column (ith row) is composed of an i-dimensional (j-dimensional) random variable _Gj ( _Gi ). In G(G _X→Y (x))-G(y), G(y) is the gradient image of the real map y, and G(G _X→Y (x)) is the gradient image of the generated fake map G _X→Y (x).

2) Content loss: aims to ensure that the generated map is similar to the ground truth in content. and is the circulation loss, /> and/> is the direct loss. In the unsupervised stage, the cycle loss is used. In the supervised stage, the direct loss is used.

In the formula, represents the cyclic loss of remote sensing images, λ is the fine-tuning coefficient, L1u represents the L1 loss in the unsupervised stage,/> It is the pixel L1 loss. This cycle loss calculates the difference between pixels through L1 loss, so that the generated map and remote sensing image maintain cycle consistency in content. /> represents sampling from remote sensing image samples, G _Y→X (G _X→Y (x))-x represents the cycle loss of calculating the fake map image generated by G _X→Y (x), and then generating the fake remote sensing image and the real remote sensing image x through G _Y→X .

In the formula, represents the cycle loss of the map,/> Represents sampling from map samples. By introducing topological consistency loss/> To maintain the cyclic consistency between the topological structure of the generated image and the topological structure of the target image. _{G X→Y} (G _Y→X (y))-y represents the cyclic loss of the fake remote sensing image generated by G _Y→X (y), and then the fake map image and the real map image y generated by G _X→Y .

Direct loss from maps to remote sensing images The L1 loss function is used to maintain content consistency. /> Where λ is the fine-tuning coefficient, L1 represents the L1 loss, /> Represents the pixel L1 loss. _{G Y→X} (y)-x represents the loss between the calculated fake remote sensing image and the real remote sensing image.

Direct loss from remote sensing images to maps By/> The loss keeps the generated map consistent with the remote sensing image in topological structure. _{G X→Y} (x)-y represents the loss between the generated fake map image and the real map image.

3) Adversarial loss: The discriminator is used to determine the difference between the generated image and the real image. The purpose of the generator G is to minimize the value of the loss function, while the purpose of the discriminator D is to maximize the value of the loss function. The formula is:

Adversarial Loss from Remote Sensing Images to Maps G _X→Y is a generator for remote sensing images to maps, and the generated image is input to the discriminator D _Y for discrimination.

Adversarial Loss from Map to Remote Sensing Imagery G _Y→X is a map-to-remote sensing image generator, and the generated image is sent to the discriminator D _X for discrimination.

4) Identity loss: used to ensure the consistency between the converted image and the original image. For example, the map generated by passing the map into the generator G _X→Y should be as consistent as possible with the input map, that is, the content and color of the map itself. The formula is:

Step 4: Input the remote sensing image into the optimal model obtained after training in step 3 and output the corresponding map image.

Compared with the prior art, the significant advantages of the present invention are: the residual module is combined with the multi-headed self-attention mechanism to capture the long-range dependencies between features, providing more accurate feature information for upsampling operations. Secondly, a new upsampling method combining traditional transposed convolution and Carafe operator is proposed, and the upsampling operation is performed through upsampling kernel prediction and feature reorganization, making the generated map clearer and more complete.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is a flow chart of the present invention;

FIG2 is a diagram of a map generation network model module of the present invention;

FIG3 is a block diagram of a style converter according to the present invention;

FIG4 is a diagram of an upsampling module of the present invention;

FIG5 is a block diagram of a discriminator of the present invention;

FIG6 is a diagram showing a comparison of visual effects between the method for generating a map based on remote sensing images of the present invention and other methods.

Detailed ways

The present invention is described in detail below with reference to the accompanying drawings and specific implementation steps.

As shown in FIG1 , a method for generating a map based on remote sensing images specifically includes the following steps:

Step 1: In this example, the public New York City dataset is used and divided into a training set, a validation set, and a test set in a ratio of 8:1:1. Data augmentation is performed on the training set and the validation set, with a zoom level of 16, containing 2194 remote sensing images and their corresponding maps, and an image size of 256×256.

Step 2: Build a map generation network model, as shown in Figure 2. The model includes an encoder, a style transfer module, and a decoder.

The encoder includes one convolution layer with a convolution kernel size of 7×7, a batch normalization layer (BN), a ReLU activation function, and two downsampling layers (including a convolution layer with a convolution kernel size of 3×3, a batch normalization layer (BN), and a ReLU activation function). A C×H×W remote sensing image is input, and the size and redundant information of the feature map are reduced through one convolution layer with a convolution kernel size of 7×7 and two downsampling layers with a convolution kernel size of 3×3 in the encoder.

As shown in Figure 3, the style conversion module uses a residual block combined with a multi-head self-attention mechanism (MHSA) as the main structure of the style converter; the feature map Feature _R obtained after being processed by the encoder is sequentially passed through 9 residual blocks consisting of 3×3 convolution layers, ReLU activation functions, and 3×3 convolution layers for preliminary style conversion. In the last layer of the style converter, a 1×1 convolution layer, a ReLU activation function, a 4-head self-attention mechanism module, a ReLU activation function, a 1×1 convolution layer, and a ReLU activation function are used in sequence to output Feature _M. By introducing the multi-head self-attention mechanism, feature representations can be learned in parallel on multiple subspaces to capture the long-range dependencies between pixels in the image, thereby increasing the nonlinear ability of the model.

The decoder includes an upsampling module and a convolution layer with a convolution kernel size of 7×7, as shown in Figure 4. Feature _M is upsampled by a traditional 3×3 transposed convolution as the first layer, which is expressed as:

F ₁ =TC(Feature _M );

Where TC represents the transposed convolution operation, and _F1 represents the result after 3×3 transposed convolution.

The Carafe upsampling operator is used in the second layer to better improve the generated details and clarity of the image. Carafe upsampling has the advantages of large receptive field, light weight and fast calculation speed. The Carafe module consists of three convolutional layers and is divided into two parts: the upsampling kernel prediction module and the feature reconstruction module. In the upsampling kernel prediction module, the input data is a feature map of size C×H×W. C represents the number of channels in an image, H represents the number of pixels in the vertical dimension of the image, and W represents the number of pixels in the horizontal dimension of the image; first, a 1×1 convolutional layer is used to compress the number of channels of the input feature map from C _{to Cn} . It is expressed by the formula: _F2 = _C1 ( _F1 );

Where _C1 represents the channel compression operation, and _F2 represents the result obtained after performing channel compression _{on F1} .

Secondly, a 3×3 convolution kernel is used to perform upsampling kernel prediction, where the input is H×W×C _n and the output is The parameters are set as σ = 2, k _up = 3, where σ represents the upsampling factor and k _up represents the size of the reorganization kernel. This operation can increase the encoder's receptive field and make full use of context information in a larger area. At the same time, the channel dimension is expanded in the spatial dimension to obtain a shape of/> The upsampling kernel is then used, and each recombined kernel of size k _up ×k _up is spatially normalized. It is expressed as:/>

In the formula, KPM represents the upsampling kernel prediction operation, Unfolding represents the spatial expansion, softmax represents the normalization operation, and F ₃ represents the result of the upsampling kernel prediction.

In the feature recombination module, the up-sampled prediction kernel is dot-producted with the k _up ×k _up region centered on a feature point of the input feature map to achieve feature recombination. Finally, a convolution layer with a convolution kernel of 1×1 is used to compress the channel and output Feature _CM . It is expressed as:

Where CARM represents the feature recombination operation, N(F _1i ,k _up ) represents the area of size k _up ×k _up centered on the feature point F _1i in the input feature map, F _3i represents the upsampling kernel of the point predicted by the upsampling kernel prediction module, C ₂ represents the channel compression operation, and Feature _CM represents the result output by TC-Carafe.

Finally, Feature _CM is fed into a 7×7 convolutional layer and Tanh activation is performed to output the generated map image.

Step 3: Send the training set and validation set in the data set obtained in step 1 to the network built in step 2, train according to the adjusted parameters, and save the optimal model;

Specifically, during training, the batch size is set to 1, and the Adam optimizer is used to optimize the network. In this paper, β ₁ = 0.5 and β ₂ = 0.999 are set. The learning rate strategy uses linear, and the initial learning rate is 0.0002. The learning rate remains unchanged from 1 to 100 epochs, and the learning rate gradually decays to 0 from 101 to 200 epochs. All models are trained for 200 epochs.

When the image generated by the generator is input into the PatchGAN discriminator as shown in Figure 5, the task of the discriminator is subdivided into the task of judging multiple local areas (patches) in the image. This decomposition method helps the discriminator understand the local structure and details of the map image more finely. Specifically, after the image passes through 5 4×4 convolutions, LeakyReLu and BN layers, the input is mapped to a 70×70 matrix, and each patch in the 70×70 matrix is judged as true or false. The discriminator will judge the authenticity of each local area one by one. If the patch is judged to be true, it is marked as 1, and if the patch is judged to be false, it is marked as 0, and finally the probability of the overall image being true is calculated. This judgment method can enable the generator to better capture the local features of the map, improve the realism and detail performance of the generated map, and optimize the model training through adversarial loss.

The loss function used in the training process is as follows:

1) Topological Consistency Loss: used to ensure the correct topological relationship of G _X→Y (remote sensing map). is the image gradient L1 loss, and L _grastr is the image gradient structure loss.

In the formula, Indicates sampling from remote sensing image samples; C ₁ and C ₂ are constant terms; M and N indicate that the input image has an M×N scale of N columns; /> is the covariance of G _j (y) and G _j (G _X→Y (x));/> is the standard deviation of G _j (y); is the standard deviation of G _j (G _X→Y (x));/> is the covariance of _Gi (y) and _Gi (G _X→Y (x));/> is the standard deviation of _Gi (y);/> is the standard deviation of _Gi (G _X→Y (x)). For the 255×255 gradient images of the real map and the generated map, their pixel matrices G(y) and G(G _X→Y (x)) have 255 rows and 255 columns. For the jth column (ith row), the pixel value of the point on this column (ith row) is composed of an i-dimensional (j-dimensional) random variable _Gj ( _Gi ). In G(G _X→Y (x))-G(y), G(y) is the gradient image of the real map y, and G(G _X→Y (x)) is the gradient image of the generated fake map G _X→Y (x).

2) Content Loss: Aims to ensure that the generated map is similar to the Ground Truth in content. and/> is the circulation loss, /> and/> is the direct loss. In the unsupervised stage, the cycle loss is used. In the supervised stage, the direct loss is used.

In the formula, represents the cyclic loss of remote sensing images, λ is the fine-tuning coefficient, L1u represents the L1 loss in the unsupervised stage,/> It is the pixel L1 loss. This cycle loss calculates the difference between pixels through L1 loss, so that the generated map and remote sensing image maintain cycle consistency in content. /> represents sampling from remote sensing image samples, G _Y→X (G _X→Y (x))-x represents the cycle loss of calculating the fake map image generated by G _X→Y (x), and then generating the fake remote sensing image and the real remote sensing image x through G _Y→X .

In the formula, represents the cycle loss of the map,/> Represents sampling from map samples. By introducing topological consistency loss/> To maintain the cyclic consistency between the topological structure of the generated image and the topological structure of the target image. _{G X→Y} (G _Y→X (y))-y represents the cyclic loss of the fake remote sensing image generated by G _Y→X (y), and then the fake map image and the real map image y generated by G _X→Y .

Direct loss from maps to remote sensing images The L1 loss function is used to maintain content consistency. /> Where λ is the fine-tuning coefficient, L1 represents the L1 loss, /> Represents the pixel L1 loss. _{G Y→X} (y)-x represents the loss between the calculated fake remote sensing image and the real remote sensing image.

Direct loss from remote sensing images to maps By/> The loss keeps the generated map consistent with the remote sensing image in topological structure. _{G X→Y} (x)-y represents the loss between the generated fake map image and the real map image.

3) Adversarial Loss: The discriminator is used to determine the difference between the generated image and the real image. The purpose of the generator G is to minimize the value of the loss function, while the purpose of the discriminator D is to maximize the value of the loss function. The formula is:

Adversarial Loss from Remote Sensing Images to Maps G _X→Y is a generator for remote sensing images to maps, and the generated image is input to the discriminator D _Y for discrimination.

Adversarial Loss from Map to Remote Sensing Imagery G _Y→X is a map-to-remote sensing image generator, and the generated image is sent to the discriminator D _X for discrimination.

4) Identity Loss: It is used to ensure the consistency between the converted image and the original image. For example, the map generated by passing the map into the generator G _X→Y should be as consistent as possible with the input map, that is, the content and color of the map itself. The formula is:

Step 4: Test the optimal model obtained in step 3 on the remote sensing image generation map to obtain a map image;

As shown in Figure 6, the generation result of the map generation model, it can be seen that both CycleGAN and Pix2pix have the problem of incorrect content generation, and SmapGAN has the problem of blurred generated content and unclear edges. The invention of this paper adopts MHSA-ResBlock in the style converter part, that is, the residual module is combined with Multi-headed Self-attention to capture the long-range dependency between features, and provide more accurate feature information for upsampling operations. Secondly, a new upsampling method TC-Carafe combining traditional transposed convolution and Carafe operator is used to perform upsampling operations through upsampling kernel prediction and feature reorganization, so that the generated map has a good visual improvement effect in terms of roads, buildings, edge details and map content color saturation. The comparison results of the indicators of the map generation method of the present invention and other methods are shown in Table 1. It can be seen that the advantages of the present invention are more obvious, and various evaluation indicators (peak signal-to-noise ratio (PSNR), structural similarity (SSIM), root mean square error (RMSE)) are better than other methods.

Table 1

Model PSNR SSIM RMSE Pix2pix 19.9255 0.6719 28.9183 CycleGAN 24.5271 0.8157 18.3162 SmapGAN 27.5014 0.8742 12.4684 Ours 28.1147 0.8784 11.7484

The specific implementation modes of the present invention have been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims (4)

1. A method for generating a map based on style transfer and remote sensing images, characterized in that: The specific steps include: Step 1: First, any public remote sensing image dataset is divided into training set, validation set and test set according to the set ratio; the dataset includes multiple paired images, each of which contains a remote sensing image and a corresponding map image; at the same time, data enhancement is performed on the training set and validation set, while no data enhancement is performed on the test set; Step 2: Build a map generation network model, which includes an encoder, a style transfer module, and a decoder; The encoder includes a convolution layer with a convolution kernel size of 7×7, a batch normalization layer, a ReLU activation function, and two downsampling layers. The downsampling layer includes a convolution layer with a convolution kernel size of 3×3, a batch normalization layer, and a ReLU activation function. The style conversion module uses a residual block combined with a multi-head self-attention mechanism as the main structure of the style converter. The feature map Feature _R obtained after the encoder processing is sequentially passed through 9 residual blocks consisting of 3×3 convolutional layers, ReLU activation functions, and 3×3 convolutional layers for preliminary style conversion. In the last layer of the style converter, a 1×1 convolutional layer, a ReLU activation function, a 4-head self-attention mechanism module, a ReLU activation function, a 1×1 convolutional layer, and a ReLU activation function are used in sequence to output Feature _M. The decoder includes an upsampling module and a convolution layer with a convolution kernel size of 7×7. Feature _M is upsampled by a traditional 3×3 transposed convolution as the first layer, which is expressed by the formula: F ₁ =TC(Feature _M ); where TC represents the transposed convolution operation and F ₁ represents the result after the 3×3 transposed convolution. The Carafe upsampling operator is used in the second layer. The Carafe module consists of three convolution layers and is divided into two parts: an upsampling kernel prediction module and a feature reconstruction module. Step 3: Input the training set and validation set in step 1 into the network model built in step 2 for optimization training to obtain the final optimized model; Step 4: Input the remote sensing image into the optimal model trained in step 3 and output the corresponding map image. 2. The method for generating maps from remote sensing images as described in claim 1, wherein the data enhancement content described in step 1 includes flipping and rotation operations. 3. The method for generating a map from remote sensing images as claimed in claim 1, wherein the processing of the Carafe module in step 2 is as follows: First, a 1×1 convolutional layer is used to compress the number of channels of the input feature map from C to C _n ; it is expressed by the formula: F ₂ =C ₁ (F ₁ ); where C ₁ represents the channel compression operation, and F ₂ represents the result obtained after channel compression of F ₁ ; Secondly, a 3×3 convolution kernel is used to perform upsampling kernel prediction, where the input is H×W×C _n and the output is The parameters are set as σ = 2, k _up = 3, where σ represents the upsampling factor and k _up represents the size of the reorganization kernel. This operation can increase the encoder's receptive field and make full use of context information in a larger area. At the same time, the channel dimension is expanded in the spatial dimension to obtain a shape of / > The upsampling kernel is used, and each recombined kernel of size k _up ×k _up is spatially normalized; it is expressed by the formula:/> In the formula, KPM represents the upsampling kernel prediction operation, Unfolding represents the spatial expansion, softmax represents the normalization operation, and F ₃ represents the result of the upsampling kernel prediction; In the feature recombination module, the up-sampled prediction kernel is dot-producted with the k _up ×k _up region centered on a feature point of the input feature map to achieve feature recombination; finally, a convolution layer with a convolution kernel of 1×1 is used to compress the channel and output Feature _CM ; it is expressed as: Where CARM represents the feature recombination operation, N(F _1i ,k _up ) represents the area of size k _up ×k _up centered on the feature point F _1i in the input feature map, F _3i represents the upsampling kernel of the point predicted by the upsampling kernel prediction module, C ₂ represents the channel compression operation, and Feature _CM represents the result output by TC-Carafe; Finally, Feature _CM is fed into a 7×7 convolutional layer and Tanh activation is performed to output the generated map image. 4. The method for generating maps from remote sensing images as claimed in claim 1, wherein the step 3 uses a loss function and a discriminator to optimize the model constructed in step 2: The discriminator uses PatchGAN, which is composed of multiple layers of convolution. The judgment task of the whole image is decomposed into the judgment tasks of multiple local areas in the image. The loss function used in the training process is as follows: 1) Topological consistency loss: is the image gradient L1 loss, L _grastr is the image gradient structure loss; In the formula, Indicates sampling from remote sensing image samples; C ₁ and C ₂ are constant terms; M and N indicate that the input image has an M×N scale of N columns; /> is the covariance of G _j (y) and G _j (G _X→Y (x));/> is the standard deviation of G _j (y);/> is the standard deviation of G _j (G _X→Y (x));/> is the covariance of _Gi (y) and _Gi (G _X→Y (x));/> is the standard deviation of _Gi (y);/> is the standard deviation of _Gi (G _X→Y (x)); for the 255×255 gradient images of the real map and the generated map, their pixel matrices G(y) and G(G _X→Y (x)) have 255 rows and 255 columns; for the jth column (the ith row), the pixel value of the point on the column (the ith row) is composed of an i-dimensional (j-dimensional) random variable _Gj ( _Gi ); in G(G _X→Y (x))-G(y), G(y) is the gradient image of the real map y, and G(G _X→Y (x)) is the gradient image of the generated fake map G _X→Y (x); 2) Content loss: aims to ensure that the generated map is similar to the ground truth in content. and/> is the circulation loss, /> and/> is a direct loss; in the unsupervised stage, a cyclic loss is used; in the supervised stage, a direct loss is used; In the formula, represents the cyclic loss of remote sensing images, λ is the fine-tuning coefficient, L1u represents the L1 loss in the unsupervised stage,/> The pixel L1 loss is used to calculate the difference between pixels through the L1 loss, so that the generated map and the remote sensing image maintain the cycle consistency in content;/> represents sampling from remote sensing image samples, G _Y→X (G _X→Y (x))-x represents the cycle loss of calculating the fake map image generated by G _X→Y (x), and then generating the fake remote sensing image and the real remote sensing image x through G _Y→X ; In the formula, represents the cycle loss of the map,/> Represents sampling from map samples; by introducing topological consistency loss/> To maintain the cyclic consistency between the topological structure of the generated image and the topological structure of the target image; G _X→Y (G _Y→X (y))-y represents the cyclic loss of the fake map image and the real map image y generated by G _X→Y after calculating the fake remote sensing image generated by G _Y→X (y); Direct loss from maps to remote sensing images Maintain content consistency through L1 loss function; /> Where λ is the fine-tuning coefficient, L1 represents the L1 loss, /> represents the pixel L1 loss; G _Y→X (y)-x represents the loss between the calculated false remote sensing image and the true remote sensing image; Direct loss from remote sensing images to maps By/> The loss keeps the generated map consistent with the remote sensing image in topological structure; G _X→Y (x)-y represents the loss between the calculated fake map image and the real map image; 3) Adversarial loss: The discriminator is used to determine the difference between the generated image and the real image. The purpose of the generator G is to minimize the loss function value, while the purpose of the discriminator D is to maximize the loss function value. The formula is: Adversarial Loss from Remote Sensing Images to Maps G _X→Y is a generator from remote sensing images to maps, and the generated image is input to the discriminator D _Y for discrimination; Adversarial Loss from Map to Remote Sensing Imagery G _Y→X is a map-to-remote sensing image generator, and the generated image is sent to the discriminator D _X for discrimination; 4) Identity loss: used to ensure the consistency between the converted image and the original image; for example, the map generated by passing the map into the generator G _X→Y should be as consistent as possible with the input map, that is, the content and color of the map itself; its formula is: