CN117351370A - Automatic out-of-range building pattern spot extraction method based on high Jing Weixing image - Google Patents

Abstract

The invention relates to the technical field related to remote sensing applications, and discloses an automatic extraction method of over-range building patches based on high-view satellite images, which includes: remote sensing image preprocessing; sample data set production; construction of a convolutional neural network; and use of sample training Network model; model inference and prediction. This invention mainly automatically extracts construction land through the improved U-Net model, uses resnet50 as the encoder part of the U-Net model, uses its powerful feature extraction capabilities to better capture feature information at different scales, and introduces the CBAM hybrid attention mechanism. The model can adaptively adjust the importance of different channels and spatial positions in the feature map to better focus on the key areas of the architectural target. At the same time, the FPN feature pyramid structure is added to the model to integrate multi-scale features at different levels. Feature information allows the model to better obtain contextual information of building targets, thereby enhancing the robustness and accuracy of building land extraction.

Description

An automatic extraction method for over-range building patches based on high-view satellite images

Technical field

The invention relates to the technical field related to remote sensing applications, specifically an automatic extraction method of over-range building spots based on high-view satellite images.

Background technique

With the rapid development of urbanization, various types of land use have been transferred to construction land. Dynamic monitoring of construction land is needed to better provide rapid and specific post-approval implementation change information for land space use control and optimize and improve the monitoring and evaluation system. . The new generation of sensors can acquire higher-resolution images, among which the spatial resolution of high-view satellite images can reach 0.5m, which greatly enhances the feature information and positioning information of ground objects, and can clearly display the spatial distribution of over-scale construction areas. conditions, boundary information, and construction status. However, the pixels of a single image also increase geometrically, and the construction land structure is complex, the spectral characteristics are diverse, and there is a large amount of interference information and similar feature information. It is difficult to extract different types of construction land boundary information from high-resolution images.

Many scholars have conducted a lot of research on building land information extraction, which can be summarized into three types according to the extraction principles: 1. Based on the geometric characteristics of the building land itself, relying on edge information and corner information to identify the external contour of the building land to further extract the construction land. 2. Extract construction land with the help of auxiliary information, combined with auxiliary information such as location and elevation obtained from lidar data, synthetic aperture radar, digital surface model and other data. 3. Based on the segmentation method, the image is divided into each unit, and information is extracted based on the characteristics within the unit. However, the above has the following shortcomings: 1. The traditional method relies on potential features such as texture, shape, and shadow of the ground objects. It has weak ability to express building land information and cannot rely on advanced semantic features such as spatial context information to extract building land. There are loopholes. The phenomena of mentioning and wrong mentioning. 2. The segmentation effect can be enhanced to a certain extent with the help of auxiliary information, but there are certain limitations due to the high cost of obtaining auxiliary information, and the extraction of building land is easily limited by the accuracy of auxiliary information. 3. In high spatial resolution images, spatial relationships and intra-category differences increase, and the segmentation scale is difficult to determine. There are problems such as under-segmentation and over-segmentation, resulting in fragmented and contiguous building land extraction. Segmentation parameters must be based on specific areas. Setting the features of ground objects has subjective factors and poor generalization ability.

Contents of the invention

The purpose of the present invention is to provide an automatic extraction method of over-range building patches based on high-view satellite images, using an improved U-Net model to extract building information, to overcome the above-mentioned defects in the existing technology, and improve the construction land use Extraction accuracy.

The present invention is achieved through the following technical solutions.

An automatic extraction method of over-range building patches based on high-view satellite images of the present invention includes:

Step S1: Remote sensing image preprocessing;

Step S2: Sample data set production;

Step S3: Construct a convolutional neural network, including:

S3.1: Improve based on the U-Net network structure, using resnet50 as the encoding part of the U-Net network;

S3.2: Introducing the CBAM hybrid attention mechanism in the improved U-Net network encoder part;

S3.3: Introduce the FPN feature pyramid structure in the improved U-Net network decoder part;

Step S4: Use samples to train the network model, including:

S4.1: Perform data augmentation on the manually outlined data set;

S4.2: Divide all augmented data sets into training sets, validation sets and test sets;

S4.3: Operating environment settings;

S4.4: Network parameter setting;

Step S5: Model inference prediction, including:

S5.1: Set the optimal weight of the model and input the test set into the network for inference prediction;

S5.2: Application of over-range building patch extraction, using the expansion cropping and splicing method to extract over-range building patches.

Further technical solutions, in the remote sensing image preprocessing in step S1, include image radiation correction, orthorectification, image fusion and light and color uniformity.

Further technical solution, in the sample data set production in step S2, use Arcgis to label the building targets through Gaojing-1 satellite images, use 512×512pixels to crop the original images and labels respectively, and generate tiff format Remote sensing sample data set.

As a further technical solution, S3.1 in step S3 specifically includes:

Replace the first layer output feature map Efeat1 of the resnet50 network stage0, the output feature map Efeat2 of stage1, the output feature map Efeat3 of stage2, the output feature map Efeat4 of stage3, and the output feature map Efeat5 of stage4 with the corresponding features of the U-Net network encoder. layer.

As a further technical solution, S3.2 in step S3 specifically includes:

The CBAM hybrid attention mechanism is added after the feature map Efeat1 and feature map Efeat5 respectively, so that the network can focus more on key areas.

As a further technical solution, S3.3 in step S3 specifically includes:

First, the output feature maps Dfeat1, Dfeat2, Dfeat3, and Dfeat4 of each layer of the encoder are fused from top to bottom to obtain the feature map Dfeat5. Then Dfeat5 is upsampled 8 times and then fused with Dfeat1 to obtain Dfeat6. Finally, Dfeat6 is Output after 2 times upsampling processing.

Further technical solution, the data augmentation processing in S4.1 mainly includes random rotation, horizontal flipping, vertical flipping and diagonal mirroring operations;

In a further technical solution, the division ratio of the data set described in S4.2 is 8:1:1.

As a further technical solution, the running environment settings described in S4.3 specifically select the windows operating system, Pytorch deep learning framework and VScode language compiler.

As a further technical solution, the network parameter settings described in S4.4 specifically select the Adam optimizer, select a combination of cross-entropy loss and dice loss as the loss function of the network, set the initial learning rate to 0.0007, and select poly learning strategy and ReLu activation. function.

As a further technical solution, S5.1 in step S5 specifically includes:

First, the optimal weight of the model is set, and then the test set is input into the network for inference prediction. In order to evaluate the actual learning ability of the network, the precision (Precision), recall (Recall), and F1 value (F1- score) and single-class intersection and union (Iou).

As a further technical solution, S5.2 in step S5 specifically includes:

In the application process of over-scale building patch extraction, the remote sensing images of the 500m buffer zone of the project's red line are selected. Since the network training size is 512×512, the remote sensing images to be predicted should be cropped to a fixed size and then spliced, and expansion cropping should be used. The splicing method extracts over-range building patches to avoid obvious splicing traces in the inference prediction results.

Beneficial effects of the present invention:

The present invention is aimed at monitoring out-of-range building pattern spots of construction projects in land space use control, so as to solve the problem of difficult and slow discovery in the monitoring of out-of-range building pattern spots in construction projects. The surface conditions of over-the-range building patches are complex and changeable, and the texture features of the identification targets displayed on high-resolution remote sensing images are very complex, which places higher requirements on the detail extraction capabilities and edge detection capabilities of the deep learning model. This invention uses the improved U-NET deep learning model to perform accurate location identification and boundary segmentation of building targets, which has the following benefits:

Use resnet50 as the encoder part of the U-Net model to use its powerful feature extraction capabilities to better capture feature information at different scales.

The introduction of the CBAM hybrid attention mechanism allows the model to adaptively adjust the importance of different channels and spatial positions in the feature map, thereby better focusing on key areas of the architectural target.

Adding the FPN feature pyramid structure to the model allows the model to better obtain contextual information of building targets by integrating multi-scale feature information at different levels, thereby enhancing the robustness and accuracy of building land extraction.

Description of drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only For some embodiments of the invention, those of ordinary skill in the art can also obtain other drawings based on these drawings without exerting creative efforts.

The present invention will be further described below in conjunction with the accompanying drawings and examples.

Figure 1 is a flow chart of an automatic extraction method of over-range building patches based on high-view satellite images according to the present invention;

Figure 2 is a schematic diagram of the network introducing resnet50 in the process of building a deep learning model;

Figure 3 is a schematic diagram of the network that introduces the CBAM hybrid attention mechanism in the process of building a deep learning model;

Figure 4 is a schematic diagram of the improved U-Net network in the present invention;

Figure 5 is a schematic diagram for comparison of building pattern extraction from the Gaojing No. 1 image according to the present invention;

Figure 6 is a schematic diagram of extracting building patches in the 500m buffer zone of the approved project red line;

Figure 7 is a schematic diagram comparing the effect of expansion cutting and splicing.

Implementation

Figure 1 shows the method flow chart provided by the patent of the present invention. The present invention is an automatic extraction method of over-range building spots based on high-view satellite images. The specific working steps are as follows:

Step S1: Remote sensing image preprocessing;

Step 1.1: Radiation correction; in the process of remote sensing imaging, the sensor itself has a certain absolute error. In addition, the sky light caused by atmospheric scattering and cloud reflection and the radiation energy of ground objects will also enter the remote sensing detector together, resulting in remote sensing The amount of radiation is distorted, the image contrast is reduced, and the quality of remote sensing images is reduced. Errors caused by sensors and atmospheric effects are eliminated through radiation correction.

Step 1.2: Orthorectification; due to the terrain, camera geometric characteristics and errors caused by the sensor during the imaging process, obvious geometric distortion and ground elevation will cause parallax, and the extraction of building land information requires high positioning accuracy and contour accuracy. , so the above effects are eliminated by orthorectifying the image.

Step 1.3: Image fusion; In order to obtain multi-channel high-resolution images, image fusion is used to fuse panchromatic sensor images and multispectral sensor images to retain the color and spectral information of the multispectral data, while enhancing the details and details of the image. resolution.

Step 1.4: Light and color uniformity; during the image acquisition process, due to imaging time and various environmental factors, there are many differences in color, brightness and other aspects of the images between each flight strip, so the fused image is subjected to light and color uniformity processing. , so that the overall accuracy of each scene image is consistent, the tone is uniform, the texture is clear, the contrast is moderate, and the color transition is natural.

Step S2: Sample data set production; use Arcgis software to manually outline the building targets in the Gaojing No. 1 image, assign the attributes of the target features as 255, and assign the values of other features as 0, and convert them into raster images in tiff format. The original image and label are then cropped according to 512×512pixels to generate a remote sensing building sample data set.

Step S3: Construct a convolutional neural network;

Step 3.1: Use resnet50 as the encoding part of the U-Net network, and use the first layer output feature map Dfeat1 of the resnet50 network stage0, the output feature map Dfeat2 of stage1, the output feature map Dfeat3 of stage2, the output feature map Dfeat4 of stage3, and the output feature map Dfeat4 of stage4. The output feature map Dfeat5 replaces the feature layer corresponding to the U-Net network encoder. The core idea of the resnet50 network is to design a residual structure. Its residual block consists of three convolutional layers: 1×1, 3×3 and 1×1. The mathematical expression of the residual structure is as follows:

x _i+1 = x _i + F( x _i ,W _i )

Among them, _xi is the input feature, xi ₊₁ is the output feature, _xi and xi ₊₁ have the same number of channels, F( _xi ,W _i ) is the residual part, and W _i represents the convolution operation.

Step 3.2: Introduce the CBAM hybrid attention mechanism into the improved U-Net network encoder part, and add the CBAM hybrid attention mechanism after the feature map Dfeat1 and feature map Dfeat5 respectively, so that the network can focus more on key areas. Figure 3 shows the network diagram of the CBAM hybrid attention mechanism. The mathematical expression of CBAM hybrid attention mechanism is as follows:

A _c (x _i ) = σ(W ₁ (W ₀ (x _iavg )) + W ₁ (W ₀ (x _imax ))

A _s (x _i ) = σ(W _7×7 (x _iavg ⊙ x _imax ))

X _i+1 = A _s (A _c (x _i )) Ⓧ A _c (x _i )

Among them, x _i is the input feature, x _i+1 is the output feature, W ₀ and W ₁ represent the convolution operation using the ReLu activation function and the sigmod activation function respectively, x _iavg and x _imax represent the average pooling and the maximum pooling respectively. ation, A _c ( _xi ) represents the channel attention function, A _s ( _xi ) represents the spatial attention function, σ is the sigmod activation function, ⊙ represents the splicing operation, and Ⓧ represents element-wise multiplication.

Step 3.3: Introduce the FPN feature pyramid structure into the improved U-Net network decoder part, first fuse the output feature maps Efeat1, Efeat2, Efeat3, and Efeat4 of each layer of the encoder from top to bottom to obtain the feature map Efeat5, and then Efeat5 is upsampled 8 times and then fused with Efeat1 to obtain Efeat6. Finally, Efeat6 is upsampled 2 times and output. Figure 4 shows a schematic diagram of the improved U-Net network adding the FPN feature pyramid structure.

Step S4: Use samples to train the network model;

Step 4.1: Perform data augmentation on the manually outlined data set, and perform random rotation, horizontal flip, vertical flip and diagonal mirroring operations of 0-360° on the original image and labels to increase the richness of the data.

Step 4.2: Divide all augmented data sets into training sets, validation sets, and test sets in a ratio of 8:1:1.

Step 4.3: Set up the running environment; select the windows operating system, Pytorch deep learning framework and VScode language compiler, run the memory 64G, and the graphics card is NVIDIA GeForce RTX 3060.

Step 4.4: Network parameter setting; select Adam optimizer. Adam absorbs the advantages of adaptive learning rate gradient descent algorithm and momentum gradient descent algorithm, can adapt to sparse gradients, and can alleviate gradient oscillation. A combination of cross-entropy loss and dice loss is selected as the hybrid loss function of the network. This type of hybrid function can not only focus on the difference between the output results and the actual results, but also take into account the problem of type imbalance in the sample. The initial learning rate is set to 0.0007, the poly learning strategy and the ReLu activation function are selected, the batchsize is set to 2, and the epoch is set to 300. The mathematical expressions of the Adam optimizer and hybrid loss function are as follows.

Adam optimizer mathematical expression:

m _t =β ₁ m _t-1 + (1-β ₁ )g _t

v _t = β ₂ m _t-1 + (1-β ₂ )g _t ²

Among them, β ₁ is the exponential weighted average of the gradient, β ₂ is the exponential weighted average of the square of the gradient, g _t represents the parameter gradient, m _t is the first-order moment estimate at time t, v _t is the second-order moment at time t estimate, and/> is the moving average corrected for the deviation of the corresponding gradient,/> Updated parameters,/> is a constant close to 0.

Mathematical expression of hybrid loss function:

L _cross = -(y*log(p)+(1-y)*log(1-p))

L _dice =

L _loss = L _cross + L _dice

Among them, y represents the label value of the input network, p represents the prediction probability value, A represents the prediction map, and B represents the label map.

Step S5: Model inference and prediction.

Step 5.1: When the loss function curve tends to be stable or converges, select the last weight in the convergence stage as the prediction weight, use the test set for inference prediction, and calculate the Precision, Recall, F1-score and Iou of the test set for quantitative testing. The actual learning capabilities of the network. In order to verify the performance of the algorithm proposed in this patent, the classic U-Net and U-Net based on the resnet50 backbone network (hereinafter referred to as R50_U-Net) were selected for comparative analysis. Figure 5 shows the results of the three algorithms extracting the Gaojing No. 1 building pattern on the test set. Each row represents, from left to right, the original image, the label image, the U-Net network building patch extraction results improved by the invention's patent, the R50_U-Net network building patch extraction results, and the U-Net network building patch extraction results. As a result, it can be seen that the segmentation result of the method proposed in the patent of the present invention is closer to the label map, the phenomenon of missed questions and wrong mentions is significantly reduced, and the extracted building pattern outlines are clear and the segmentation effect is better.

Step 5.2: In order to reflect the role of the method proposed in this patent in the monitoring of out-of-range building patterns in construction projects in land space use control, several red lines of approved projects are now selected, and a 500m buffer zone is set for image cropping. In order to avoid The inference and prediction results show obvious splicing traces, which reduces the accuracy of building extraction. This patent uses the expansion cropping and splicing method to perform inference and prediction on the 500m buffer image of the approved project red line (as shown in Figure 6), while using conventional cropping. The splicing methods are compared (as shown in Figure 7).

The specific working steps of dilated cropping and splicing prediction are as follows:

Step 5.3.1: Obtain the width and height of the image to be predicted, set the sliding step size to 256, divide the width and height by 256 and take the remainder. If the remainder is less than 256, fill the width and height to 256 respectively.

Step 5.3.1: Set the sliding window to 512, and perform sliding cropping of the predicted image according to the sliding step size.

Step 5.3.2: Perform inference prediction on the sliding cropped image to obtain the prediction result map of the corresponding image.

Step 5.3.3: Take the 256×256 size image at the center of each prediction result image, and then splice them together in order.

The above embodiments are only for illustrating the technical concepts and characteristics of the present invention. Their purpose is to enable those familiar with the technology in this field to understand the content of the present invention and implement it, and cannot limit the scope of protection of the present invention. All equivalent changes or modifications made based on the spirit of the present invention should be included in the protection scope of the present invention.

Claims (9)

1. An automatic extraction method for out-of-range building pattern spots based on high Jing Weixing images is characterized by comprising the following steps:

step S1: preprocessing a remote sensing image;

step S2: sample data set preparation;

step S3: constructing a convolutional neural network, comprising:

s3.1: based on the improvement of the U-Net network structure, the res 50 is used as the coding part of the U-Net network;

s3.2: introducing a CBAM mixed attention mechanism in the improved U-Net network encoder part;

s3.3: introducing an FPN feature pyramid structure into the improved U-Net network decoder part;

step S4: training a network model using samples, comprising:

s4.1: performing data augmentation processing on the manually delineated data set;

s4.2: dividing all the amplified data sets according to a training set, a verification set and a test set;

s4.3: setting an operation environment;

s4.4: setting network parameters;

step S5: model inference prediction, comprising:

s5.1: setting the optimal weight of the model, and inputting the test set into a network for reasoning and predicting;

s5.2: and (3) extracting the out-of-range building pattern spots by adopting an expansion cutting and splicing method.

2. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: in the remote sensing image preprocessing of step S1, image radiation correction, orthographic correction, image fusion and dodging are included.

3. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: in the sample data set production of the step S2, the building target is marked with samples by Arcgis through the first satellite image of the scene, and the original image and the tag are respectively cut by 512×512pixels, so as to generate a tiff remote sensing sample data set.

4. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: the step S3.1 in the step S3 specifically includes:

and replacing the feature layers corresponding to the U-Net network encoder with the first layer output feature map Efeat1 of the network stage0 of the resnet50, the output feature map Efeat2 of the stage1, the output feature map Efeat3 of the stage2, the output feature map Efeat4 of the stage3 and the output feature map Efeat5 of the stage 4.

5. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: the step S3.2 in the step S3 specifically includes:

after the feature map Efeat1 and the feature map Efeat5 are respectively added with a CBAM mixed attention mechanism, so that the network is focused on a key area.

6. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: the step S3.3 in the step S3 specifically includes:

the method comprises the steps of firstly, carrying out information fusion on output feature graphs Dfeat1, dfeat2, dfeat3 and Dfeat4 of each layer of an encoder from top to bottom to obtain a feature graph Dfeat5, then carrying out 8 times of upsampling treatment on the Dfeat5, then carrying out fusion with the Dfeat1 to obtain Dfeat6, and finally carrying out 2 times of upsampling treatment on the Dfeat6 and outputting.

7. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that:

the data augmentation processing in the S4.1 mainly comprises random rotation, horizontal overturn, vertical overturn and diagonal mirror image operation;

the dividing ratio of the data set in S4.2 is 8:1:1, a step of;

the running environment in the S4.3 is set by a windows operating system, a Pytorch deep learning frame and a VScode language compiler;

the network parameter setting in S4.4 specifically selects Adam optimizers, selects a combination of cross entropy loss and dice loss as a loss function of the network, sets an initial learning rate to 0.0007, and selects a poly learning strategy and a ReLu activation function.

8. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: the step S5.1 in the step S5 specifically includes:

firstly, setting the optimal weight of a model, inputting a test set into a network to perform reasoning prediction, and calculating the accuracy (Precision), recall (Recall), F1 value (F1-score) and single-class cross-merging ratio (Iou) of the test set for evaluating the actual learning capability of the network.

9. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: the step S5.2 in the step S5 specifically includes:

in the process of extracting and applying the beyond-range building pattern spots, selecting remote sensing images in a project red line 500m buffer area, cutting the remote sensing images to be predicted in a fixed size and then splicing the remote sensing images due to the fact that the network training size is 512 multiplied by 512, and extracting the beyond-range building pattern spots by adopting an expansion cutting splicing method to avoid obvious splicing marks from appearing in the reasoning and predicting results.