CN117351370A - Automatic out-of-range building pattern spot extraction method based on high Jing Weixing image - Google Patents

Abstract

The invention relates to the technical field of remote sensing application, and discloses an automatic extraction method for out-of-range building pattern spots based on high Jing Weixing images, which comprises the following steps: preprocessing a remote sensing image; sample data set preparation; constructing a convolutional neural network; training a network model using the samples; model reasoning prediction. According to the invention, the building site is automatically extracted through the improved U-Net model, the resnet50 is used as an encoder part of the U-Net model, the characteristic information of different scales is better captured by utilizing the strong characteristic extraction capability of the resnet50, and the importance of different channels and space positions in a characteristic diagram can be adaptively adjusted by introducing a CBAM mixed attention mechanism, so that the model is better focused on a key region of a building target, meanwhile, an FPN characteristic pyramid structure is added into the model, and the model can better acquire the context information of the building target by fusing the characteristic information of multiple scales on different levels, thereby enhancing the robustness and accuracy of the extraction of the building site.

Description

Automatic out-of-range building pattern spot extraction method based on high Jing Weixing image

Technical Field

The invention relates to the technical field related to remote sensing application, in particular to an automatic extraction method for out-of-range building pattern spots based on high Jing Weixing images.

Background

With the rapid development of town, various land uses are changed into building lands, dynamic monitoring is needed to be carried out on the building lands, so that rapid and specific construction land lot post-implementation change information is provided for the control of the national land space use, and a monitoring evaluation system is optimized and perfected. The new generation sensor can acquire images with higher resolution, wherein the spatial resolution of the image of the high-scene satellite can reach 0.5m, the feature information and the positioning information of the ground feature are greatly enhanced, and the spatial distribution condition, the boundary information and the construction condition of the construction area with an out-of-range construction can be clearly displayed. However, the geometric multiple of the pixels of a single image is increased, the construction land structure is complex, the spectrum characteristics are various, a large amount of interference information and similar feature information exist, and it is difficult to extract different kinds of construction land boundary information from the high-resolution image.

Many scholars have conducted a great deal of research on the extraction of construction site information, and three types can be summarized according to the extraction principle: 1. based on the self geometric characteristics of the construction land, the construction land is further extracted by means of identifying the external contour of the construction land according to the edge information and the corner information. 2. And extracting the construction land by means of auxiliary information in combination with auxiliary information such as positions and elevations acquired from laser radar data, synthetic aperture radar, digital surface models and the like. 3. Based on the segmentation method, the image is segmented into each unit, and information extraction is performed according to the characteristics in the units. However, the above has the following disadvantages: 1. the traditional method relies on potential characteristics such as texture, shape, shadow and the like of the ground object, has weak expression capability on the information of the building ground, cannot extract the building ground by means of advanced semantic characteristics such as space context information and the like, and has the phenomena of miss-extraction and miss-extraction. 2. The segmentation effect can be enhanced to a certain extent by means of the auxiliary information, but there is a limit due to the high cost of acquiring the auxiliary information, and the extraction for the construction site is easily limited by the accuracy of the auxiliary information. 3. The space relation and intra-category difference in the high-spatial resolution image are increased, the segmentation scale is difficult to determine, the problems of under segmentation, over segmentation and the like exist, the situation that the construction land is extracted more fragly, connected and the like is caused, the segmentation parameters are required to be set according to the specific regional ground feature, subjective factors exist, and the generalization capability is poor.

Disclosure of Invention

The invention aims to provide an automatic extraction method of out-of-range building pattern spots based on a high Jing Weixing image, which utilizes an improved U-Net model to extract building information, overcomes the defects in the prior art and improves the accuracy of building land extraction.

The invention is realized by the following technical scheme.

The invention discloses an automatic extraction method of out-of-range building pattern spots based on a high Jing Weixing image, which comprises the following steps:

step S1: preprocessing a remote sensing image;

step S2: sample data set preparation;

step S3: constructing a convolutional neural network, comprising:

s3.1: based on the improvement of the U-Net network structure, the res 50 is used as the coding part of the U-Net network;

s3.2: introducing a CBAM mixed attention mechanism in the improved U-Net network encoder part;

s3.3: introducing an FPN feature pyramid structure into the improved U-Net network decoder part;

step S4: training a network model using samples, comprising:

s4.1: performing data augmentation processing on the manually delineated data set;

s4.2: dividing all the amplified data sets according to a training set, a verification set and a test set;

s4.3: setting an operation environment;

s4.4: setting network parameters;

step S5: model inference prediction, comprising:

s5.1: setting the optimal weight of the model, and inputting the test set into a network for reasoning and predicting;

s5.2: and (3) extracting the out-of-range building pattern spots by adopting an expansion cutting and splicing method.

In a further technical scheme, in the remote sensing image preprocessing of step S1, the method includes image radiation correction, orthographic correction, image fusion and dodging.

In a further technical scheme, in the sample data set manufacturing of the step S2, the object of the building is marked with samples by Arcgis through the first satellite image of the scene, and the original image and the tag are respectively cut by 512×512pixels, so as to generate a tiff-format remote sensing sample data set.

According to a further technical scheme, the step S3.1 in the step S3 specifically includes:

and replacing the feature layers corresponding to the U-Net network encoder with the first layer output feature map Efeat1 of the network stage0 of the resnet50, the output feature map Efeat2 of the stage1, the output feature map Efeat3 of the stage2, the output feature map Efeat4 of the stage3 and the output feature map Efeat5 of the stage 4.

According to a further technical scheme, the step S3.2 in the step S3 specifically includes:

after the feature map Efeat1 and the feature map Efeat5 are respectively added with a CBAM mixed attention mechanism, so that the network is focused on a key area.

According to a further technical scheme, the step S3.3 in the step S3 specifically includes:

the method comprises the steps of firstly, carrying out information fusion on output feature graphs Dfeat1, dfeat2, dfeat3 and Dfeat4 of each layer of an encoder from top to bottom to obtain a feature graph Dfeat5, then carrying out 8 times of upsampling treatment on the Dfeat5, then carrying out fusion with the Dfeat1 to obtain Dfeat6, and finally carrying out 2 times of upsampling treatment on the Dfeat6 and outputting.

According to a further technical scheme, the data augmentation processing in the step S4.1 mainly comprises random rotation, horizontal overturning, vertical overturning and diagonal mirroring operation;

according to a further technical scheme, the dividing ratio of the data set in the step S4.2 is 8:1:1.

according to a further technical scheme, the running environment in S4.3 is specifically selected from a windows operating system, a Pytorch deep learning framework and a VScode language compiler.

According to a further technical scheme, in the step S4.4, the network parameter setting specifically selects an Adam optimizer, selects a combination of cross entropy loss and dice loss as a loss function of the network, sets an initial learning rate to be 0.0007, and selects a poly learning strategy and a ReLu activation function.

According to a further technical scheme, the step S5.1 in the step S5 specifically includes:

firstly, setting the optimal weight of a model, inputting a test set into a network to perform reasoning prediction, and calculating the accuracy (Precision), recall (Recall), F1 value (F1-score) and single-class cross-merging ratio (Iou) of the test set for evaluating the actual learning capability of the network.

According to a further technical scheme, the step S5.2 in the step S5 specifically includes:

in the process of extracting and applying the beyond-range building pattern spots, selecting remote sensing images in a project red line 500m buffer area, cutting the remote sensing images to be predicted in a fixed size and then splicing the remote sensing images due to the fact that the network training size is 512 multiplied by 512, and extracting the beyond-range building pattern spots by adopting an expansion cutting splicing method to avoid obvious splicing marks from appearing in the reasoning and predicting results.

The invention has the beneficial effects that:

the invention aims at the monitoring work of the construction project out-of-range building pattern spots in the control of the application of the homeland space, so as to solve the problem of difficult and slow discovery in the monitoring of the construction project out-of-range building pattern spots. The ground surface condition of the out-of-range building map spots is complex and changeable, the texture features of the recognition targets on the high-resolution remote sensing images are very complex, and high requirements are put on the detail extraction capability and the edge detection capability of the deep learning model. The invention uses the improved U-NET deep learning model to accurately identify the position and divide the boundary of the building target, and has the following benefits:

using the resnet50 as the encoder portion of the U-Net model, feature information of different scales is better captured with its powerful feature extraction capabilities.

The introduction of CBAM mixed-attention mechanisms allows the model to adaptively adjust the importance of different channels and spatial locations in the feature map to better focus on critical areas of the architectural goals.

And adding the FPN characteristic pyramid structure into the model, and fusing multi-scale characteristic information on different levels to enable the model to better acquire the context information of the building target, so that the robustness and accuracy of the building land extraction are enhanced.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only some embodiments of the invention, and that other drawings can be obtained from these drawings without inventive effort for a person skilled in the art.

The invention will be further described with reference to the drawings and examples.

FIG. 1 is a flow chart of an automatic extraction method of out-of-range building spots based on high Jing Weixing images;

FIG. 2 is a schematic diagram of a network in which a resnet50 is introduced in the process of constructing a deep learning model;

FIG. 3 is a schematic diagram of a network incorporating a CBAM mixed attention mechanism in the process of constructing a deep learning model;

FIG. 4 is a schematic diagram of an improved U-Net network in accordance with the present invention;

FIG. 5 is a schematic diagram showing the extraction and comparison of the pattern spots of the first-image building of the high scene of the invention;

FIG. 6 is a schematic diagram of a 500m buffer building spot extraction for a red line of a batched project;

fig. 7 is a schematic diagram showing the comparison of expansion cutting and splicing effects.

Description of the embodiments

Fig. 1 is a flowchart of a method provided by the present invention, and the method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image comprises the following specific working steps:

step S1: preprocessing a remote sensing image;

step 1.1: radiation correction; in the process of remote sensing imaging, a certain absolute error exists in the sensor, in addition, sky light formed by atmospheric scattering and cloud layer reflection can enter the remote sensing detector together with the radiation energy of a ground object target, so that remote sensing radiation amount distortion is caused, image contrast is reduced, the quality of a remote sensing image is reduced, and errors generated by the sensor, atmospheric influence and the like are eliminated through radiation correction.

Step 1.2: correcting orthographic emission; the topography, the geometric characteristics of a camera and errors brought by the camera and the sensor can cause obvious geometric distortion and parallax caused by high Cheng Hui ground, and the construction land information extraction has higher requirements on positioning precision and contour precision, so that the influence is eliminated by carrying out orthographic correction on the image.

Step 1.3: fusing images; in order to acquire the multichannel high-resolution image, the full-color sensor image and the multispectral sensor image are fused in an image fusion mode, so that the color and spectral information of multispectral data are reserved, and meanwhile, the detail and resolution of the image are enhanced.

Step 1.4: homogenizing light and color; in the image acquisition process, because various differences of chromatic aberration, brightness and the like exist in the images among each navigation belt due to imaging time and various environmental factors, the fused images are subjected to light and color homogenizing treatment, so that the whole images are consistent in precision, uniform in color tone, clear in texture, moderate in contrast and natural in color transition.

Step S2: sample data set preparation; and manually sketching a first-image building target of the high scene by using Arcgis software, assigning 255 to the target feature attribute, assigning 0 to other features, converting the target feature attribute into a grid image in a tiff format, and then respectively cutting an original image and a label according to 512X 512pixels to generate a remote sensing building sample data set.

Step S3: constructing a convolutional neural network;

step 3.1: using the resnet50 as an encoding part of the U-Net network, replacing the feature layers corresponding to the U-Net network encoder with the first layer output feature map Dfeat1 of the resnet50 network stage0, the output feature map Dfeat2 of the stage1, the output feature map Dfeat3 of the stage2, the output feature map Dfeat4 of the stage3 and the output feature map Dfeat5 of the stage 4. The core idea of the resnet50 network is to design a residual structure, the residual block of which consists of three convolution layers 1×1, 3×3 and 1×1, and the mathematical expression of the residual structure is as follows:

x _i+1 = x _i + F( x _i ,W _i )

wherein x is _i To input features, x _i+1 To output characteristics, x _i And x _i+1 The number of channels is the same, F (x _i ,W _i ) As residual part, W _i Representing a convolution operation.

Step 3.2: the CBAM mixed attention mechanism is introduced into the improved U-Net network encoder, and is added after the characteristic diagram Dfeat1 and the characteristic diagram Dfeat5 respectively, so that the network is focused on a key area. Fig. 3 is a network schematic diagram of a CBAM mixed-attention mechanism. The mathematical expression of the CBAM mixed-attention mechanism is as follows:

A _c (x _i ) = σ(W ₁ (W ₀ (x _iavg )) + W ₁ (W ₀ (x _imax ))

A _s (x _i ) = σ(W _7×7 (x _iavg ⊙ x _imax ))

X _i+1 = A _s (A _c (x _i )) Ⓧ A _c (x _i )

wherein x is _i To input features, x _i+1 To output characteristics, W ₀ And W is ₁ Respectively, a convolution operation using a ReLu activation function and a sigmod activation function, x _iavg And x _imax Mean pooling and maximum pooling, respectively, A _c (x _i ) Representing a channel attention function, A _s (x _i ) Representing the spatial attention function, σ is the sigmod activation function, and as such, indicates the stitching operation, Ⓧ indicates the element-wise multiplication.

Step 3.3: and introducing an FPN characteristic pyramid structure into the improved U-Net network decoder, firstly, carrying out information fusion on output characteristic graphs of each layer of the encoder, namely, an Efeat1, an Efeat2, an Efeat3 and an Efeat4 from top to bottom to obtain a characteristic graph Efeat5, carrying out 8 times of upsampling treatment on the Efeat5, then fusing the upsampled processed Efeat5 with the Efeat1 to obtain an Efeat6, and finally, carrying out 2 times of upsampling treatment on the Efeat6 and outputting the upsampled processed Efeat 6. FIG. 4 is a schematic diagram of an improved U-Net network incorporating the FPN feature pyramid structure.

Step S4: training a network model using the samples;

step 4.1: and (3) carrying out data augmentation processing on the manually sketched data set, and simultaneously carrying out 0-360-degree random rotation, horizontal overturning, vertical overturning and diagonal mirroring operation on the original image and the label, so that the richness of the data is increased.

Step 4.2: all data sets after augmentation were combined according to 8:1:1, a training set, a validation set and a test set.

Step 4.3: setting an operation environment; and a windows operating system, a Pytorch deep learning frame and a VScode language compiler are selected, the memory 64G is operated, and the display card is NVIDIA GeForce RTX 3060.

Step 4.4: setting network parameters; the Adam optimizer is selected, so that the Adam absorbs the advantages of a gradient descent algorithm and a momentum gradient descent algorithm of a self-adaptive learning rate, can adapt to sparse gradients and can alleviate gradient oscillation. The combination of cross entropy loss and dice loss is selected as a mixed loss function of the network, and the mixed function can pay attention to the difference between an output result and an actual result and can also give consideration to the problem of unbalanced types in a sample. The initial learning rate was set to 0.0007, the poly learning strategy and ReLu activation function were chosen, the batch size was set to 2, and the epoch was set to 300. The mathematical expression of the Adam optimizer and the mixing loss function is shown below.

Adam optimizer mathematical expression:

m _t =β ₁ m _t-1 + (1-β ₁ )g _t

v _t = β ₂ m _t-1 + (1-β ₂ )g _t ²

wherein beta is ₁ Is an exponentially weighted average of gradients, beta ₂ An exponentially weighted average of the squares of the gradients, g _t Representing the parameter gradient, m _t For the first moment estimation at time t, v _t For the second moment estimation at time t,and->For the deviation corrected moving average of the corresponding gradients, +.>Updated parameters->Is a constant close to 0.

Mathematical expression of the mixing loss function:

L _cross = -(y*log(p)+(1-y)*log(1-p))

L _dice =

L _loss = L _cross + L _dice

wherein y represents a label value of an input network, p represents a prediction probability value, A represents a prediction graph, and B represents a table-type label graph.

Step S5: model reasoning prediction.

Step 5.1: when the loss function curve graph tends to be stable, namely convergence, the last weight in the convergence stage is selected as a prediction weight, reasoning and prediction are carried out by using a test set, precision, recall, F-score and Iou of the test set are calculated, and the actual learning capacity of the network is quantitatively tested. In order to verify the performance of the algorithm provided by the patent, classical U-Net and U-Net based on a reset 50 backbone network (hereinafter referred to as R50_U-Net) are selected for comparison analysis. Fig. 5 is a graph showing the results of three algorithms for extracting a first building patch from a test set. Each row from left to right respectively represents an original image, a label image, an improved U-Net network building image spot extraction result, an R50_U-Net network building image spot extraction result and an improved U-Net network building image spot extraction result, and the method provided by the invention is more similar to the label image in the segmentation result, so that the phenomena of missing questions and misplacement are obviously reduced, the extracted building image spot has clear outline and a better segmentation effect.

Step 5.2: in order to embody the effect of the method in construction project overseas building pattern spot monitoring work in the national space use control, a plurality of batched project red lines are selected, a buffer zone of 500m is set for image cutting, obvious splicing marks appear in the reasoning prediction result so as to reduce the accuracy of building extraction, the method for expansion cutting and splicing is adopted to carry out reasoning prediction on the image of the batched project red lines of 500m buffer zone (shown in figure 6), and meanwhile, the conventional cutting and splicing method is used for comparison (shown in figure 7).

The specific working steps of expansion cutting splicing prediction are as follows:

step 5.3.1: and acquiring the width and the height of the image to be predicted, setting the sliding step length to be 256, dividing the width and the height by 256 respectively, and filling the width and the height of the remainder less than 256 respectively to 256.

Step 5.3.1: the sliding window is set to 512, and the predicted image is subjected to sliding clipping according to the sliding step length.

Step 5.3.2: and carrying out reasoning and prediction on the sliding cut image, and obtaining a prediction result diagram of the corresponding image.

Step 5.3.3: and taking 256 multiplied by 256-size images of the central position of each prediction result graph, and then splicing the images in sequence.

The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the present invention and implement it without limiting the scope of the present invention. All equivalent changes or modifications made in accordance with the spirit of the present invention should be construed to be included in the scope of the present invention.

Claims (9)

1. An automatic extraction method for out-of-range building pattern spots based on high Jing Weixing images is characterized by comprising the following steps:

step S1: preprocessing a remote sensing image;

step S2: sample data set preparation;

step S3: constructing a convolutional neural network, comprising:

s3.1: based on the improvement of the U-Net network structure, the res 50 is used as the coding part of the U-Net network;

s3.2: introducing a CBAM mixed attention mechanism in the improved U-Net network encoder part;

s3.3: introducing an FPN feature pyramid structure into the improved U-Net network decoder part;

step S4: training a network model using samples, comprising:

s4.1: performing data augmentation processing on the manually delineated data set;

s4.2: dividing all the amplified data sets according to a training set, a verification set and a test set;

s4.3: setting an operation environment;

s4.4: setting network parameters;

step S5: model inference prediction, comprising:

s5.1: setting the optimal weight of the model, and inputting the test set into a network for reasoning and predicting;

s5.2: and (3) extracting the out-of-range building pattern spots by adopting an expansion cutting and splicing method.

2. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: in the remote sensing image preprocessing of step S1, image radiation correction, orthographic correction, image fusion and dodging are included.

3. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: in the sample data set production of the step S2, the building target is marked with samples by Arcgis through the first satellite image of the scene, and the original image and the tag are respectively cut by 512×512pixels, so as to generate a tiff remote sensing sample data set.

4. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: the step S3.1 in the step S3 specifically includes:

and replacing the feature layers corresponding to the U-Net network encoder with the first layer output feature map Efeat1 of the network stage0 of the resnet50, the output feature map Efeat2 of the stage1, the output feature map Efeat3 of the stage2, the output feature map Efeat4 of the stage3 and the output feature map Efeat5 of the stage 4.

5. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: the step S3.2 in the step S3 specifically includes:

after the feature map Efeat1 and the feature map Efeat5 are respectively added with a CBAM mixed attention mechanism, so that the network is focused on a key area.

6. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: the step S3.3 in the step S3 specifically includes:

the method comprises the steps of firstly, carrying out information fusion on output feature graphs Dfeat1, dfeat2, dfeat3 and Dfeat4 of each layer of an encoder from top to bottom to obtain a feature graph Dfeat5, then carrying out 8 times of upsampling treatment on the Dfeat5, then carrying out fusion with the Dfeat1 to obtain Dfeat6, and finally carrying out 2 times of upsampling treatment on the Dfeat6 and outputting.

7. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that:

the data augmentation processing in the S4.1 mainly comprises random rotation, horizontal overturn, vertical overturn and diagonal mirror image operation;

the dividing ratio of the data set in S4.2 is 8:1:1, a step of;

the running environment in the S4.3 is set by a windows operating system, a Pytorch deep learning frame and a VScode language compiler;

the network parameter setting in S4.4 specifically selects Adam optimizers, selects a combination of cross entropy loss and dice loss as a loss function of the network, sets an initial learning rate to 0.0007, and selects a poly learning strategy and a ReLu activation function.

8. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: the step S5.1 in the step S5 specifically includes:

firstly, setting the optimal weight of a model, inputting a test set into a network to perform reasoning prediction, and calculating the accuracy (Precision), recall (Recall), F1 value (F1-score) and single-class cross-merging ratio (Iou) of the test set for evaluating the actual learning capability of the network.

9. The method for automatically extracting the pattern spots of the over-range building based on the high Jing Weixing image according to claim 1, which is characterized in that: the step S5.2 in the step S5 specifically includes:

in the process of extracting and applying the beyond-range building pattern spots, selecting remote sensing images in a project red line 500m buffer area, cutting the remote sensing images to be predicted in a fixed size and then splicing the remote sensing images due to the fact that the network training size is 512 multiplied by 512, and extracting the beyond-range building pattern spots by adopting an expansion cutting splicing method to avoid obvious splicing marks from appearing in the reasoning and predicting results.