CN112884791B - Method for constructing large-scale remote sensing image semantic segmentation model training sample set - Google Patents
Method for constructing large-scale remote sensing image semantic segmentation model training sample set Download PDFInfo
- Publication number
- CN112884791B CN112884791B CN202110140509.6A CN202110140509A CN112884791B CN 112884791 B CN112884791 B CN 112884791B CN 202110140509 A CN202110140509 A CN 202110140509A CN 112884791 B CN112884791 B CN 112884791B
- Authority
- CN
- China
- Prior art keywords
- sample set
- remote sensing
- image
- sensing image
- semantic segmentation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 38
- 230000011218 segmentation Effects 0.000 title claims abstract description 36
- 238000012549 training Methods 0.000 title claims abstract description 30
- 239000013598 vector Substances 0.000 claims abstract description 24
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 19
- 238000004519 manufacturing process Methods 0.000 claims abstract description 14
- 238000005457 optimization Methods 0.000 claims abstract description 11
- 238000000605 extraction Methods 0.000 claims abstract description 8
- 230000004927 fusion Effects 0.000 claims description 13
- 238000005520 cutting process Methods 0.000 claims description 10
- 238000009826 distribution Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 4
- 239000003595 mist Substances 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract 1
- 238000011156 evaluation Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000000746 purification Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/40—Analysis of texture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method for constructing a large-scale remote sensing image semantic segmentation model training sample set, which is characterized in that existing remote sensing image vector data and a multi-stage remote sensing image are registered, and a primary sample set is automatically cut and extracted through a sliding window algorithm according to the density of image spots; extracting the characteristics of each image in the primary sample set, classifying by adopting a clustering algorithm, and removing samples with poor image quality to obtain an intermediate sample set; and inputting the intermediate sample set into the semantic segmentation model in batches for iterative optimization training, predicting the samples after each iterative optimization is completed, and eliminating error samples in the intermediate sample set to obtain a target sample set. The remarkable effects are as follows: the generation of a whole image and a mask with a large occupied space can be avoided, the sliding times of a sliding window are reduced, and the extraction speed and the data quality of a sample are improved; the purity of the correct sample in the sample set is improved, and the cost for manufacturing a large-scale sample set is greatly reduced.
Description
Technical Field
The invention relates to the technical field of remote sensing image feature extraction, remote sensing image semantic segmentation and sample set manufacturing, in particular to a method for constructing a large-scale remote sensing image semantic segmentation model training sample set.
Background
The high-resolution train satellite is successfully launched, PB-level remote sensing image data can be provided every year, meanwhile, a large amount of vector data are marked for ground objects by using the remote sensing images in projects such as 'national soil survey for the third time' and 'geographical national condition monitoring', if the data are effectively utilized to form a sample set with a uniform format and an ultra-large magnitude, a deep learning model with higher precision and higher generalization capability can be trained, and important support is provided for related projects and scientific research work. The currently mainstream methods for making a remote sensing image sample set mainly include the following two methods:
firstly, manually and directly carrying out visual interpretation and marking on a remote sensing image to manufacture a sample;
secondly, a sample set is manufactured by utilizing the existing manual interpretation vector result and the remote sensing image strictly corresponding to the manual interpretation vector result through an automatic cutting method.
Although the sample sets manufactured by the two methods have high precision, the sample set manufactured by the first method consumes a large amount of manpower and material resources, is extremely low in efficiency, and is difficult to construct a million-level sample set; in the second method, the vector and the remote sensing image must be strictly corresponding, i.e. the vector is obtained by sketching the image, but the strictly corresponding data is very limited, and the requirement of large-scale sample set production cannot be met, if the vector and other images are used for producing samples, the problems that the sample set obtained by automatic cutting has wrong samples caused by cloud coverage, shadow shielding and the like due to time phase change, image quality and other factors, and the sample label precision is not accurate due to incomplete matching of the vector and the image can be caused. Both methods cannot be used for large-scale sample set production, the first method causes the sample set production cost to be too high, and the second method causes the sample set quality to be difficult to guarantee.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a method for constructing a large-scale remote sensing image semantic segmentation model training sample set, which can register the existing vector data with a multi-period remote sensing image, automatically cut and extract samples, improve the manufacturing efficiency of the sample set and the quality precision of the sample set by optimizing the sample cutting and extracting process, introducing the methods of automatic detection and elimination of wrong samples and iterative optimization and purification of the sample set, and can be applied to the fields of remote sensing image semantic segmentation, remote sensing image data set manufacturing and the like.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
a method for constructing a large-scale remote sensing image semantic segmentation model training sample set is characterized by comprising the following steps:
step 1, registering existing remote sensing image vector data with a multi-stage remote sensing image, and automatically cutting and extracting a primary sample set through a sliding window algorithm;
in the extraction process of the primary sample set, when the distribution of the pattern spots in the remote sensing image vector data after registration is sparse, the steps are as follows:
a1, acquiring the four-to-range of each image spot, and expanding a buffer area with random size to the periphery on the basis of the four-to-range;
step A2, rasterizing the image spot range after the buffer area is expanded to obtain a mask of a target ground object in the area;
step A3, sliding a sliding window with fixed size on the mask, and calculating the ratio between the foreground and the background;
a4, when the ratio is larger than the set threshold, cutting the data of the area on the image and the mask as the image and the label;
step A5, integrating the cut data and the labels to obtain a primary sample set;
when the distribution of the pattern spots in the remote sensing image vector data after registration is dense, the method comprises the following steps:
b1, rasterizing the vector data of the remote sensing image to obtain a mask covering the whole remote sensing image;
step B2, using a sliding window with fixed size to slide on the remote sensing image and the mask simultaneously to extract data as an image and a label;
b3, integrating the cut data and the label to obtain a primary sample set;
step 2, extracting the characteristics of each image in the primary sample set, classifying by adopting a clustering algorithm, and removing samples with poor image quality to obtain an intermediate sample set;
and 3, inputting the intermediate sample set into the semantic segmentation model in batches for iterative optimization training, predicting the samples after each iterative optimization is completed, and eliminating error samples in the intermediate sample set to obtain a target sample set.
Further, the range size of the buffer area is 1-2 times of that of the sliding window.
Further, the step of obtaining the intermediate sample set in step 2 is:
2.1, respectively extracting texture features and color features of each image in the primary sample set;
2.2, fusing the texture features and the color features in a direct splicing mode to obtain a fusion feature set of the image;
and 2.3, clustering the fusion feature set of the image through a clustering algorithm, analyzing the category generated by clustering the primary sample set, deleting the samples containing cloud and shadow occlusion in the primary sample set, and reserving the samples containing the target ground objects.
Further, in step 2.1, GLCM is used to extract the texture features, and a color histogram is used to extract the color features.
Further, the clustering algorithm in step 2.3 adopts a density clustering algorithm, which specifically comprises the following steps:
step 2.3.1, setting a domain distance threshold value among elements, setting a minimum contained point number in a range taking the domain distance threshold value as a neighborhood radius, and if the number of elements contained in the neighborhood radius of one element is not less than the minimum contained point number, taking the element as a core object;
step 2.3.2, finding out a core object in the fusion feature set of the image, and adding the core object into the core object set;
step 2.3.3, randomly selecting an element which is not accessed in the core object set, firstly marking the element as accessed, then marking the type of the element, and finally adding the non-core object which is not accessed in the neighborhood radius into the seed set;
step 2.3.4, judging whether the seed set is empty, if so, finishing the generation of the cluster and entering step 2.3.6, otherwise, entering step 2.3.5;
2.3.5, randomly selecting a seed from the seed set, judging that the seed is a non-core object, if so, adding an object in the radius of the seed field into the seed set, and skipping to the step 2.3.4;
and 2.3.6, judging whether all the elements in the core object set are accessed, if so, finishing clustering the fusion feature set of the images, otherwise, skipping to the step 2.3.3.
Further, the semantic segmentation model is any one of an FCN network model, a SegNet network model, a cavity convolution model, a deep Lab series model, a RefineNet model and a PSPNet model.
Further, the step of obtaining the target sample set in step 3 is as follows:
step 3.1, training the selected semantic segmentation model by adopting an intermediate sample set;
step 3.2, predicting the samples in the middle sample set by adopting the trained semantic segmentation model, and comparing the prediction result with the true value of the samples to obtain the prediction accuracy of each sample;
3.3, deleting samples with prediction accuracy rates smaller than a preset coincidence rate threshold value;
and 3.4, judging whether the accuracy of the model is not less than a preset accuracy threshold, if so, finishing the manufacture of the target sample set, and otherwise, returning to the step 3.1.
Further, the training step of the semantic segmentation model in step 3.1 is as follows:
step 3.1.1, selecting a model, a loss function and an optimizer, and setting related parameters;
step 3.1.2, inputting samples in the intermediate sample set into the selected model in batches, extracting image features through downsampling by utilizing a cavity convolution structure, and upsampling the feature map to obtain an output result;
step 3.1.3, calculating the error between the output result and the label;
step 3.1.4, calculating the gradient of each parameter in the model, and optimizing the parameters in a back propagation mode;
and 3.1.5, judging whether the model is converged, if so, stopping training, and obtaining the accuracy rate of the model, otherwise, skipping to the step 3.1.2.
Further, the calculation formula of the error between the output result in step 3.1.3 and the label is:
Lfl＝-(1-y′*y)γlog(y′)，
wherein y' is a model output result, y is a model input result, and gamma is a hyper-parameter.
Compared with the prior art, the invention has the following beneficial effects:
(1) the image cutting process is improved, only the area near the image spots is rasterized, when the image spots are sparsely distributed, the generation of the whole image and a mask with a large occupied space can be avoided, the sliding times of a sliding window are reduced, and the extraction speed of a sample is improved.
(2) The samples with poor image quality in the sample set are automatically screened out through a feature extraction algorithm and an automatic clustering algorithm, the samples with cloud shielding and shadow shielding are removed, and the data quality of the sample set is improved.
(3) The sample set is purified in an iterative optimization semantic segmentation model mode, wrong samples with inaccurate labels are automatically screened out, the purity of correct samples in the sample set is improved, and the labor cost for manufacturing the sample set is greatly reduced.
(4) The method can fully utilize the existing vector results, correspondingly extract data and labels from the vectors and the multiple remote sensing images, and ensure the quality of the sample set by a series of methods, thereby reducing the cost for manufacturing a large-scale sample set.
Drawings
FIG. 1 is a flow chart of a method of the present invention;
FIG. 2 is a schematic diagram of a sample for screening for poor image quality according to the present invention;
FIG. 3 is a schematic representation of the purification of a sample set using DeepLabV3+ according to the present invention.
Detailed Description
The following provides a more detailed description of the embodiments and the operation of the present invention with reference to the accompanying drawings.
As shown in fig. 1, a method for constructing a large-scale remote sensing image semantic segmentation model training sample set includes the following specific steps:
step 1, registering existing remote sensing image vector data with a multi-stage remote sensing image, and automatically cutting and extracting a primary sample set through a sliding window algorithm;
in this example, in order to avoid generating a mask with a large occupied space and a whole image when the distribution of the image patches is sparse, in the extraction process of the primary sample set, respective processing is performed according to the density of the image patches to reduce the sliding frequency of the sliding window and improve the extraction speed of the sample.
When the distribution of the pattern spots in the remote sensing image vector data after registration is sparse, the steps are as follows:
a1, obtaining the range of four to four of each image spot, and expanding a buffer area with random size to the periphery on the basis of the range of four to four, wherein the range of the buffer area is 1-2 times of that of the sliding window to ensure that the size of the mask after rasterization is larger than that of the sliding window, and the buffer area is used for ensuring that the size of the mask is larger than that of the sliding window and simultaneously retaining the context information around the image spot;
step A2, rasterizing the image spot range after the buffer area is expanded to obtain a mask of a target ground object in the area;
step A3, slide on mask with fixed size sliding window (1024 × 1024), and pass
Calculate the ratio between foreground and background, CountstargetNumber of foreground pixels, CountsbackgrounIs the background pixel number;
a4, when the ratio is larger than the set threshold 0, cutting the image and the data of the area on the mask as the image and the label;
step A5, integrating the cut data and the labels to obtain a primary sample set;
when the distribution of the pattern spots in the remote sensing image vector data after registration is dense, the method comprises the following steps:
b1, rasterizing the vector data of the remote sensing image to obtain a mask covering the whole remote sensing image;
step B2, using a sliding window (1024 multiplied by 1024) with fixed size to simultaneously slide and extract data on the remote sensing image and the mask as an image and a label;
b3, integrating the cut data and the label to obtain a primary sample set;
step 2, extracting features of each image in the primary sample set, classifying the images by adopting a clustering algorithm, removing samples with poor image quality, and obtaining an intermediate sample set, wherein the method comprises the following specific steps as shown in fig. 2:
step 2.1, because the texture features and the color features help to detect the cloud and the shadow in the image, respectively for each image I in the primary sample setiAnd calculating the texture characteristic r by using a GLCM algorithm according to the following formulai G：
r(a,b|d,θ)＝{(a,b)|f(a,b)＝i,f(a+da,b+db)＝j；a,b＝0,1,2,...,N-1}；
Wherein d is the relative distance expressed by the number of pixels, theta is the angle of the calculation direction, (a, b) are the pixel coordinates of the image, and N is the number of gray levels of the image;
and extracting image I through a color histogramiColor characteristic r ofi H；
Step 2.2, fusing the texture features r in a direct splicing modei GAnd color characteristics ri HObtaining a fusion characteristic set r of the imagei；
Step 2.3, clustering the fusion feature set of the image through a DBSCAN (density clustering) algorithm, analyzing the class generated by clustering the primary sample set, deleting the samples containing cloud shielding and shadow shielding in the primary sample set, and reserving the samples containing the target ground objects, wherein the method specifically comprises the following steps:
step 2.3.1, initializing the class k to 0, setting the domain distance threshold epsilon between elements to 10, and calculating the distance between elements according to the following formula:
Meanwhile, the minimum contained point MinPts in the range taking the domain distance threshold epsilon as the neighborhood radius is set, the value of the minimum contained point MinPts is set to be 100, and if the number of elements contained in the neighborhood radius by one element is not less than the minimum contained point MinPts, the element riIs a core object;
step 2.3.2, traversing the fusion feature set r of the imageiIf the core object in the fusion feature set of the image is found, adding the core object into a core object set omega;
step 2.3.3, randomly selecting an element I which is not accessed in the core object set omega, firstly marking the element I as accessed, then marking the type of the element I as k, and finally adding the non-core object which is not accessed in the neighborhood radius into the seed set Seeds;
step 2.3.4, judging whether the seed set Seeds are empty, if so, finishing the generation of the cluster and entering step 2.3.6, otherwise, entering step 2.3.5;
2.3.5, randomly selecting a seed from the seed set Seeds, judging that the seed is a non-core object, adding an object within the radius of the seed field into the seed set if the seed is the non-core object, and skipping to the step 2.3.4;
and 2.3.6, judging whether all the elements in the core object set are accessed, if so, finishing clustering the fusion feature set of the image, otherwise, making k equal to k +1, and jumping to the step 2.3.3.
And 3, inputting the intermediate sample set into the semantic segmentation model in batches for iterative optimization training, predicting the samples after each iterative optimization is completed, and eliminating error samples in the intermediate sample set to obtain a target sample set.
In the specific implementation process, the semantic segmentation model is any one of an FCN network model, a SegNet network model, a hole convolution model, a deep lab series model, a reflonenet model and a PSPNet model. In this embodiment, a deplaybv 3+ model in a deplab series is selected as a semantic segmentation model M, the model M is used to purify the intermediate sample set, and samples with low labeling precision are removed, as shown in fig. 3, the specific steps are as follows:
step 3.1, training the selected semantic segmentation model by adopting an intermediate sample set, wherein the training step is as follows:
3.1.1, selecting a model, wherein the size of an input image is 1024 × 1024 × 3, the Focal local is used as a Loss function, Adam is selected as an optimizer, and related parameters are set, wherein the initial learning rate is set to be 0.001, the attenuation of the learning rate is set to be 0.9, the iteration times of the model is set to be 500, and the number of input images in each batch is set to be 64;
step 3.1.2, inputting samples in the middle sample set into the selected model M in batches, utilizing a cavity convolution structure to extract image characteristics through the following formula downsampling,
where, rate represents the void convolution expansion rate and w [ k ] represents the kth parameter of the convolution kernel. After down sampling, up sampling the characteristic graph to obtain an output result y';
step 3.1.3, calculating the error loss between the output result and the label by the following formula:
Lfl＝-(1-y′*y)γlog(y′)，
wherein y' is the model output result, y is the model input result, and γ is the hyper-parameter, set to 0.5.
Step 3.1.4, calculating the gradient of each parameter in the model
Parameters are optimized in a back propagation mode;
and 3.1.5, judging whether the model is converged, if so, stopping training, and obtaining the accuracy rate of the model, otherwise, skipping to the step 3.1.2.
Step 3.2, predicting the samples in the middle sample set by adopting the trained semantic segmentation model, and comparing the prediction result with the true value of the sample to obtain the coincidence rate of the prediction result and the true value of each sample;
calculating the coincidence rate of the prediction result and the sample truth value by the following formula:
pi＝f(M(Di),li),i∈[i,n]，
wherein p isiFor prediction accuracy, n is the number of samples in the intermediate sample set, M (D)i) To predict the result, liThe sample true value is shown, f is an evaluation index, PA (Pixel accuracy) is used as the evaluation index in the experiment, and the calculation formula of PA is as follows:
step 3.3, delete coincidence rate piSamples smaller than a preset coincidence rate threshold value alpha;
and 3.4, judging whether the accuracy of the model is not less than a preset accuracy threshold value precision, if so, finishing the manufacture of the target sample set, and otherwise, returning to the step 3.1.
In this example, the initial value of the overlap ratio threshold α is 25%, and the final value is 85%; the model accuracy threshold precision is 90%.
The invention can utilize the existing vector results and combine with the multi-stage remote sensing images to construct a large-scale semantic segmentation model training sample set. The invention screens the samples with poor image quality and inaccurate labels in the sample set by using a clustering algorithm and an iterative optimization method, improves the data quality of the sample set, and most of the operations are automatically completed by using a computer program, thereby avoiding manual one-by-one screening and effectively reducing the cost for manufacturing large-scale sample sets.
The technical solution provided by the present invention is described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (9)
1. A method for constructing a large-scale remote sensing image semantic segmentation model training sample set is characterized by comprising the following steps:
step 1, registering existing remote sensing image vector data with a multi-stage remote sensing image, and automatically cutting and extracting a primary sample set through a sliding window algorithm;
in the extraction process of the primary sample set, when the distribution of the pattern spots in the remote sensing image vector data after registration is sparse, the steps are as follows:
a1, acquiring the four-to-range of each image spot, and expanding a buffer area with random size to the periphery on the basis of the four-to-range;
step A2, rasterizing the image spot range after the buffer area is expanded to obtain a mask of a target ground object in the image spot range after the buffer area is expanded;
step A3, sliding a sliding window with fixed size on the mask, and calculating the ratio between the foreground and the background;
a4, when the ratio is larger than the set threshold, cutting the data of the area on the image and the mask as the image and the label;
step A5, integrating the cut data and the labels to obtain a primary sample set;
when the distribution of the pattern spots in the remote sensing image vector data after registration is dense, the method comprises the following steps:
b1, rasterizing the vector data of the remote sensing image to obtain a mask covering the whole remote sensing image;
step B2, using a sliding window with fixed size to slide on the remote sensing image and the mask simultaneously to extract data as an image and a label;
b3, integrating the cut data and the label to obtain a primary sample set;
step 2, extracting the characteristics of each image in the primary sample set, classifying by adopting a clustering algorithm, and removing samples containing cloud and mist occlusion and shadow occlusion in the primary sample set to obtain an intermediate sample set;
and 3, inputting the intermediate sample set into the semantic segmentation model in batches for iterative optimization training, predicting the samples after each iterative optimization is completed, and eliminating error samples in the intermediate sample set to obtain a target sample set.
2. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 1, characterized in that: the range size of the buffer area is 1-2 times of that of the sliding window.
3. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 1, characterized in that: the step 2 of obtaining the intermediate sample set comprises the following steps:
2.1, respectively extracting texture features and color features of each image in the primary sample set;
2.2, fusing the texture features and the color features in a direct splicing mode to obtain a fusion feature set of the image;
and 2.3, clustering the fusion feature set of the image through a clustering algorithm, analyzing the category generated by clustering the primary sample set, deleting the samples containing cloud and shadow occlusion in the primary sample set, and reserving the samples containing the target ground objects.
4. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 3, characterized in that: in step 2.1, GLCM is adopted to extract the texture features, and a color histogram is adopted to extract the color features.
5. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 3 or 4, characterized in that: the clustering algorithm in the step 2.3 adopts a density clustering algorithm, and the specific steps are as follows:
step 2.3.1, setting a domain distance threshold value among elements, setting a minimum contained point number in a range taking the domain distance threshold value as a neighborhood radius, and if the number of elements contained in the neighborhood radius of one element is not less than the minimum contained point number, taking the element as a core object;
step 2.3.2, finding out a core object in the fusion feature set of the image, and adding the core object into the core object set;
step 2.3.3, randomly selecting an element which is not accessed in the core object set, firstly marking the element as accessed, then marking the type of the element, and finally adding the non-core object which is not accessed in the neighborhood radius into the seed set;
step 2.3.4, judging whether the seed set is empty, if so, finishing the generation of the cluster and entering step 2.3.6, otherwise, entering step 2.3.5;
2.3.5, randomly selecting a seed from the seed set, judging that the seed is a non-core object, if so, adding an object in the radius of the seed field into the seed set, and skipping to the step 2.3.4;
and 2.3.6, judging whether all the elements in the core object set are accessed, if so, finishing clustering the fusion feature set of the images, otherwise, skipping to the step 2.3.3.
6. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 1, characterized in that: the semantic segmentation model is any one of an FCN network model, a SegNet network model, a cavity convolution model, a deep Lab series model, a RefineNet model and a PSPNet model.
7. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 1 or 6, characterized in that: the step of obtaining the target sample set in step 3 is as follows:
step 3.1, training the selected semantic segmentation model by adopting an intermediate sample set;
step 3.2, predicting the samples in the middle sample set by adopting the trained semantic segmentation model, and comparing the prediction result with the true value of the samples to obtain the prediction accuracy of each sample;
3.3, deleting samples with prediction accuracy rates smaller than a preset coincidence rate threshold value;
and 3.4, judging whether the accuracy of the model is not less than a preset accuracy threshold, if so, finishing the manufacture of the target sample set, and otherwise, returning to the step 3.1.
8. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 7, characterized in that: the training step of the semantic segmentation model in step 3.1 is as follows:
step 3.1.1, selecting a model, a loss function and an optimizer, and setting related parameters;
step 3.1.2, inputting samples in the intermediate sample set into the selected model in batches, extracting image features through downsampling by utilizing a cavity convolution structure, and upsampling the feature map to obtain an output result;
step 3.1.3, calculating the error between the output result and the label;
step 3.1.4, calculating the gradient of each parameter in the model, and optimizing the parameters in a back propagation mode;
and 3.1.5, judging whether the model is converged, if so, stopping training, and obtaining the accuracy rate of the model, otherwise, skipping to the step 3.1.2.
9. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 8, characterized in that: the calculation formula of the error between the output result and the label in step 3.1.3 is:
Lfl＝-(1-y′*y)γlog(y′)，
wherein y' is a model output result, y is a model input result, and gamma is a hyper-parameter.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110140509.6A CN112884791B (en) | 2021-02-02 | 2021-02-02 | Method for constructing large-scale remote sensing image semantic segmentation model training sample set |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110140509.6A CN112884791B (en) | 2021-02-02 | 2021-02-02 | Method for constructing large-scale remote sensing image semantic segmentation model training sample set |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112884791A CN112884791A (en) | 2021-06-01 |
| CN112884791B true CN112884791B (en) | 2021-11-26 |
Family
ID=76052551
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110140509.6A Active CN112884791B (en) | 2021-02-02 | 2021-02-02 | Method for constructing large-scale remote sensing image semantic segmentation model training sample set |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112884791B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113822900B (en) * | 2021-07-09 | 2023-09-22 | 武汉大学 | Method and system for automatically selecting new image sample based on vector constraint object-oriented |
| CN116051683B (en) * | 2022-12-20 | 2023-07-04 | 中国科学院空天信息创新研究院 | Remote sensing image generation method, storage medium and device based on style self-organization |
| CN115984559B (en) * | 2022-12-27 | 2024-01-12 | 二十一世纪空间技术应用股份有限公司 | Intelligent sample selection method and related device |
| CN116486077B (en) * | 2023-04-04 | 2024-04-30 | 中国科学院地理科学与资源研究所 | Remote sensing image semantic segmentation model sample set generation method and device |
| CN117237648B (en) * | 2023-11-16 | 2024-02-23 | 中国农业科学院农业资源与农业区划研究所 | Training method, device and equipment of semantic segmentation model based on context awareness |
| CN117349462B (en) * | 2023-12-06 | 2024-03-12 | 自然资源陕西省卫星应用技术中心 | Remote sensing intelligent interpretation sample data set generation method |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103258214B (en) * | 2013-04-26 | 2016-03-23 | 南京信息工程大学 | Based on the Classifying Method in Remote Sensing Image of image block Active Learning |
| CN108921025A (en) * | 2018-06-01 | 2018-11-30 | 苏州中科天启遥感科技有限公司 | A kind of object level classification samples automatic selecting method of collaborative variation detection |
| US11046030B2 (en) * | 2019-02-25 | 2021-06-29 | Roshdy George S. Barsoum | Rapid response fabrication of marine vessel platforms |
| CN110363798B (en) * | 2019-07-24 | 2022-02-18 | 宁波市测绘和遥感技术研究院 | Method for generating remote sensing image interpretation sample set |
| CN111144487B (en) * | 2019-12-27 | 2023-09-26 | 二十一世纪空间技术应用股份有限公司 | Method for establishing and updating remote sensing image sample library |
| CN111325116A (en) * | 2020-02-05 | 2020-06-23 | 武汉大学 | Remote sensing image target detection method capable of evolving based on offline training-online learning depth |
-
2021
- 2021-02-02 CN CN202110140509.6A patent/CN112884791B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN112884791A (en) | 2021-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112884791B (en) | Method for constructing large-scale remote sensing image semantic segmentation model training sample set | |
| CN112287807B (en) | Remote sensing image road extraction method based on multi-branch pyramid neural network | |
| CN111553303B (en) | Remote sensing orthographic image dense building extraction method based on convolutional neural network | |
| CN108596055B (en) | Airport target detection method of high-resolution remote sensing image under complex background | |
| CN111626947B (en) | Map vectorization sample enhancement method and system based on generation of countermeasure network | |
| CN113449594B (en) | Multilayer network combined remote sensing image ground semantic segmentation and area calculation method | |
| CN107067405B (en) | Remote sensing image segmentation method based on scale optimization | |
| CN106897681B (en) | Remote sensing image contrast analysis method and system | |
| CN111967313B (en) | Unmanned aerial vehicle image annotation method assisted by deep learning target detection algorithm | |
| CN112232371B (en) | American license plate recognition method based on YOLOv3 and text recognition | |
| CN113065594B (en) | Road network extraction method and device based on Beidou data and remote sensing image fusion | |
| CN112613350A (en) | High-resolution optical remote sensing image airplane target detection method based on deep neural network | |
| CN110909623B (en) | Three-dimensional target detection method and three-dimensional target detector | |
| CN110659601B (en) | Depth full convolution network remote sensing image dense vehicle detection method based on central point | |
| CN110176005B (en) | Remote sensing image segmentation method based on normalized index and multi-scale model | |
| CN112084871B (en) | High-resolution remote sensing target boundary extraction method based on weak supervised learning | |
| CN112686902A (en) | Two-stage calculation method for brain glioma identification and segmentation in nuclear magnetic resonance image | |
| CN111104850A (en) | Remote sensing image building automatic extraction method and system based on residual error network | |
| CN112651989A (en) | SEM image molecular sieve particle size statistical method and system based on Mask RCNN example segmentation | |
| CN116343157A (en) | Deep learning extraction method for road surface cracks | |
| CN117572457B (en) | Cross-scene multispectral point cloud classification method based on pseudo tag learning | |
| CN113033386A (en) | High-resolution remote sensing image-based transmission line channel hidden danger identification method and system | |
| Zeng et al. | Top-Down aircraft detection in large-scale scenes based on multi-source data and FEF-R-CNN | |
| CN116434054A (en) | Intensive remote sensing ground object extraction method based on line-plane combination | |
| CN115656952A (en) | Underground target multi-classification detection method based on U-shaped multi-dimensional multi-scale fusion neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CB03 | Change of inventor or designer information |
Inventor after: Ding Yi Inventor after: Fan Wenwu Inventor after: Wang Xiaopan Inventor after: Liu Jian Inventor after: Liu Xulei Inventor after: Zheng Zhong Inventor after: Chen Yang Inventor after: Wen Li Inventor after: Hu Yan Inventor after: Li Penglong Inventor after: Ma Zezhong Inventor after: Xiao He Inventor after: Zhang Zelie Inventor after: Wang Yalin Inventor after: Ao Ying Inventor before: Ding Yi Inventor before: Fan Wenwu Inventor before: Wang Xiaopan Inventor before: Liu Jian Inventor before: Liu Xulei Inventor before: Zheng Zhong Inventor before: Chen Yang Inventor before: Wen Li Inventor before: Hu Yan Inventor before: Li Penglong Inventor before: Ma Zezhong Inventor before: Xiao He Inventor before: Zhang Zelie Inventor before: Wang Yalin Inventor before: Ao Ying |
|
| CB03 | Change of inventor or designer information |