CN112884791A - Method for constructing large-scale remote sensing image semantic segmentation model training sample set - Google Patents

Method for constructing large-scale remote sensing image semantic segmentation model training sample set Download PDF

Info

Publication number
CN112884791A
CN112884791A CN202110140509.6A CN202110140509A CN112884791A CN 112884791 A CN112884791 A CN 112884791A CN 202110140509 A CN202110140509 A CN 202110140509A CN 112884791 A CN112884791 A CN 112884791A
Authority
CN
China
Prior art keywords
sample set
remote sensing
image
sensing image
semantic segmentation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110140509.6A
Other languages
Chinese (zh)
Other versions
CN112884791B (en
Inventor
丁忆
文力
胡艳
李朋龙
马泽忠
肖禾
张泽烈
王亚林
敖影
范文武
王小攀
刘建
刘旭蕾
郑中
陈阳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing Geographic Information And Remote Sensing Application Center
Original Assignee
Chongqing Geographic Information And Remote Sensing Application Center
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing Geographic Information And Remote Sensing Application Center filed Critical Chongqing Geographic Information And Remote Sensing Application Center
Priority to CN202110140509.6A priority Critical patent/CN112884791B/en
Publication of CN112884791A publication Critical patent/CN112884791A/en
Application granted granted Critical
Publication of CN112884791B publication Critical patent/CN112884791B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/40—Analysis of texture
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10032—Satellite or aerial image; Remote sensing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30204—Marker

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method for constructing a large-scale remote sensing image semantic segmentation model training sample set, which is characterized in that existing remote sensing image vector data and a multi-stage remote sensing image are registered, and a primary sample set is automatically cut and extracted through a sliding window algorithm according to the density of image spots; extracting the characteristics of each image in the primary sample set, classifying by adopting a clustering algorithm, and removing samples with poor image quality to obtain an intermediate sample set; and inputting the intermediate sample set into the semantic segmentation model in batches for iterative optimization training, predicting the samples after each iterative optimization is completed, and eliminating error samples in the intermediate sample set to obtain a target sample set. The remarkable effects are as follows: the generation of a whole image and a mask with a large occupied space can be avoided, the sliding times of a sliding window are reduced, and the extraction speed and the data quality of a sample are improved; the purity of the correct sample in the sample set is improved, and the cost for manufacturing a large-scale sample set is greatly reduced.

Description

Method for constructing large-scale remote sensing image semantic segmentation model training sample set
Technical Field
The invention relates to the technical field of remote sensing image feature extraction, remote sensing image semantic segmentation and sample set manufacturing, in particular to a method for constructing a large-scale remote sensing image semantic segmentation model training sample set.
Background
The high-resolution train satellite is successfully launched, PB-level remote sensing image data can be provided every year, meanwhile, a large amount of vector data are marked for ground objects by using the remote sensing images in projects such as 'national soil survey for the third time' and 'geographical national condition monitoring', if the data are effectively utilized to form a sample set with a uniform format and an ultra-large magnitude, a deep learning model with higher precision and higher generalization capability can be trained, and important support is provided for related projects and scientific research work. The currently mainstream methods for making a remote sensing image sample set mainly include the following two methods:
firstly, manually and directly carrying out visual interpretation and marking on a remote sensing image to manufacture a sample;
secondly, a sample set is manufactured by utilizing the existing manual interpretation vector result and the remote sensing image strictly corresponding to the manual interpretation vector result through an automatic cutting method.
Although the sample sets manufactured by the two methods have high precision, the sample set manufactured by the first method consumes a large amount of manpower and material resources, is extremely low in efficiency, and is difficult to construct a million-level sample set; in the second method, the vector and the remote sensing image must be strictly corresponding, i.e. the vector is obtained by sketching the image, but the strictly corresponding data is very limited, and the requirement of large-scale sample set production cannot be met, if the vector and other images are used for producing samples, the problems that the sample set obtained by automatic cutting has wrong samples caused by cloud coverage, shadow shielding and the like due to time phase change, image quality and other factors, and the sample label precision is not accurate due to incomplete matching of the vector and the image can be caused. Both methods cannot be used for large-scale sample set production, the first method causes the sample set production cost to be too high, and the second method causes the sample set quality to be difficult to guarantee.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide a method for constructing a large-scale remote sensing image semantic segmentation model training sample set, which can register the existing vector data with a multi-period remote sensing image, automatically cut and extract samples, improve the manufacturing efficiency of the sample set and the quality precision of the sample set by optimizing the sample cutting and extracting process, introducing the methods of automatic detection and elimination of wrong samples and iterative optimization and purification of the sample set, and can be applied to the fields of remote sensing image semantic segmentation, remote sensing image data set manufacturing and the like.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
a method for constructing a large-scale remote sensing image semantic segmentation model training sample set is characterized by comprising the following steps:
step 1, registering existing remote sensing image vector data with a multi-stage remote sensing image, and automatically cutting and extracting a primary sample set through a sliding window algorithm;
in the extraction process of the primary sample set, when the distribution of the pattern spots in the remote sensing image vector data after registration is sparse, the steps are as follows:
a1, acquiring the four-to-range of each image spot, and expanding a buffer area with random size to the periphery on the basis of the four-to-range;
step A2, rasterizing the image spot range after the buffer area is expanded to obtain a mask of a target ground object in the area;
step A3, sliding a sliding window with fixed size on the mask, and calculating the ratio between the foreground and the background;
a4, when the ratio is larger than the set threshold, cutting the data of the area on the image and the mask as the image and the label;
step A5, integrating the cut data and the labels to obtain a primary sample set;
when the distribution of the pattern spots in the remote sensing image vector data after registration is dense, the method comprises the following steps:
b1, rasterizing the vector data of the remote sensing image to obtain a mask covering the whole remote sensing image;
step B2, using a sliding window with fixed size to slide on the remote sensing image and the mask simultaneously to extract data as an image and a label;
b3, integrating the cut data and the label to obtain a primary sample set;
step 2, extracting the characteristics of each image in the primary sample set, classifying by adopting a clustering algorithm, and removing samples with poor image quality to obtain an intermediate sample set;
and 3, inputting the intermediate sample set into the semantic segmentation model in batches for iterative optimization training, predicting the samples after each iterative optimization is completed, and eliminating error samples in the intermediate sample set to obtain a target sample set.
Further, the range size of the buffer area is 1-2 times of that of the sliding window.
Further, the step of obtaining the intermediate sample set in step 2 is:
2.1, respectively extracting texture features and color features of each image in the primary sample set;
2.2, fusing the texture features and the color features in a direct splicing mode to obtain a fusion feature set of the image;
and 2.3, clustering the fusion feature set of the image through a clustering algorithm, analyzing the category generated by clustering the primary sample set, deleting the samples containing cloud and shadow occlusion in the primary sample set, and reserving the samples containing the target ground objects.
Further, in step 2.1, GLCM is used to extract the texture features, and a color histogram is used to extract the color features.
Further, the clustering algorithm in step 2.3 adopts a density clustering algorithm, which specifically comprises the following steps:
step 2.3.1, setting a domain distance threshold value among elements, setting a minimum contained point number in a range taking the domain distance threshold value as a neighborhood radius, and if the number of elements contained in the neighborhood radius of one element is not less than the minimum contained point number, taking the element as a core object;
step 2.3.2, finding out a core object in the fusion feature set of the image, and adding the core object into the core object set;
step 2.3.3, randomly selecting an element which is not accessed in the core object set, firstly marking the element as accessed, then marking the type of the element, and finally adding the non-core object which is not accessed in the neighborhood radius into the seed set;
step 2.3.4, judging whether the seed set is empty, if so, finishing the generation of the cluster and entering step 2.3.6, otherwise, entering step 2.3.5;
2.3.5, randomly selecting a seed from the seed set, judging that the seed is a non-core object, if so, adding an object in the radius of the seed field into the seed set, and skipping to the step 2.3.4;
and 2.3.6, judging whether all the elements in the core object set are accessed, if so, finishing clustering the fusion feature set of the images, otherwise, skipping to the step 2.3.3.
Further, the semantic segmentation model is any one of an FCN network model, a SegNet network model, a cavity convolution model, a deep Lab series model, a RefineNet model and a PSPNet model.
Further, the step of obtaining the target sample set in step 3 is as follows:
step 3.1, training the selected semantic segmentation model by adopting an intermediate sample set;
step 3.2, predicting the samples in the middle sample set by adopting the trained semantic segmentation model, and comparing the prediction result with the true value of the samples to obtain the prediction accuracy of each sample;
3.3, deleting samples with prediction accuracy rates smaller than a preset coincidence rate threshold value;
and 3.4, judging whether the accuracy of the model is not less than a preset accuracy threshold, if so, finishing the manufacture of the target sample set, and otherwise, returning to the step 3.1.
Further, the training step of the semantic segmentation model in step 3.1 is as follows:
step 3.1.1, selecting a model, a loss function and an optimizer, and setting related parameters;
step 3.1.2, inputting samples in the intermediate sample set into the selected model in batches, extracting image features through downsampling by utilizing a cavity convolution structure, and upsampling the feature map to obtain an output result;
step 3.1.3, calculating the error between the output result and the label;
step 3.1.4, calculating the gradient of each parameter in the model, and optimizing the parameters in a back propagation mode;
and 3.1.5, judging whether the model is converged, if so, stopping training, and obtaining the accuracy rate of the model, otherwise, skipping to the step 3.1.2.
Further, the calculation formula of the error between the output result in step 3.1.3 and the label is:
Lfl＝-(1-y′*y)γlog(y′)，
wherein y' is a model output result, y is a model input result, and gamma is a hyper-parameter.
Compared with the prior art, the invention has the following beneficial effects:
(1) the image cutting process is improved, only the area near the image spots is rasterized, when the image spots are sparsely distributed, the generation of the whole image and a mask with a large occupied space can be avoided, the sliding times of a sliding window are reduced, and the extraction speed of a sample is improved.
(2) The samples with poor image quality in the sample set are automatically screened out through a feature extraction algorithm and an automatic clustering algorithm, the samples with cloud shielding and shadow shielding are removed, and the data quality of the sample set is improved.
(3) The sample set is purified in an iterative optimization semantic segmentation model mode, wrong samples with inaccurate labels are automatically screened out, the purity of correct samples in the sample set is improved, and the labor cost for manufacturing the sample set is greatly reduced.
(4) The method can fully utilize the existing vector results, correspondingly extract data and labels from the vectors and the multiple remote sensing images, and ensure the quality of the sample set by a series of methods, thereby reducing the cost for manufacturing a large-scale sample set.
Drawings
FIG. 1 is a flow chart of a method of the present invention;
FIG. 2 is a schematic diagram of a sample for screening for poor image quality according to the present invention;
FIG. 3 is a schematic representation of the purification of a sample set using DeepLabV3+ according to the present invention.
Detailed Description
The following provides a more detailed description of the embodiments and the operation of the present invention with reference to the accompanying drawings.
As shown in fig. 1, a method for constructing a large-scale remote sensing image semantic segmentation model training sample set includes the following specific steps:
step 1, registering existing remote sensing image vector data with a multi-stage remote sensing image, and automatically cutting and extracting a primary sample set through a sliding window algorithm;
in this example, in order to avoid generating a mask with a large occupied space and a whole image when the distribution of the image patches is sparse, in the extraction process of the primary sample set, respective processing is performed according to the density of the image patches to reduce the sliding frequency of the sliding window and improve the extraction speed of the sample.
When the distribution of the pattern spots in the remote sensing image vector data after registration is sparse, the steps are as follows:
a1, obtaining the range of four to four of each image spot, and expanding a buffer area with random size to the periphery on the basis of the range of four to four, wherein the range of the buffer area is 1-2 times of that of the sliding window to ensure that the size of the mask after rasterization is larger than that of the sliding window, and the buffer area is used for ensuring that the size of the mask is larger than that of the sliding window and simultaneously retaining the context information around the image spot;
step A2, rasterizing the image spot range after the buffer area is expanded to obtain a mask of a target ground object in the area;
step A3, slide on mask with fixed size sliding window (1024 × 1024), and pass
Figure BDA0002928387390000051
Calculate the ratio between foreground and background, CountstargetNumber of foreground pixels, CountsbackgrounIs the background pixel number;
a4, when the ratio is larger than the set threshold 0, cutting the image and the data of the area on the mask as the image and the label;
step A5, integrating the cut data and the labels to obtain a primary sample set;
when the distribution of the pattern spots in the remote sensing image vector data after registration is dense, the method comprises the following steps:
b1, rasterizing the vector data of the remote sensing image to obtain a mask covering the whole remote sensing image;
step B2, using a sliding window (1024 multiplied by 1024) with fixed size to simultaneously slide and extract data on the remote sensing image and the mask as an image and a label;
b3, integrating the cut data and the label to obtain a primary sample set;
step 2, extracting features of each image in the primary sample set, classifying the images by adopting a clustering algorithm, removing samples with poor image quality, and obtaining an intermediate sample set, wherein the method comprises the following specific steps as shown in fig. 2:
step 2.1, because the texture features and the color features help to detect the cloud and the shadow in the image, respectively for each image I in the primary sample setiAnd calculating the texture characteristic r by using a GLCM algorithm according to the following formulai G：
r(a,b|d,θ)＝{(a,b)|f(a,b)＝i,f(a+da,b+db)＝j；a,b＝0,1,2,...,N-1}；
Wherein d is the relative distance expressed by the number of pixels, theta is the angle of the calculation direction, (a, b) are the pixel coordinates of the image, and N is the number of gray levels of the image;
and extracting image I through a color histogramiColor characteristic r ofi H；
Step 2.2, fusing the texture features r in a direct splicing modei GAnd color characteristics ri HObtaining a fusion characteristic set r of the imagei；
Step 2.3, clustering the fusion feature set of the image through a DBSCAN (density clustering) algorithm, analyzing the class generated by clustering the primary sample set, deleting the samples containing cloud shielding and shadow shielding in the primary sample set, and reserving the samples containing the target ground objects, wherein the method specifically comprises the following steps:
step 2.3.1, initializing the class k to 0, setting the domain distance threshold epsilon between elements to 10, and calculating the distance between elements according to the following formula:
Figure BDA0002928387390000065
wherein n is the number of elements.
Meanwhile, the minimum contained point MinPts in the range taking the domain distance threshold epsilon as the neighborhood radius is set, the value of the minimum contained point MinPts is set to be 100, and if the number of elements contained in the neighborhood radius by one element is not less than the minimum contained point MinPts, the element riIs a core object;
step 2.3.2, traversing the fusion feature set r of the imageiIf the core object in the fusion feature set of the image is found, adding the core object into a core object set omega;
step 2.3.3, randomly selecting an element I which is not accessed in the core object set omega, firstly marking the element I as accessed, then marking the type of the element I as k, and finally adding the non-core object which is not accessed in the neighborhood radius into the seed set Seeds;
step 2.3.4, judging whether the seed set Seeds are empty, if so, finishing the generation of the cluster and entering step 2.3.6, otherwise, entering step 2.3.5;
2.3.5, randomly selecting a seed from the seed set Seeds, judging that the seed is a non-core object, adding an object within the radius of the seed field into the seed set if the seed is the non-core object, and skipping to the step 2.3.4;
and 2.3.6, judging whether all the elements in the core object set are accessed, if so, finishing clustering the fusion feature set of the image, otherwise, making k equal to k +1, and jumping to the step 2.3.3.
And 3, inputting the intermediate sample set into the semantic segmentation model in batches for iterative optimization training, predicting the samples after each iterative optimization is completed, and eliminating error samples in the intermediate sample set to obtain a target sample set.
In the specific implementation process, the semantic segmentation model is any one of an FCN network model, a SegNet network model, a hole convolution model, a deep lab series model, a reflonenet model and a PSPNet model. In this embodiment, a deplaybv 3+ model in a deplab series is selected as a semantic segmentation model M, the model M is used to purify the intermediate sample set, and samples with low labeling precision are removed, as shown in fig. 3, the specific steps are as follows:
step 3.1, training the selected semantic segmentation model by adopting an intermediate sample set, wherein the training step is as follows:
3.1.1, selecting a model, wherein the size of an input image is 1024 × 1024 × 3, the Focal local is used as a Loss function, Adam is selected as an optimizer, and related parameters are set, wherein the initial learning rate is set to be 0.001, the attenuation of the learning rate is set to be 0.9, the iteration times of the model is set to be 500, and the number of input images in each batch is set to be 64;
step 3.1.2, inputting samples in the middle sample set into the selected model M in batches, utilizing a cavity convolution structure to extract image characteristics through the following formula downsampling,
Figure BDA0002928387390000071
where, rate represents the void convolution expansion rate and w [ k ] represents the kth parameter of the convolution kernel. After down sampling, up sampling the characteristic graph to obtain an output result y';
step 3.1.3, calculating the error loss between the output result and the label by the following formula:
Lfl＝-(1-y′*y)γlog(y′)，
wherein y' is the model output result, y is the model input result, and γ is the hyper-parameter, set to 0.5.
Step 3.1.4, calculating the gradient of each parameter in the model
Figure BDA0002928387390000081
Parameters are optimized in a back propagation mode;
and 3.1.5, judging whether the model is converged, if so, stopping training, and obtaining the accuracy rate of the model, otherwise, skipping to the step 3.1.2.
Step 3.2, predicting the samples in the middle sample set by adopting the trained semantic segmentation model, and comparing the prediction result with the true value of the sample to obtain the coincidence rate of the prediction result and the true value of each sample;
calculating the coincidence rate of the prediction result and the sample truth value by the following formula:
pi＝f(M(Di),li),i∈[i,n]，
wherein p isiFor prediction accuracy, n is the number of samples in the intermediate sample set, M (D)i) To predict the result, liThe sample true value is shown, f is an evaluation index, PA (Pixel accuracy) is used as the evaluation index in the experiment, and the calculation formula of PA is as follows:
Figure BDA0002928387390000082
step 3.3, delete coincidence rate piSamples smaller than a preset coincidence rate threshold value alpha;
and 3.4, judging whether the accuracy of the model is not less than a preset accuracy threshold value precision, if so, finishing the manufacture of the target sample set, and otherwise, returning to the step 3.1.
In this example, the initial value of the overlap ratio threshold α is 25%, and the final value is 85%; the model accuracy threshold precision is 90%.
The invention can utilize the existing vector results and combine with the multi-stage remote sensing images to construct a large-scale semantic segmentation model training sample set. The invention screens the samples with poor image quality and inaccurate labels in the sample set by using a clustering algorithm and an iterative optimization method, improves the data quality of the sample set, and most of the operations are automatically completed by using a computer program, thereby avoiding manual one-by-one screening and effectively reducing the cost for manufacturing large-scale sample sets.
The technical solution provided by the present invention is described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.

Claims (9)

1. A method for constructing a large-scale remote sensing image semantic segmentation model training sample set is characterized by comprising the following steps:
step 1, registering existing remote sensing image vector data with a multi-stage remote sensing image, and automatically cutting and extracting a primary sample set through a sliding window algorithm;
in the extraction process of the primary sample set, when the distribution of the pattern spots in the remote sensing image vector data after registration is sparse, the steps are as follows:
a1, acquiring the four-to-range of each image spot, and expanding a buffer area with random size to the periphery on the basis of the four-to-range;
step A2, rasterizing the image spot range after the buffer area is expanded to obtain a mask of a target ground object in the area;
step A3, sliding a sliding window with fixed size on the mask, and calculating the ratio between the foreground and the background;
a4, when the ratio is larger than the set threshold, cutting the data of the area on the image and the mask as the image and the label;
step A5, integrating the cut data and the labels to obtain a primary sample set;
when the distribution of the pattern spots in the remote sensing image vector data after registration is dense, the method comprises the following steps:
b1, rasterizing the vector data of the remote sensing image to obtain a mask covering the whole remote sensing image;
step B2, using a sliding window with fixed size to slide on the remote sensing image and the mask simultaneously to extract data as an image and a label;
b3, integrating the cut data and the label to obtain a primary sample set;
step 2, extracting the characteristics of each image in the primary sample set, classifying by adopting a clustering algorithm, and removing samples with poor image quality to obtain an intermediate sample set;
and 3, inputting the intermediate sample set into the semantic segmentation model in batches for iterative optimization training, predicting the samples after each iterative optimization is completed, and eliminating error samples in the intermediate sample set to obtain a target sample set.
2. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 1, characterized in that: the range size of the buffer area is 1-2 times of that of the sliding window.
3. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 1, characterized in that: the step 2 of obtaining the intermediate sample set comprises the following steps:
2.1, respectively extracting texture features and color features of each image in the primary sample set;
2.2, fusing the texture features and the color features in a direct splicing mode to obtain a fusion feature set of the image;
and 2.3, clustering the fusion feature set of the image through a clustering algorithm, analyzing the category generated by clustering the primary sample set, deleting the samples containing cloud and shadow occlusion in the primary sample set, and reserving the samples containing the target ground objects.
4. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 3, characterized in that: in step 2.1, GLCM is adopted to extract the texture features, and a color histogram is adopted to extract the color features.
5. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 3 or 4, characterized in that: the clustering algorithm in the step 2.3 adopts a density clustering algorithm, and the specific steps are as follows:
step 2.3.1, setting a domain distance threshold value among elements, setting a minimum contained point number in a range taking the domain distance threshold value as a neighborhood radius, and if the number of elements contained in the neighborhood radius of one element is not less than the minimum contained point number, taking the element as a core object;
step 2.3.2, finding out a core object in the fusion feature set of the image, and adding the core object into the core object set;
step 2.3.3, randomly selecting an element which is not accessed in the core object set, firstly marking the element as accessed, then marking the type of the element, and finally adding the non-core object which is not accessed in the neighborhood radius into the seed set;
step 2.3.4, judging whether the seed set is empty, if so, finishing the generation of the cluster and entering step 2.3.6, otherwise, entering step 2.3.5;
2.3.5, randomly selecting a seed from the seed set, judging that the seed is a non-core object, if so, adding an object in the radius of the seed field into the seed set, and skipping to the step 2.3.4;
and 2.3.6, judging whether all the elements in the core object set are accessed, if so, finishing clustering the fusion feature set of the images, otherwise, skipping to the step 2.3.3.
6. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 1, characterized in that: the semantic segmentation model is any one of an FCN network model, a SegNet network model, a cavity convolution model, a deep Lab series model, a RefineNet model and a PSPNet model.
7. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 1 or 6, characterized in that: the step of obtaining the target sample set in step 3 is as follows:
step 3.1, training the selected semantic segmentation model by adopting an intermediate sample set;
step 3.2, predicting the samples in the middle sample set by adopting the trained semantic segmentation model, and comparing the prediction result with the true value of the samples to obtain the prediction accuracy of each sample;
3.3, deleting samples with prediction accuracy rates smaller than a preset coincidence rate threshold value;
and 3.4, judging whether the accuracy of the model is not less than a preset accuracy threshold, if so, finishing the manufacture of the target sample set, and otherwise, returning to the step 3.1.
8. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 7, characterized in that: the training step of the semantic segmentation model in step 3.1 is as follows:
step 3.1.1, selecting a model, a loss function and an optimizer, and setting related parameters;
step 3.1.2, inputting samples in the intermediate sample set into the selected model in batches, extracting image features through downsampling by utilizing a cavity convolution structure, and upsampling the feature map to obtain an output result;
step 3.1.3, calculating the error between the output result and the label;
step 3.1.4, calculating the gradient of each parameter in the model, and optimizing the parameters in a back propagation mode;
and 3.1.5, judging whether the model is converged, if so, stopping training, and obtaining the accuracy rate of the model, otherwise, skipping to the step 3.1.2.
9. The method for constructing the large-scale remote sensing image semantic segmentation model training sample set according to claim 8, characterized in that: the calculation formula of the error between the output result and the label in step 3.1.3 is:
Lfl＝-(1-y′*y)γlog(y′)，
wherein y' is a model output result, y is a model input result, and gamma is a hyper-parameter.
CN202110140509.6A 2021-02-02 2021-02-02 Method for constructing large-scale remote sensing image semantic segmentation model training sample set Active CN112884791B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110140509.6A CN112884791B (en) 2021-02-02 2021-02-02 Method for constructing large-scale remote sensing image semantic segmentation model training sample set

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110140509.6A CN112884791B (en) 2021-02-02 2021-02-02 Method for constructing large-scale remote sensing image semantic segmentation model training sample set

Publications (2)

Publication Number Publication Date
CN112884791A true CN112884791A (en) 2021-06-01
CN112884791B CN112884791B (en) 2021-11-26

Family

ID=76052551

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110140509.6A Active CN112884791B (en) 2021-02-02 2021-02-02 Method for constructing large-scale remote sensing image semantic segmentation model training sample set

Country Status (1)

Country Link
CN (1) CN112884791B (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113822900A (en) * 2021-07-09 2021-12-21 武汉大学 Vector constraint object-oriented new image sample automatic selection method and system
CN115984559A (en) * 2022-12-27 2023-04-18 二十一世纪空间技术应用股份有限公司 Intelligent sample selection method and related device
CN116051683A (en) * 2022-12-20 2023-05-02 中国科学院空天信息创新研究院 Remote sensing image generation method, storage medium and device based on style self-organization
CN116486077A (en) * 2023-04-04 2023-07-25 中国科学院地理科学与资源研究所 Remote sensing image semantic segmentation model sample set generation method and device
CN117237648A (en) * 2023-11-16 2023-12-15 中国农业科学院农业资源与农业区划研究所 Training method, device and equipment of semantic segmentation model based on context awareness
CN117349462A (en) * 2023-12-06 2024-01-05 自然资源陕西省卫星应用技术中心 Remote sensing intelligent interpretation sample data set generation method
CN118314028A (en) * 2024-04-07 2024-07-09 自然资源部国土卫星遥感应用中心 Road sample data manufacturing method based on vector image spots
CN118470092A (en) * 2024-06-04 2024-08-09 航天宏图信息技术股份有限公司 Crop planting area extraction method, device, equipment and medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103258214A (en) * 2013-04-26 2013-08-21 南京信息工程大学 Remote sensing image classification method based on image block active learning
CN108921025A (en) * 2018-06-01 2018-11-30 苏州中科天启遥感科技有限公司 A kind of object level classification samples automatic selecting method of collaborative variation detection
CN110363798A (en) * 2019-07-24 2019-10-22 宁波市测绘设计研究院 A kind of generation method of remote sensing image interpretation sample set
CN111144487A (en) * 2019-12-27 2020-05-12 二十一世纪空间技术应用股份有限公司 Method for establishing and updating remote sensing image sample library
CN111325116A (en) * 2020-02-05 2020-06-23 武汉大学 Remote sensing image target detection method capable of evolving based on offline training-online learning depth
US20200269530A1 (en) * 2019-02-25 2020-08-27 Roshdy George S. Barsoum Rapid response fabrication of marine vessel platforms

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103258214A (en) * 2013-04-26 2013-08-21 南京信息工程大学 Remote sensing image classification method based on image block active learning
CN108921025A (en) * 2018-06-01 2018-11-30 苏州中科天启遥感科技有限公司 A kind of object level classification samples automatic selecting method of collaborative variation detection
US20200269530A1 (en) * 2019-02-25 2020-08-27 Roshdy George S. Barsoum Rapid response fabrication of marine vessel platforms
CN110363798A (en) * 2019-07-24 2019-10-22 宁波市测绘设计研究院 A kind of generation method of remote sensing image interpretation sample set
CN111144487A (en) * 2019-12-27 2020-05-12 二十一世纪空间技术应用股份有限公司 Method for establishing and updating remote sensing image sample library
CN111325116A (en) * 2020-02-05 2020-06-23 武汉大学 Remote sensing image target detection method capable of evolving based on offline training-online learning depth

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
WEIXUN ZHOU 等: "PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval", 《ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING》 *
戚尚育: "高分辫率遥感图像样本提取和样本库的研究", 《万方数据知识服务平台》 *
黄亚博 等: "多源数据的土地覆被样本自动提取", 《遥感学报》 *

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113822900B (en) * 2021-07-09 2023-09-22 武汉大学 Method and system for automatically selecting new image sample based on vector constraint object-oriented
CN113822900A (en) * 2021-07-09 2021-12-21 武汉大学 Vector constraint object-oriented new image sample automatic selection method and system
CN116051683A (en) * 2022-12-20 2023-05-02 中国科学院空天信息创新研究院 Remote sensing image generation method, storage medium and device based on style self-organization
CN116051683B (en) * 2022-12-20 2023-07-04 中国科学院空天信息创新研究院 Remote sensing image generation method, storage medium and device based on style self-organization
CN115984559B (en) * 2022-12-27 2024-01-12 二十一世纪空间技术应用股份有限公司 Intelligent sample selection method and related device
CN115984559A (en) * 2022-12-27 2023-04-18 二十一世纪空间技术应用股份有限公司 Intelligent sample selection method and related device
CN116486077A (en) * 2023-04-04 2023-07-25 中国科学院地理科学与资源研究所 Remote sensing image semantic segmentation model sample set generation method and device
CN116486077B (en) * 2023-04-04 2024-04-30 中国科学院地理科学与资源研究所 Remote sensing image semantic segmentation model sample set generation method and device
CN117237648A (en) * 2023-11-16 2023-12-15 中国农业科学院农业资源与农业区划研究所 Training method, device and equipment of semantic segmentation model based on context awareness
CN117237648B (en) * 2023-11-16 2024-02-23 中国农业科学院农业资源与农业区划研究所 Training method, device and equipment of semantic segmentation model based on context awareness
CN117349462B (en) * 2023-12-06 2024-03-12 自然资源陕西省卫星应用技术中心 Remote sensing intelligent interpretation sample data set generation method
CN117349462A (en) * 2023-12-06 2024-01-05 自然资源陕西省卫星应用技术中心 Remote sensing intelligent interpretation sample data set generation method
CN118314028A (en) * 2024-04-07 2024-07-09 自然资源部国土卫星遥感应用中心 Road sample data manufacturing method based on vector image spots
CN118470092A (en) * 2024-06-04 2024-08-09 航天宏图信息技术股份有限公司 Crop planting area extraction method, device, equipment and medium

Also Published As

Publication number Publication date
CN112884791B (en) 2021-11-26

Similar Documents

Publication Publication Date Title
CN112884791B (en) Method for constructing large-scale remote sensing image semantic segmentation model training sample set
CN111986099B (en) Tillage monitoring method and system based on convolutional neural network with residual error correction fused
CN112287807B (en) Remote sensing image road extraction method based on multi-branch pyramid neural network
CN111553303B (en) Remote sensing orthographic image dense building extraction method based on convolutional neural network
CN113449594B (en) Multilayer network combined remote sensing image ground semantic segmentation and area calculation method
CN108596055B (en) Airport target detection method of high-resolution remote sensing image under complex background
CN105139395B (en) SAR image segmentation method based on small echo pond convolutional neural networks
CN111626947B (en) Map vectorization sample enhancement method and system based on generation of countermeasure network
CN112232371B (en) American license plate recognition method based on YOLOv3 and text recognition
CN111967313B (en) Unmanned aerial vehicle image annotation method assisted by deep learning target detection algorithm
CN112613350A (en) High-resolution optical remote sensing image airplane target detection method based on deep neural network
CN110176005B (en) Remote sensing image segmentation method based on normalized index and multi-scale model
CN112084871B (en) High-resolution remote sensing target boundary extraction method based on weak supervised learning
CN114898327B (en) Vehicle detection method based on lightweight deep learning network
CN114092794B (en) Sea ice image classification method, system, medium, equipment and processing terminal
CN110659601A (en) Depth full convolution network remote sensing image dense vehicle detection method based on central point
CN112686902A (en) Two-stage calculation method for brain glioma identification and segmentation in nuclear magnetic resonance image
CN117572457B (en) Cross-scene multispectral point cloud classification method based on pseudo tag learning
CN110909623A (en) Three-dimensional target detection method and three-dimensional target detector
CN111104850A (en) Remote sensing image building automatic extraction method and system based on residual error network
CN117541930A (en) River basin scale high space-time resolution river network remote sensing information extraction method and system
CN115423802A (en) Automatic classification and segmentation method for squamous epithelial tumor cell picture based on deep learning
CN113077438B (en) Cell nucleus region extraction method and imaging method for multi-cell nucleus color image
Zeng et al. Top-Down aircraft detection in large-scale scenes based on multi-source data and FEF-R-CNN
CN115620169B (en) Building main angle correction method based on regional consistency

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CB03 Change of inventor or designer information

Inventor after: Ding Yi

Inventor after: Fan Wenwu

Inventor after: Wang Xiaopan

Inventor after: Liu Jian

Inventor after: Liu Xulei

Inventor after: Zheng Zhong

Inventor after: Chen Yang

Inventor after: Wen Li

Inventor after: Hu Yan

Inventor after: Li Penglong

Inventor after: Ma Zezhong

Inventor after: Xiao He

Inventor after: Zhang Zelie

Inventor after: Wang Yalin

Inventor after: Ao Ying

Inventor before: Ding Yi

Inventor before: Fan Wenwu

Inventor before: Wang Xiaopan

Inventor before: Liu Jian

Inventor before: Liu Xulei

Inventor before: Zheng Zhong

Inventor before: Chen Yang

Inventor before: Wen Li

Inventor before: Hu Yan

Inventor before: Li Penglong

Inventor before: Ma Zezhong

Inventor before: Xiao He

Inventor before: Zhang Zelie

Inventor before: Wang Yalin

Inventor before: Ao Ying

CB03 Change of inventor or designer information