CN114445726A - Sample library establishing method and device based on deep learning - Google Patents
Sample library establishing method and device based on deep learning Download PDFInfo
- Publication number
- CN114445726A CN114445726A CN202111521298.7A CN202111521298A CN114445726A CN 114445726 A CN114445726 A CN 114445726A CN 202111521298 A CN202111521298 A CN 202111521298A CN 114445726 A CN114445726 A CN 114445726A
- Authority
- CN
- China
- Prior art keywords
- samples
- sample
- sample library
- library
- pixel precision
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the invention provides a sample library establishing method and device based on deep learning, wherein the method comprises the following steps: training a cloud sample to obtain a cloud detection model, and screening and processing the sample to form a high-quality sample library; augmenting the sample; training a sample to obtain a first interpretation model; interpreting samples in a sample library, calculating pixel precision, taking the samples meeting the requirements as a new sample library, amplifying the samples, and adding the samples obtained by amplification into the new sample library; iteratively optimizing the first interpretation model to obtain a second interpretation model; and interpreting the samples which are not preprocessed, calculating the pixel precision, and amplifying the samples meeting the requirements to obtain a natural resource sample library. Compared with the prior art, the method effectively retains the geometric information of the image through the HRNet network, realizes multi-scale feature integration, and enhances the context feature extraction capability; automatic augmentation of samples can be simultaneously realized in the process of establishing a sample library, and the precision of the model and the quality of the samples are improved through deep learning.
Description
Technical Field
The invention relates to the field of natural resource interpretation, in particular to a sample library establishing method and device based on deep learning.
Background
At present, deep learning is widely applied to the field of natural resource interpretation. The remote sensing image is used as an important data source for natural resource investigation and evaluation, and the remote sensing image interpretation based on deep learning becomes an advanced technical means of a natural resource remote sensing monitoring service system. However, the existing remote sensing intelligent detection technology still has the problems of low accuracy and the like, and cannot meet the quality requirements of reality, accuracy, reliability and the like.
The remote sensing intelligent interpretation based on deep learning depends on the quantity and quality of samples, and on one hand, the existing natural resource intelligent interpretation samples are mostly obtained by manual drawing or existing data such as the three tone data of the state and the soil, the geographical national situation data and the like. And although the manual drawing of the sample meeting the deep learning training requirement is time-consuming and labor-consuming, and the timeliness is difficult to meet the intelligent interpretation requirement. Although the existing data set meets the actual production requirement, the deep learning intelligence is interpreted to ensure the model precision, the requirement on the quality of the sample is higher, the data acquisition results completed by different operators are inconsistent, the quality of the sample cannot be ensured and is difficult to unify. On the other hand, the sample set prepared according to the earth surface coverage classification can cause the uneven distribution of the quality and quantity of the samples due to the difference of images of different sensors, and the quality of deep learning intelligent interpretation is influenced.
The prior art can not fully utilize the existing data, timely obtains a sample library with guaranteed quality and quantity, lacks a technical means for expanding the sample library, and enriches different expression forms of samples.
Disclosure of Invention
The invention provides a sample library establishing method and device based on deep learning, and aims to solve the technical problem that automatic augmentation cannot be simultaneously performed in the sample library establishing process.
In order to solve the above technical problem, an embodiment of the present invention provides a sample library establishing method based on deep learning, including:
constructing an HRNet network, training a plurality of cloud samples to obtain a cloud detection model, preprocessing the samples with the cloud content smaller than a first preset value in an initial sample library through the cloud detection model, and using the preprocessed samples as a high-quality sample library;
performing augmentation treatment on all samples in the high-quality sample library, and adding the samples obtained through the augmentation treatment into the high-quality sample library;
training all samples of the high-quality sample library through the HRNet network to obtain a first interpretation model;
interpreting all samples in the high-quality sample library by using the first interpretation model, respectively calculating the pixel precision of each sample, bringing the samples meeting a first preset pixel precision requirement into a new sample library, carrying out augmentation processing on the samples meeting the first preset pixel precision requirement, and bringing the samples obtained through the augmentation processing into the new sample library;
performing iterative optimization on the first interpretation model by using the new sample library until convergence to obtain a second interpretation model;
and utilizing the second interpretation model to interpret the samples which are not preprocessed and have the cloud content smaller than the first preset value in the initial sample library, respectively calculating the pixel precision, adding a new sample library into the samples meeting the second preset pixel precision requirement, performing amplification treatment, and adding the samples obtained through the amplification treatment into the new sample library to form a natural resource sample library.
As a preferred scheme, the performing an amplification process on all samples in the high-quality sample library specifically includes:
performing augmentation processing on all samples in the high-quality sample library by using a least square generation type countermeasure network model;
the method comprises the following steps of performing augmentation treatment on a sample meeting a first preset pixel precision requirement, specifically:
carrying out augmentation treatment on the sample meeting the first preset pixel precision requirement by utilizing the least square generation type confrontation network model;
the method for adding the new sample library to the samples meeting the second preset pixel precision requirement and performing augmentation processing specifically comprises the following steps:
and adding a new sample base to the samples meeting the second preset pixel precision requirement, and performing augmentation processing by using the least square generation type countermeasure network model.
Preferably, the loss function design of the least square generation type countermeasure network model aims at Kagan divergence.
Preferably, the method of the augmentation process further includes: and performing Gamma stretching, Gaussian blurring, scaling, rotation and overturning on the sample.
As a preferred scheme, the preprocessing of the sample with the cloud content smaller than the first preset value in the initial sample library and the use of the sample as the high-quality sample library specifically comprise: screening all samples with the cloud content smaller than a first preset value in the initial sample library, and selecting not less than 10000 samples as a high-quality sample library for each scene from the screened samples.
Correspondingly, the embodiment of the invention also provides a sample library establishing device based on deep learning, which comprises the following steps: the system comprises a construction module, an augmentation module, a training module, a new sample module, an iterative optimization module and a natural resource sample module;
the building module is used for building an HRNet network, training a plurality of cloud samples to obtain a cloud detection model, and preprocessing the samples with the cloud content smaller than a first preset value in an initial sample library through the cloud detection model to be used as a high-quality sample library;
the amplification module is used for performing amplification treatment on all samples in the high-quality sample library and adding the samples obtained through the amplification treatment into the high-quality sample library;
the training module is used for training all samples of the high-quality sample library through the HRNet network to obtain a first interpretation model;
the new sample module is used for interpreting all samples in the high-quality sample library by using the first interpretation model, respectively calculating the pixel precision of each sample, bringing the samples meeting a first preset pixel precision requirement into the new sample library, carrying out augmentation processing on the samples meeting the first preset pixel precision requirement, and bringing the samples obtained by the augmentation processing into the new sample library;
the iterative optimization module is used for carrying out iterative optimization on the first interpretation model by utilizing the new sample library until convergence to obtain a second interpretation model;
the natural resource sample module is used for utilizing the second interpretation model to interpret samples which are not preprocessed and have the cloud content smaller than a first preset value in the initial sample library, respectively calculating pixel precision, adding a new sample library into the samples meeting the requirement of second preset pixel precision, carrying out amplification processing, adding the samples obtained through the amplification processing into the new sample library, and forming a natural resource sample library.
Preferably, the augmentation module performs augmentation processing on all samples in the high-quality sample library, specifically:
the augmentation module performs augmentation processing on all samples in the high-quality sample library by using a least square generation type confrontation network model;
the new sample module is used for carrying out augmentation processing on a sample meeting a first preset pixel precision requirement, and specifically comprises the following steps:
the new sample module is used for carrying out augmentation treatment on the sample meeting the first preset pixel precision requirement by using the least square generation type confrontation network model and bringing the sample into a new sample library;
the natural resource sample module adds a new sample library to a sample meeting a second preset pixel precision requirement and performs augmentation processing, and specifically comprises the following steps:
and the natural resource sample module adds a new sample base to the sample meeting the second preset pixel precision requirement and performs augmentation processing by using the least square generation type countermeasure network model.
Preferably, the loss function design of the least square generation type countermeasure network model aims at Kagan divergence.
Preferably, the method of the augmentation process further includes: and performing Gamma stretching, Gaussian blurring, scaling, rotation and overturning on the sample.
As a preferred scheme, the construction module preprocesses a sample with a cloud content smaller than a first preset value in an initial sample library and uses the sample as a high-quality sample library, specifically: the construction module screens out all samples with the cloud content smaller than a first preset value in an initial sample library, selects not less than 10000 samples for each scene from the screened samples, and takes the samples as a high-quality sample library after manual modification until the samples meet preset requirements.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
the embodiment of the invention provides a sample library establishing method and device based on deep learning, wherein the method comprises the following steps: training a plurality of cloud samples to obtain a cloud detection model, preprocessing all samples with the cloud content smaller than a first preset value in an initial sample library through the cloud detection model, and taking the preprocessed samples as a high-quality sample library; carrying out augmentation treatment on the samples in the high-quality sample library; training the sample through an HRNet network to obtain a first interpretation model; interpreting all samples in the high-quality sample library by using the first interpretation model, respectively calculating the pixel precision of each sample, carrying out augmentation processing on the samples meeting the preset pixel precision requirement, bringing the samples into a new sample library, and bringing the samples obtained through the augmentation processing into the new sample library; performing iterative optimization on the first interpretation model by using the new sample library until convergence to obtain a second interpretation model; and by utilizing the second interpretation model, the samples which are not preprocessed and have the cloud content smaller than the first preset value in the initial sample library are interpreted, the pixel precision is respectively calculated, the samples meeting the preset pixel precision requirement are subjected to amplification processing, and a natural resource sample library is obtained. Compared with the prior art, the method effectively retains the geometric information of the image through the HRNet network, realizes multi-scale feature integration, and obviously enhances the context feature extraction capability of the model; in the process of establishing the natural resource sample library, automatic augmentation operation on samples can be simultaneously realized, manual addition of new samples or manual modification on the samples are not needed, the precision of the interpretation model is improved through a deep learning technology, the sample quality is improved, the requirement of intelligent interpretation is met, the timeliness is improved, and the purpose of saving manpower and material resources is achieved.
Drawings
FIG. 1: the invention provides a flow chart diagram of an embodiment of a sample library establishing method.
FIG. 2: the HRNet network structure diagram adopted by the embodiment of the sample library establishing method provided by the invention is provided.
FIG. 3: the schematic diagram of the network convolution operation of the HRNet adopted in the embodiment of the sample library establishment method provided by the invention is shown.
FIG. 4: the principle schematic diagram of network feature integration of the HRNet is adopted for one embodiment of the sample library establishment method provided by the invention.
FIG. 5: the present invention provides a schematic structural diagram of an example of a generative confrontation network model for an embodiment of a sample library establishing method.
FIG. 6: the invention provides a structural schematic diagram of an example of a condition generating type confrontation network model in an embodiment of a sample library establishing method.
FIG. 7: the invention provides a structural schematic diagram of a conditional least square generation type confrontation network model adopted by one embodiment of a sample library establishing method
FIG. 8: the principle schematic diagram of the sample rotation operation adopted by the embodiment of the sample library establishment method provided by the invention is shown.
FIG. 9: the schematic diagram of the principle of performing Gamma stretching operation on the sample is adopted for one embodiment of the sample library establishing method provided by the invention.
FIG. 10: the present invention provides a schematic structural diagram of an embodiment of a sample library creating apparatus.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment is as follows:
referring to fig. 1, fig. 1 is a sample library establishing method based on deep learning according to an embodiment of the present invention, including steps S1 to S6, wherein,
step S1, an HRNet network is built, a plurality of cloud samples are trained to obtain a cloud detection model, and samples with the cloud content smaller than a first preset value in an initial sample library are preprocessed to serve as a high-quality sample library through the cloud detection model.
In this embodiment, the existing data set (influence and complement vectors) is first sampled to form an initial sample library, and the sample size is 512 pixels × 512 pixels.
And constructing an HRNet network, and training by utilizing a plurality of existing cloud samples to obtain a cloud detection model. The cloud content of each sample in the initial sample library is extracted through a cloud detection model, the samples with the cloud content smaller than a first preset value in the initial sample library are preprocessed and used as a high-quality sample library, and the samples with the cloud content larger than or equal to the first preset value in the initial sample library are directly removed. As an example of this embodiment, the preprocessing specifically includes: and screening all samples with the cloud content smaller than a first preset value in the initial sample library, selecting not less than 10000 samples for each scene, and manually drawing and modifying until the samples meet preset requirements to form a high-quality sample library.
The schematic diagram of the network structure of the HRNet is shown in FIG. 2. The HRNet network is a parallel connection network structure, high-resolution and low-resolution feature layers are fused, the high-resolution feature layers are kept, geometric information of an image is effectively kept, multi-scale feature integration operation is completed by repeatedly integrating the same-level and multi-level feature layers, and the capability of a model for extracting context features can be remarkably enhanced.
The HRNet takes a high-resolution subnet as a first stage, gradually increases the subnets from high resolution to low resolution to form more stages, connects the multi-resolution subnets, and realizes multi-scale repeated fusion by repeatedly exchanging information on the parallel multi-resolution subnets.
The HRNet has the biggest characteristic that multi-level feature integration is carried out while a high-resolution feature layer is kept. The principle is shown in fig. 3 and 4. Fig. 3 is a schematic diagram of a network convolution operation of the HRNet according to an embodiment of the method for establishing the sample library provided by the present invention, and fig. 4 is a schematic diagram of a network feature integration of the HRNet according to an embodiment of the method for establishing the sample library provided by the present invention.
The computational process of the convolution operation of fig. 3 can be expressed as:
Y＝H(X)；
where h (x) represents a convolution calculation performed on the feature map layer. Wherein X is a feature map layer, which is composed of different levels of features, i.e. integrated from X1 to X4. Y is formed by integrating Y1-Y4 and is the result of convolution calculation on the X characteristic map layer.
FIG. 4 is a representation of the integration of features:
Y1＝H1(concat(X1，…，X4))；
Y2＝H2(concat(X1，…，X4))；
Y3＝H3(concat(X1，…，X4))；
Y4＝H4(concat(X1，…，X4))；
after the formula is combined, the calculation process is practically equal to the convolution operation process. I.e. the feature integration in fig. 4 is equivalent to performing a convolution operation on the feature map layer. This can reduce the semantic gap when integrating features at different levels.
On the basis, the HRNet network performs downsampling on the feature layers except for X1 and Y1, so that the calculated amount of a model can be reduced, the features can be further semantically summarized, the multi-level features of the image are extracted, and the semantic features are more obvious when the size of the feature layers is reduced.
And step S2, performing augmentation processing on all samples in the high-quality sample library, and adding the samples obtained through the augmentation processing into the high-quality sample library.
Specifically, all samples in the high-quality sample library are subjected to augmentation processing by using a least square generation type confrontation network model, and the samples obtained through augmentation processing are added into the high-quality sample library.
The basic model structure of the generative countermeasure network is shown in fig. 5. G and D denote the generator and the discriminator, respectively. The generator G inputs a random variable z and the arbiter D inputs the data G (z) generated by the generator and the real data x, G (z) representing samples generated by the generator G that approximate the distribution of the real data. The purpose of generator G is to make the performance D (G (z)) of the generated data G (z) on discriminator D fool discriminator D into that the discriminator D cannot distinguish whether the data source is true or false. The purpose of discriminator D is to discriminate between true and false data sources. Where "true" is derived from the distribution of data x and "false" is derived from the distribution of generator generated data g (z). The two game countermeasures and iterative optimization are carried out mutually, when the discrimination capability of the D is not improved any more, the performance of the G is optimal, and the generator G learns the distribution close to the real data.
A schematic diagram of the structure of the conditional countermeasure network is shown in fig. 6. Conditional generative countermeasure networks additional information y is introduced into the generator G and the discriminator D on the basis of the generative countermeasure network. Here, y may be any auxiliary information, such as remote sensing image category information or vector information.
In the embodiment of the invention, the generation of the remote sensing image sample is generated towards a specific purpose, so the divergence Kagan divergence is adopted as a target of loss function design, and the least square generation type countermeasure network method is further expanded to the condition of the least square generation type countermeasure network, so that the method is suitable for the expansion of the high-resolution remote sensing image training sample. Fig. 7 is a schematic structural diagram of a conditional least squares generation type confrontation network model according to an embodiment of the present invention.
The conditional least square generation type confrontation network model mainly comprises a generator G and a discriminator D. The G sub-network of the generator is composed of an encoder/decoder and a TanH function, and the nonlinear activation function of each layer is a modified linear unit function (ReLU).
Further, the output feature dimensions of the encoder/decoder convolutional layers in the generator structure are as follows:
encoder (Encoder): C256-C128-C64-C32-C16-C8-C4-C2;
decoder (Decoder): C2-C4-C8-C16-C32-C64-C128-C256;
the discriminator D is composed of five convolution layers, and the structure is as follows: C256-C128-C64-C32-C30. The nonlinear activation function also adopts a ReLU function, and simultaneously, in consideration of the purpose of a design condition least square generation type countermeasure network, a weight sharing mechanism is adopted when a discriminator D is designed, and a Softmax loss layer is added to be used as a semantic segmentation class prediction layer so as to monitor the performance of the semantic segmentation network and terminate training at proper time.
The samples based on conditional least square generation type antagonistic network expansion can be used for solving the problem that part of samples are less distributed, so that the samples are more uniformly distributed, and the robustness of an interpretation model is improved.
Further, in addition to performing the augmentation process based on the conditional least square generation countermeasure network, the method of augmentation process further includes but is not limited to performing Gamma stretching, gaussian blurring, scaling, rotation, and flipping processes on the sample.
Specifically, referring to fig. 8, the rotation operation of the sample is accomplished by rotating the matrix. Rotating the vector a to the position of the vector b around the origin by the angle of rotation
If the length of the vector is r, then:
xa＝rcosα；
ya＝rsinα；
wherein (x)a，ya) Is the coordinate of vector a, (x)b，yb) As the coordinates of vector b.
Further, the method can be obtained as follows:
the corresponding rotation matrix is then:
the matrix form of the sample rotation is:
the scaling transform changes the sample size and resolution without changing the sample shape. The scaling matrix is as follows:
s is a scaling factor. The matrix form of the sample scaling is:
the sample turning comprises horizontal turning and vertical turning, and the horizontal turning and the vertical turning are carried out simultaneously. Take horizontal flipping as an example:
gamma stretching detects dark and light color parts in the sample by a nonlinear tone editing method, so that the proportion is increased, and the contrast effect of the sample is improved. The basic form of Gamma stretching is shown in fig. 9.
As can be seen from fig. 9, when the Gamma value is less than 1, the region of the sample with lower gray level is stretched, and the region with higher gray level is compressed. The Gamma value is larger than 1, the area with higher gray level in the sample is stretched, and the area with lower gray level is compressed.
Gaussian blur is performed by using a pixel value around a certain pixel to perform Gaussian model processing, specifically, a certain pixel is used as a central point, a blur radius is set, a weight is calculated for the surrounding pixels, and the surrounding pixels are used for weighting to obtain an enhanced pixel value.
And step S3, training all samples of the high-quality sample library through the HRNet network to obtain a first interpretation model.
Step S4, using the first interpretation model to interpret all samples in the high quality sample library and calculate the pixel precision of each sample, bringing the samples meeting the first predetermined pixel precision requirement into a new sample library, performing augmentation processing on the samples meeting the first predetermined pixel precision requirement, and bringing the samples obtained by the augmentation processing into the new sample library.
In this embodiment, all samples in the high quality sample library are interpreted and the pixel precision of each sample is calculated respectively by using the first interpretation model, specifically, the statistics of precision is performed in units of pixels with the deep learning interpretation grid result as a reference value. The size of the sample in this embodiment is 512 pixels × 512 pixels, and as an example of this embodiment, the total number of pixels is 262144. And (4) the pixel value of the sample is consistent with the pixel value of the deep learning interpretation result to be a correct pixel, and the pixel precision of the sample is counted. Assuming that the number of correct pixels in a sample is X, the pixel accuracy of the sample is:
the accuracy rate is X/262144 multiplied by 100%;
and bringing the sample meeting the first preset pixel precision requirement into a new sample library, carrying out amplification processing on the sample meeting the first preset pixel precision requirement, and bringing the sample obtained through the amplification processing into the new sample library. The method comprises the following steps of performing augmentation treatment on a sample meeting a first preset pixel precision requirement, specifically: and performing augmentation treatment on the sample meeting the first preset pixel precision requirement by using the least square generation type countermeasure network model, or performing operations such as Gamma stretching, Gaussian blurring, zooming, rotating, overturning and the like on the sample to perform augmentation.
And step S5, utilizing the new sample library to perform iterative optimization on the first interpretation model until convergence, so as to obtain a second interpretation model.
Specifically, in this embodiment, the first interpretation model is used as an initial model, and the new sample library is used to perform iterative optimization on the first interpretation model, where each iterative optimization performs the following steps: interpreting the samples which are not preprocessed and have the cloud content smaller than the first preset value (namely, interpreting the samples which are detected by the cloud detection model and have the cloud content smaller than the first preset value but are not screened out and modified by manual drawing) in the initial sample library of the step S1, and calculating the corresponding pixel precision according to the interpretation result; and bringing the sample meeting the first preset pixel precision requirement into a new sample library, carrying out amplification processing on the sample meeting the first preset pixel precision requirement, and bringing the sample obtained through the amplification processing into the new sample library. And training the first interpretation model by using the latest new sample library until the first interpretation model tends to be stable, and obtaining a second interpretation model.
And step S6, utilizing the second interpretation model to interpret the samples which are not preprocessed and have the cloud content smaller than the first preset value in the initial sample library and respectively calculate the pixel precision, adding a new sample library into the samples meeting the requirement of the second preset pixel precision and carrying out augmentation treatment, and adding the samples obtained through the augmentation treatment into the new sample library to form a natural resource sample library.
Step S6 employs the same method of calculating pixel accuracy as step S4, but adds a new sample library to the samples that satisfy the second preset pixel accuracy requirement and performs augmentation processing. The second preset pixel precision requirement may be the same as or different from the first preset pixel precision requirement, and is specifically selected according to the actual needs of the model. And adding the sample obtained through the amplification treatment into the new sample library to form a natural resource sample library. The augmentation process can utilize the least square generation type confrontation network model, and can also adopt Gamma stretching, Gaussian blurring, scaling, rotation and overturning processes.
Correspondingly, referring to fig. 10, fig. 10 is a sample library establishing apparatus based on deep learning according to the present invention, including: the system comprises a construction module 101, an augmentation module 102, a training module 103, a new sample module 104, an iterative optimization module 105 and a natural resource sample module 106;
the building module 101 is used for building an HRNet network, training a plurality of cloud samples to obtain a cloud detection model, and preprocessing the samples with the cloud content smaller than a first preset value in an initial sample library through the cloud detection model to be used as a high-quality sample library;
the augmentation module 102 is configured to perform augmentation processing on all samples in the high-quality sample library, and add the samples obtained through the augmentation processing into the high-quality sample library;
the training module 103 is configured to train all samples of the high-quality sample library through the HRNet network to obtain a first interpretation model;
the new sample module 104 is configured to interpret all samples in the high-quality sample library by using the first interpretation model, calculate pixel precision of each sample, bring a sample meeting a first preset pixel precision requirement into the new sample library, perform augmentation processing on the sample meeting the first preset pixel precision requirement, and bring a sample obtained through the augmentation processing into the new sample library;
the iterative optimization module 105 is configured to perform iterative optimization on the first interpretation model by using the new sample library until convergence, so as to obtain a second interpretation model;
the natural resource sample module 106 is configured to interpret, by using the second interpretation model, the non-preprocessed samples with the cloud content smaller than the first preset value in the initial sample library, calculate pixel accuracies respectively, add a new sample library to the samples meeting the second preset pixel accuracy requirement, perform augmentation processing, and add the samples obtained through the augmentation processing to the new sample library to form a natural resource sample library.
In this embodiment, the amplification module 102 performs an amplification process on all samples in the high-quality sample library, specifically:
the augmentation module 102 performs augmentation processing on all samples in the high-quality sample library by using a least square generation type confrontation network model;
the new sample module 104 performs an amplification process on a sample meeting a first preset pixel precision requirement, specifically:
the new sample module 104 performs augmentation processing on the sample meeting the first preset pixel precision requirement by using the least square generation type confrontation network model and brings the augmented sample into a new sample library;
the natural resource sample module 106 adds a new sample library to the sample meeting the second preset pixel precision requirement, and performs augmentation processing, specifically:
the natural resource sample module 106 adds a new sample library to the sample meeting the second predetermined pixel precision requirement, and performs augmentation processing on the countermeasure network model using the least square generation method.
In this embodiment, the objective of the least squares generation countermeasure network model's loss function design is Kagan divergence.
In this embodiment, the method of the augmentation processing further includes: and performing Gamma stretching, Gaussian blurring, scaling, rotation and overturning on the sample.
In this embodiment, the constructing module preprocesses the sample with the cloud content smaller than the first preset value in the initial sample library and uses the sample as the high-quality sample library, specifically: the construction module screens out all samples with the cloud content smaller than a first preset value in an initial sample library, selects not less than 10000 samples for each scene from the screened samples, and takes the samples as a high-quality sample library after manual modification until the samples meet preset requirements.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
the embodiment of the invention provides a sample library establishing method and device based on deep learning, wherein the method comprises the following steps: training a plurality of cloud samples to obtain a cloud detection model, preprocessing all samples with the cloud content smaller than a first preset value in an initial sample library through the cloud detection model, and taking the preprocessed samples as a high-quality sample library; carrying out augmentation treatment on the samples in the high-quality sample library; training the sample through an HRNet network to obtain a first interpretation model; interpreting all samples in the high-quality sample library by using the first interpretation model, respectively calculating the pixel precision of each sample, carrying out augmentation processing on the samples meeting the preset pixel precision requirement, bringing the samples into a new sample library, and bringing the samples obtained through the augmentation processing into the new sample library; performing iterative optimization on the first interpretation model by using the new sample library until convergence to obtain a second interpretation model; and by utilizing the second interpretation model, interpreting the samples which are not preprocessed and have the cloud content smaller than the first preset value in the initial sample library, respectively calculating the pixel precision, and performing augmentation processing on the samples meeting the preset pixel precision requirement to obtain a natural resource sample library. Compared with the prior art, the method effectively retains the geometric information of the image through the HRNet network, realizes multi-scale feature integration, and obviously enhances the context feature extraction capability of the model; in the process of establishing the natural resource sample library, automatic augmentation operation on samples can be simultaneously realized, manual addition of new samples or manual modification on the samples are not needed, the precision of the interpretation model is improved through a deep learning technology, the sample quality is improved, the requirement of intelligent interpretation is met, the timeliness is improved, and the purpose of saving manpower and material resources is achieved.
The above-mentioned embodiments are provided to further explain the objects, technical solutions and advantages of the present invention in detail, and it should be understood that the above-mentioned embodiments are only examples of the present invention and are not intended to limit the scope of the present invention. It should be understood that any modifications, equivalents, improvements and the like, which come within the spirit and principle of the invention, may occur to those skilled in the art and are intended to be included within the scope of the invention.
Claims (10)
1. A sample library establishing method based on deep learning is characterized by comprising the following steps:
constructing an HRNet network, training a plurality of cloud samples to obtain a cloud detection model, preprocessing the samples with the cloud content smaller than a first preset value in an initial sample library through the cloud detection model, and using the preprocessed samples as a high-quality sample library;
performing augmentation treatment on all samples in the high-quality sample library, and adding the samples obtained through the augmentation treatment into the high-quality sample library;
training all samples of the high-quality sample library through the HRNet network to obtain a first interpretation model;
interpreting all samples in the high-quality sample library by using the first interpretation model, respectively calculating the pixel precision of each sample, bringing the samples meeting a first preset pixel precision requirement into a new sample library, performing augmentation processing on the samples meeting the first preset pixel precision requirement, and bringing the samples obtained by the augmentation processing into the new sample library;
performing iterative optimization on the first interpretation model by using the new sample library until convergence to obtain a second interpretation model;
and utilizing the second interpretation model to interpret the samples which are not preprocessed and have the cloud content smaller than the first preset value in the initial sample library, respectively calculating the pixel precision, adding a new sample library into the samples meeting the second preset pixel precision requirement, performing amplification treatment, and adding the samples obtained through the amplification treatment into the new sample library to form a natural resource sample library.
2. The method for establishing the sample library based on the deep learning of claim 1, wherein the augmenting process is performed on all samples in the high-quality sample library, specifically:
performing augmentation processing on all samples in the high-quality sample library by using a least square generation type countermeasure network model;
the method comprises the following steps of performing augmentation processing on a sample meeting a first preset pixel precision requirement, specifically:
carrying out augmentation treatment on the sample meeting the first preset pixel precision requirement by utilizing the least square generation type confrontation network model;
the method comprises the following steps of adding a new sample library to a sample meeting a second preset pixel precision requirement and carrying out augmentation treatment, and specifically comprises the following steps:
and adding a new sample base to the samples meeting the second preset pixel precision requirement, and performing augmentation processing by using the least square generation type countermeasure network model.
3. The method as claimed in claim 2, wherein the least square generated solution network model loss function design is targeted to Kagan divergence.
4. The method for building the sample library based on the deep learning of claim 1, wherein the method for the augmentation process further comprises: and performing Gamma stretching, Gaussian blurring, scaling, rotation and overturning on the sample.
5. The deep learning-based sample bank establishing method according to any one of claims 1 to 4, wherein the sample with the cloud content less than the first preset value in the initial sample bank is preprocessed and used as a high-quality sample bank, and specifically comprises: screening all samples with the cloud content smaller than a first preset value in the initial sample library, and selecting not less than 10000 samples as a high-quality sample library for each scene from the screened samples.
6. A sample base establishing device based on deep learning is characterized by comprising: the system comprises a construction module, an augmentation module, a training module, a new sample module, an iterative optimization module and a natural resource sample module;
the building module is used for building an HRNet network, training a plurality of cloud samples to obtain a cloud detection model, and preprocessing the samples with the cloud content smaller than a first preset value in an initial sample library through the cloud detection model to be used as a high-quality sample library;
the amplification module is used for performing amplification treatment on all samples in the high-quality sample library and adding the samples obtained through the amplification treatment into the high-quality sample library;
the training module is used for training all samples of the high-quality sample library through the HRNet network to obtain a first interpretation model;
the new sample module is used for interpreting all samples in the high-quality sample library by using the first interpretation model, respectively calculating the pixel precision of each sample, bringing the samples meeting a first preset pixel precision requirement into the new sample library, carrying out augmentation processing on the samples meeting the first preset pixel precision requirement, and bringing the samples obtained by the augmentation processing into the new sample library;
the iterative optimization module is used for carrying out iterative optimization on the first interpretation model by utilizing the new sample library until convergence to obtain a second interpretation model;
the natural resource sample module is used for utilizing the second interpretation model to interpret samples which are not preprocessed and have the cloud content smaller than a first preset value in the initial sample library, respectively calculating pixel precision, adding a new sample library into the samples meeting the requirement of second preset pixel precision, carrying out amplification processing, adding the samples obtained through the amplification processing into the new sample library, and forming a natural resource sample library.
7. The deep learning-based sample library creation device according to claim 6, wherein the augmentation module performs augmentation processing on all samples in the high-quality sample library, specifically:
the augmentation module performs augmentation processing on all samples in the high-quality sample library by using a least square generation type confrontation network model;
the new sample module is used for carrying out augmentation processing on a sample meeting a first preset pixel precision requirement, and specifically comprises the following steps:
the new sample module is used for carrying out augmentation treatment on the sample meeting the first preset pixel precision requirement by using the least square generation type confrontation network model and bringing the sample into a new sample library;
the natural resource sample module adds a new sample library to a sample meeting a second preset pixel precision requirement and performs augmentation processing, and specifically comprises the following steps:
and the natural resource sample module adds a new sample base to the sample meeting the second preset pixel precision requirement and performs augmentation processing by using the least square generation type countermeasure network model.
8. The apparatus as claimed in claim 7, wherein the least squares generated confrontation network model loss function is designed with a target of Kagan divergence.
9. The deep learning-based sample bank building device according to claim 6, wherein the method of augmentation processing further comprises: and performing Gamma stretching, Gaussian blurring, scaling, rotation and overturning on the sample.
10. The deep learning-based sample bank establishing device according to any one of claims 6 to 9, wherein the constructing module preprocesses a sample with a cloud content smaller than a first preset value in an initial sample bank and uses the sample as a high-quality sample bank, and specifically comprises: the construction module screens all samples with the cloud content smaller than a first preset value in an initial sample library, selects not less than 10000 samples for each scene from the screened samples, and takes the samples as a high-quality sample library after manual modification until the samples meet preset requirements.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111521298.7A CN114445726B (en) | 2021-12-13 | 2021-12-13 | Sample library establishing method and device based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111521298.7A CN114445726B (en) | 2021-12-13 | 2021-12-13 | Sample library establishing method and device based on deep learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114445726A true CN114445726A (en) | 2022-05-06 |
| CN114445726B CN114445726B (en) | 2022-08-02 |
Family
ID=81364824
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111521298.7A Active CN114445726B (en) | 2021-12-13 | 2021-12-13 | Sample library establishing method and device based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114445726B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117392486A (en) * | 2023-12-12 | 2024-01-12 | 湖北珞珈实验室 | Method, device, equipment and storage medium for constructing natural resource element sample library |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3157017A1 (en) * | 2015-10-18 | 2017-04-19 | Carl Zeiss X-Ray Microscopy, Inc. | Method for combining tomographic volume data sets and image analysis tool of an x-ray imaging microscopy system |
| CN109741340A (en) * | 2018-12-16 | 2019-05-10 | 北京工业大学 | Refinement method of ice layer segmentation in ice sheet radar image based on FCN-ASPP network |
| CN109840553A (en) * | 2019-01-17 | 2019-06-04 | 苏州中科天启遥感科技有限公司 | The extracting method and system, storage medium, electronic equipment for agrotype of ploughing |
| CN112883839A (en) * | 2021-02-02 | 2021-06-01 | 重庆市地理信息和遥感应用中心 | Remote sensing image interpretation method based on adaptive sample set construction and deep learning |
| US20210241027A1 (en) * | 2018-11-30 | 2021-08-05 | Tencent Technology (Shenzhen) Company Limited | Image segmentation method and apparatus, diagnosis system, storage medium, and computer device |
-
2021
- 2021-12-13 CN CN202111521298.7A patent/CN114445726B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3157017A1 (en) * | 2015-10-18 | 2017-04-19 | Carl Zeiss X-Ray Microscopy, Inc. | Method for combining tomographic volume data sets and image analysis tool of an x-ray imaging microscopy system |
| US20210241027A1 (en) * | 2018-11-30 | 2021-08-05 | Tencent Technology (Shenzhen) Company Limited | Image segmentation method and apparatus, diagnosis system, storage medium, and computer device |
| CN109741340A (en) * | 2018-12-16 | 2019-05-10 | 北京工业大学 | Refinement method of ice layer segmentation in ice sheet radar image based on FCN-ASPP network |
| CN109840553A (en) * | 2019-01-17 | 2019-06-04 | 苏州中科天启遥感科技有限公司 | The extracting method and system, storage medium, electronic equipment for agrotype of ploughing |
| CN112883839A (en) * | 2021-02-02 | 2021-06-01 | 重庆市地理信息和遥感应用中心 | Remote sensing image interpretation method based on adaptive sample set construction and deep learning |
Non-Patent Citations (1)
| Title |
|---|
| 郭海京;高绵新: "基于LiDAR的DEM水域高程赋值研究", 《地理空间信息》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117392486A (en) * | 2023-12-12 | 2024-01-12 | 湖北珞珈实验室 | Method, device, equipment and storage medium for constructing natural resource element sample library |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114445726B (en) | 2022-08-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Li et al. | Building-a-nets: Robust building extraction from high-resolution remote sensing images with adversarial networks | |
| CN113628180B (en) | Remote sensing building detection method and system based on semantic segmentation network | |
| CN111126412B (en) | Image key point detection method based on characteristic pyramid network | |
| CN111860514B (en) | Multi-category real-time segmentation method for orchard scene based on improvement DeepLab | |
| CN114596290B (en) | Defect detection method and device, storage medium, and program product | |
| CN110781882A (en) | License plate positioning and identifying method based on YOLO model | |
| CN113516126A (en) | An adaptive threshold scene text detection method based on attention feature fusion | |
| CN109886159B (en) | Face detection method under non-limited condition | |
| Fan et al. | Registration of multiresolution remote sensing images based on L2-siamese model | |
| CN116258608B (en) | Water conservancy real-time monitoring information management system integrating GIS and BIM three-dimensional technology | |
| CN115546502A (en) | Infrared small target detection method based on YOLOv4 multi-scale feature fusion | |
| CN111310609B (en) | Video target detection method based on time sequence information and local feature similarity | |
| CN113159466A (en) | Short-time photovoltaic power generation prediction system and method | |
| CN113838064A (en) | A Cloud Removal Method Using Multitemporal Remote Sensing Data Based on Branch GAN | |
| CN113205028A (en) | Pedestrian detection method and system based on improved YOLOv3 model | |
| CN114758123A (en) | A method for enhancing remote sensing image target samples | |
| CN112329771B (en) | Deep learning-based building material sample identification method | |
| CN114445726B (en) | Sample library establishing method and device based on deep learning | |
| CN116030361A (en) | CIM-T architecture-based high-resolution image change detection method | |
| CN114463624B (en) | Method and device for detecting illegal building applied to urban management supervision | |
| CN116977674A (en) | Image matching method, related device, storage medium and program product | |
| CN113971760A (en) | High-quality quasi-dense complementary feature extraction method based on deep learning | |
| CN113963265A (en) | Small sample small target rapid detection and identification method for complex remote sensing land environment | |
| Skartados et al. | Finding Waldo: Towards Efficient Exploration of NeRF Scene Spaces | |
| CN116129191B (en) | Multi-target intelligent identification and fine classification method based on remote sensing AI |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |