CN115909059A - Natural resource sample library establishing method and device - Google Patents
Natural resource sample library establishing method and device Download PDFInfo
- Publication number
- CN115909059A CN115909059A CN202211410724.4A CN202211410724A CN115909059A CN 115909059 A CN115909059 A CN 115909059A CN 202211410724 A CN202211410724 A CN 202211410724A CN 115909059 A CN115909059 A CN 115909059A
- Authority
- CN
- China
- Prior art keywords
- remote sensing
- tile data
- tile
- target
- vector
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Abstract
The invention provides a method and a device for establishing a natural resource sample library, which relate to the technical field of remote sensing image processing and comprise the following steps: obtaining a sample remote sensing image, wherein the sample remote sensing image comprises: the remote sensing image is used for performing semantic segmentation, target recognition and change detection; carrying out artificial visual interpretation on the sample remote sensing image to obtain a target sample remote sensing image; carrying out tile segmentation on the target sample remote sensing image by using vector labeling data of the sample remote sensing image to obtain a tile data set, and configuring a label file for the tile data in the tile data set; a natural resource sample library is constructed based on a tile data set and a label file, and the technical problem that samples in different batches are often processed and converted in different modes to be input into a deep learning model for training, so that the training efficiency of the deep learning model is low is solved.
Description
Technical Field
The invention relates to the technical field of remote sensing image processing, in particular to a method and a device for establishing a natural resource sample library.
Background
With the development of remote sensing technology, the remote sensing technology has evolved towards multispectral and high-resolution, and especially the field of monitoring natural resources by using high-resolution remote sensing images has great significance. In the face of complex ground targets and massive remote sensing data, how to perform efficient processing and accurate information extraction on remote sensing images becomes a key problem to be solved urgently. The remote sensing image automatic interpretation based on deep learning is an important scheme for solving the problem.
Deep learning is a machine learning method that can be used as an artificial neural network to independently build (train) ground rules from example data in the learning process. In the field of machine vision in particular, neural networks are often trained using supervised learning, i.e. by example data and predefined results of example data. Example data is remote sensing images, and example data is sample labels marked based on the images. For the natural resource remote sensing image interpretation task, the problems which can be solved by the deep learning technology mainly include semantic segmentation, target recognition and change detection.
A massive and multi-type remote sensing image sample library is the basis for realizing high-precision intelligent interpretation of large-range heterogeneous remote sensing images. However, in the current business, due to the lack of the construction specifications of the remote sensing image sample library, samples in different batches often need to be processed and converted in different ways to be input into the deep learning model for training, and even the samples can not meet the training standards, so that the efficiency of deep learning model training is greatly hindered. In addition, the result directly obtained through deep learning model reasoning is in a single-channel tile image form, so that visual test and evaluation work is difficult to perform, the production requirement is not met, and post-processing work such as splicing is needed.
No effective solution has been proposed to the above problems.
Disclosure of Invention
In view of this, the present invention provides a method and an apparatus for establishing a natural resource sample library, so as to solve the technical problem that samples of different batches often need to be processed and converted in different ways to be input into a deep learning model for training, which results in low training efficiency of the deep learning model.
In a first aspect, an embodiment of the present invention provides a method for establishing a natural resource sample library, including: obtaining a sample remote sensing image, wherein the sample remote sensing image comprises: the remote sensing image is used for performing semantic segmentation, target recognition and change detection; carrying out artificial visual interpretation on the sample remote sensing image to obtain a target sample remote sensing image; carrying out tile segmentation on the target sample remote sensing image by using vector labeling data of the sample remote sensing image to obtain a tile data set, and configuring a label file for tile data in the tile data set; and constructing a natural resource sample library based on the tile data set and the label file.
Further, if the sample remote sensing image is a remote sensing image for semantic segmentation, tile segmentation is performed on the target sample remote sensing image by using vector labeling data of the sample remote sensing image to obtain a tile data set, and a label file is configured for the tile data in the tile data set, including: determining whether an overlapping area exists between the sample remote sensing image and the vector marking data; if the tile data exists, the sample remote sensing image and the vector marking data are respectively segmented according to a preset size and a preset overlap size to obtain first image tile data of the sample remote sensing image and first vector tile data of the vector marking data, wherein the preset overlap size is the size of an overlap area between any two adjacent tiles; determining first target image tile data in the image tile data and first target vector tile data in the vector tile data, wherein the first target image tile data and the first target vector tile data coincide with each other; constructing a first tile data set based on the first target image tile data; and generating a first tile label sample picture based on the target vector tile data, and determining the first tile label sample picture as a first label file, wherein the pixel values of background pixels and various ground object pixels in the first tile label sample picture are different.
Further, if the sample remote sensing image is a remote sensing image used for target identification, tile segmentation is performed on the target sample remote sensing image by using vector labeling data of the sample remote sensing image to obtain a tile data set, and a tag file is configured for tile data in the tile data set, including: determining whether an overlapping area exists between the sample remote sensing image and the vector marking data; if the tile data exists, the sample remote sensing image and the vector marking data are respectively segmented according to a preset size and a preset overlap size to obtain second image tile data of the sample remote sensing image and second vector tile data of the vector marking data, wherein the preset overlap size is the size of an overlap area between any two adjacent tiles; determining second target image tile data in the image tile data and second target vector tile data in the vector tile data, wherein the second target image tile data and the second target vector tile data coincide with each other; constructing a second tile data set based on the second target image tile data; generating a text file based on the circumscribed rectangle of each target ground object in the second target vector tile data, and determining the text file as a second label file, wherein the text file comprises the position information and the category information of the target ground object.
Further, if the sample remote sensing image is a remote sensing image for change detection, the sample remote sensing image includes a front time phase image and a time phase image; utilizing the vector labeling data of the sample remote sensing image to perform tile segmentation on the target sample remote sensing image to obtain a tile data set, and configuring a label file for the tile data in the tile data set, wherein the tile segmentation comprises the following steps: determining whether an overlapping area exists between the sample remote sensing image and the vector marking data; if the tile data exists, the sample remote sensing image and the vector marking data are respectively segmented according to a preset size and a preset overlap size to obtain third image tile data of the sample remote sensing image and third vector tile data of the vector marking data, wherein the preset overlap size is the size of an overlap area between any two adjacent tiles; determining third target image tile data in the image tile data and third target vector tile data in the vector tile data, wherein the third target image tile data and the third target vector tile data coincide with each other; constructing a third tile data set based on the third target image tile data; and generating a second tile label sample picture based on the third target vector tile data, and determining the tile label sample picture as a second label file, wherein the pixel values of background pixels and various ground object pixels in the second tile label sample picture are different.
Further, the method further comprises: respectively inputting the sample remote sensing images into corresponding deep learning models to obtain vector results of the deep learning models, wherein the deep learning models comprise: a semantic segmentation model, a target identification model and a change detection model; and determining a precision evaluation result based on the vector result of each deep learning model, the first target vector tile data, the second target vector tile data and the third target vector tile data, wherein the precision evaluation result is used for representing the performance of each deep learning model.
In a second aspect, an embodiment of the present invention further provides an apparatus for creating a natural resource sample library, including: the acquisition unit is used for acquiring a sample remote sensing image, wherein the sample remote sensing image comprises: the remote sensing image is used for performing semantic segmentation, target recognition and change detection; the interpretation unit is used for carrying out artificial visual interpretation on the sample remote sensing image to obtain a target sample remote sensing image; the segmentation unit is used for performing tile segmentation on the target sample remote sensing image by using the vector labeling data of the sample remote sensing image to obtain a tile data set, and configuring a label file for the tile data in the tile data set; and the construction unit is used for constructing a natural resource sample library based on the tile data set and the label file.
Further, if the sample remote sensing image is a remote sensing image for performing semantic segmentation, the segmentation unit is configured to: determining whether an overlapping area exists between the sample remote sensing image and the vector marking data; if the tile data exists, the sample remote sensing image and the vector marking data are respectively segmented according to a preset size and a preset overlap size to obtain first image tile data of the sample remote sensing image and first vector tile data of the vector marking data, wherein the preset overlap size is the size of an overlap area between any two adjacent tiles; determining first target image tile data in the image tile data and first target vector tile data in the vector tile data, wherein the first target image tile data and the first target vector tile data coincide with each other; constructing a first tile data set based on the first target image tile data; and generating a first tile label sample picture based on the target vector tile data, and determining the first tile label sample picture as a first label file, wherein the pixel values of background pixels and various ground object pixels in the first tile label sample picture are different.
Further, if the sample remote sensing image is a remote sensing image for target recognition, the segmentation unit is configured to: determining whether an overlapping area exists between the sample remote sensing image and the vector marking data; if the tile data exists, the sample remote sensing image and the vector marking data are respectively segmented according to a preset size and a preset overlap size to obtain second image tile data of the sample remote sensing image and second vector tile data of the vector marking data, wherein the preset overlap size is the size of an overlap area between any two adjacent tiles; determining second target image tile data in the image tile data and second target vector tile data in the vector tile data, wherein the second target image tile data and the second target vector tile data coincide with each other; constructing a second tile data set based on the second target image tile data; and generating a text file based on the circumscribed rectangle of each target ground object in the second target vector tile data, and determining the text file as a second label file, wherein the text file comprises the position information and the category information of the target ground object.
In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, where the memory is used to store a program that supports the processor to execute the method in the first aspect, and the processor is configured to execute the program stored in the memory.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored.
In the embodiment of the present invention, a sample remote sensing image is obtained, where the sample remote sensing image includes: the remote sensing image is used for performing semantic segmentation, target recognition and change detection; carrying out artificial visual interpretation on the sample remote sensing image to obtain a target sample remote sensing image; carrying out tile segmentation on the target sample remote sensing image by using vector labeling data of the sample remote sensing image to obtain a tile data set, and configuring a label file for tile data in the tile data set; based on the tile data set and the label file, a natural resource sample library is constructed, the purpose of providing training samples with the same standard for various deep learning models is achieved, and the technical problem that samples in different batches can be input into the deep learning models for training only by being processed and converted in different modes, so that the training efficiency of the deep learning models is low is solved, and the technical effect of improving the training efficiency of the deep learning models is achieved.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a flowchart of a natural resource sample library establishment method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a natural resource sample library creating apparatus according to an embodiment of the present invention;
fig. 3 is a schematic diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
To make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment is as follows:
in accordance with an embodiment of the present invention, there is provided an embodiment of a natural resource sample repository establishment method, it should be noted that the steps illustrated in the flowchart of the drawings may be performed in a computer system such as a set of computer executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.
Fig. 1 is a flowchart of a natural resource sample library establishing method according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:
step S102, obtaining a sample remote sensing image, wherein the sample remote sensing image comprises: the remote sensing image is used for performing semantic segmentation, target recognition and change detection;
step S104, carrying out artificial visual interpretation on the sample remote sensing image to obtain a target sample remote sensing image;
step S106, utilizing vector labeling data of the sample remote sensing image to perform tile segmentation on the target sample remote sensing image to obtain a tile data set, and configuring a label file for the tile data in the tile data set;
and S108, constructing a natural resource sample library based on the tile data set and the label file.
In the embodiment of the present invention, a sample remote sensing image is obtained, where the sample remote sensing image includes: the remote sensing image is used for performing semantic segmentation, target recognition and change detection; carrying out artificial visual interpretation on the sample remote sensing image to obtain a target sample remote sensing image; performing tile segmentation on the target sample remote sensing image by using vector labeling data of the sample remote sensing image to obtain a tile data set, and configuring a label file for tile data in the tile data set; based on the tile data set and the label file, a natural resource sample library is constructed, the purpose of providing training samples with the same standard for various deep learning models is achieved, and the technical problem that samples in different batches are often processed and converted in different modes to be input into the deep learning models for training, so that the training efficiency of the deep learning models is low is solved, and the technical effect of improving the training efficiency of the deep learning models is achieved.
It should be noted that the sample remote sensing image is composed of an image file and a vector file. The sample collection was performed in a manner that was interpreted manually and visually. Visual interpretation refers to a thinking process of comprehensive analysis and logical reasoning from one to another, from table to inside and from false-proof and true-proof by using the image characteristics (hue or color, namely, spectral characteristics) and spatial characteristics (shape, size, shadow, texture, graph, position and layout) of an image and combining with various non-remote sensing information data (such as a topographic map and various thematic data) and applying the relevant rules. Although the manual visual interpretation takes long time and has low efficiency, the samples with accurate definition and high quality can be selected.
Semantic segmentation, a classic computer vision problem, involves taking raw data as input and converting them into a mask with highlighted regions of interest, where each pixel in the image is assigned a class ID according to the object to which it belongs. The semantic segmentation sample of the remote sensing image consists of a raster data set, vector labels and a metadata file.
The semantic segmentation sample raw file is as follows:
| item(s) | Format | Number of | Remarks for note |
| Image forming method | IMG | 1 | Mosaic raster data set |
| Vector labeling | SHP | 1 | |
| Metadata | XML | 1 |
The target recognition task is similar to the semantic segmentation task in that objects are labeled and specific classification information of the labeled objects is recorded, but the target recognition label is a circumscribed rectangular frame of the objects instead of the outline. The semantic segmentation sample of the remote sensing image consists of a raster data set, vector labels and a metadata file.
The target identification sample raw files are as follows:
| item(s) | Format | Number of | Remarks to note |
| Image forming method | IMG | 1 | Mosaic raster data set |
| Vector labeling | SHP | 1 | |
| Metadata | XML | 1 |
The remote sensing image change detection is to analyze the change of the earth surface and the ground features by using remote sensing images covering the same earth surface area and other auxiliary data acquired in multiple time phases, can determine the change of the ground features or phenomena in a certain time interval, and provides qualitative and quantitative information of the spatial distribution of the ground features and the change of the ground features. It requires determining the type of terrain, boundaries and analyzing the attributes of the changes before and after the change.
The change detection sample raw files are as follows:
| item(s) | Format | Number of | Remarks for note |
| Anterior phase image | IMG | 1 | Tessellating a raster dataset for anterior phase |
| Posterior phase image | IMG | 1 | Tessellating a trellis data set for a posterior temporal phase |
| Vector labeling | SHP | 1 | |
| Metadata | XML | 1 |
In the embodiment of the invention, as for the deep learning model, the sample remote sensing image and the vector file sample cannot be directly input into the model for training, and need to be processed into the tile image and the corresponding label.
Therefore, if the sample remote sensing image is a remote sensing image for performing semantic segmentation, step S106 includes the following steps:
s11, determining whether an overlapping area exists between the sample remote sensing image and the vector marking data;
step S12, if the tile data exists, respectively segmenting the sample remote sensing image and the vector annotation data according to a preset size and a preset overlap size to obtain first image tile data of the sample remote sensing image and first vector tile data of the vector annotation data, wherein the preset overlap size is the size of an overlap area between any two adjacent tiles;
step S13, determining first target image tile data in the image tile data and first target vector tile data in the vector tile data, wherein the first target image tile data and the first target vector tile data are coincident with each other;
step S14, constructing a first tile data set based on the first target image tile data;
step S15, generating a first tile label sample picture based on the target vector tile data, and determining the first tile label sample picture as a first label file, wherein the pixel values of background pixels and various ground object pixels in the first tile label sample picture are different.
In the embodiment of the invention, firstly, the sample remote sensing image and the vector marking data are input, and whether the spatial ranges of the sample remote sensing image and the vector marking data are overlapped or not is judged. And (4) starting to manufacture the tile sample required by the model when the sample remote sensing image and the vector marking data have a coincidence range, or else, jumping out of the program.
The tile with the size of s x s is read from the sample remote sensing image and the vector marking data to obtain first image tile data and first vector tile data, whether the first image tile data and the first vector tile data have coincident data or not is judged, and therefore first target image tile data and first target vector tile data are determined. By default, when the tile image and the tile vector do not coincide, the current tile will not be saved. Tiles with images that do not coincide with vectors will not be saved to reduce unnecessary sample generation. The default parameters may be adjusted if tile image samples that do not intersect the vector range need to be retained.
Meanwhile, when the tile sample is cut, the condition that the marked object is cut can be met, and negative influence can be caused on the model training precision. To mitigate this effect, the present invention introduces a cutting mechanism with overlapping regions. A sample pixel size is designated as s (i.e., a preset size) and an overlap region size is designated as b (i.e., a preset overlap size). When the sample is cut, the tile samples overlap each other. When the target ground object is cut apart, the overlapping area of another tile sample can cover the features containing the target ground object which is not cut apart, so that the target ground object can have more complete features to participate in model training.
According to the shape of each target ground object in the vector file, a tile label sample picture (namely, a first label file) of a single channel is generated, wherein the background pixel value in the tile label picture is 0, and the building pixel value is 1. In the multi-classification task, the background pixel value is 0, and the pixel values of different classes are sequentially increased from 1.
In this embodiment of the present invention, if the sample remote sensing image is a remote sensing image for target identification, step S106 includes the following steps:
step S21, determining whether an overlapping area exists between the sample remote sensing image and the vector marking data;
step S22, if the tile data exists, the sample remote sensing image and the vector marking data are respectively segmented according to a preset size and a preset overlap size to obtain second image tile data of the sample remote sensing image and second vector tile data of the vector marking data, wherein the preset overlap size is the size of an overlap area between any two adjacent tiles;
step S23, determining second target image tile data in the image tile data and second target vector tile data in the vector tile data, wherein the second target image tile data and the second target vector tile data are overlapped with each other;
step S24, constructing a second tile data set based on the second target image tile data;
step S25, generating a text file based on the circumscribed rectangle of each target ground object in the second target vector tile data, and determining the text file as a second label file, wherein the text file comprises the position information and the category information of the target ground object.
In the embodiment of the present invention, the processes from step S21 to step S24 are the same, and are not described herein again.
The manufacturing process of the second label file will be described in detail below.
And generating a text document in the txt format to record the position and the category information of the target ground object. And generating a line of text for each target, and respectively recording the category id of the target, the abscissa X of the central point of the target, the ordinate Y of the central point of the target, the width ratio W of the target and the height ratio H of the target. Where the class id is incremented starting with 1. The coordinates and dimensions of the object are represented by a cartesian coordinate system. The coordinates of the upper left corner and the lower right corner of each tile image are (0, 0) and (1, 1). The horizontal and vertical coordinates of the central point of the target represent the relative position of the target in the tile image, and the value range is 0-1 interval. The target width ratio is represented by the ratio of the target width to the tile width, the range of values is 0-1, and the target height ratio is the same. In order to obtain X, Y, W, H in a cartesian coordinate system, the projection coordinate system information in the vector file needs to be converted according to the affine transformation information of the image. The affine information of the image comprises a projection abscissa x0 of the upper left corner of the image map Line rotation, pixel width w width Projection ordinate y0 of the upper left corner map Column rotation, pixel height y height . Vertex coordinates x of the circumscribed rectangle for each target in the vector file map ,y map The formula is as follows:
obtaining Cartesian coordinates of the upper left corner and the lower right corner of each target circumscribed rectangle:
the upper left corner: ltx pixel ,lty pixel ；
Lower right corner: rdx pixel ,rdy pixel ；
Then from the upper left cartesian coordinates x0, y0 and tile size s of the current tile:
in this embodiment of the present invention, if the sample remote sensing image is a remote sensing image for change detection, the sample remote sensing image includes a time phase image and a time phase image, and step S106 includes the following steps:
step S31, determining whether an overlapping area exists between the sample remote sensing image and the vector marking data;
step S32, if the tile data exists, respectively segmenting the sample remote sensing image and the vector annotation data according to a preset size and a preset overlap size to obtain third image tile data of the sample remote sensing image and third vector tile data of the vector annotation data, wherein the preset overlap size is the size of an overlap area between any two adjacent tiles;
step S33, determining third target image tile data in the image tile data and third target vector tile data in the vector tile data, wherein the third target image tile data and the third target vector tile data coincide with each other;
step S34, constructing a third tile data set based on the third target image tile data;
step S35, generating a second tile label sample picture based on the third target vector tile data, and determining the tile label sample picture as a second label file, wherein the pixel values of background pixels and various types of ground object pixels in the second tile label sample picture are different.
In the embodiment of the present invention, steps S31 to S35 are substantially the same, and the difference is only that the sample remote sensing image includes a front time phase image and a time phase image, and the third tile data set includes tile data corresponding to the front time phase image and the time phase image, and the rest of processes are not described again.
In an embodiment of the present invention, the method further includes the steps of:
respectively inputting the sample remote sensing images into corresponding deep learning models to obtain vector results of the deep learning models, wherein the deep learning models comprise: a semantic segmentation model, a target identification model and a change detection model;
and determining an accuracy evaluation result based on the vector result of each deep learning model, the first target vector tile data, the second target vector tile data and the third target vector tile data, wherein the accuracy evaluation result is used for representing the performance of each deep learning model.
In the embodiment of the present invention, the following describes the inference flow of the semantic segmentation model, the target recognition model, and the change detection model.
A sample remote sensing image for semantic segmentation is input into a semantic segmentation model, tiles with the size of s x s are read from the image every time, A1, A2 and A3 8230are obtained, and the rest is done in the same way. The specified overlap area size between each tile is b.
The image tiles A1, A2 and A3 \8230areinput into a semantic segmentation model and subjected to model reasoning, and a single-channel prediction result with the same size s x s is obtained after each image tile is subjected to reasoning.
For each predicted result tile with size s x s, cutting out the frame with size b, reserving the area with the size (s-2 b) x (s-2 b) in the center, namely a1, a2, a3 \8230, and splicing the areas into a complete large graph to obtain a result grid. The result is spliced in the sliding window mode, so that the problem that edge cutting traces are obvious in a semantic segmentation result can be effectively solved.
And carrying out grid vectorization on the result grid, wherein different values in the grid correspond to different categories, and a vector result is obtained.
And inputting the sample remote sensing image for target identification into the target identification model, and reading the tile with the size of s x s from the image each time.
And inputting the image tiles into a target detection model, performing model reasoning, and obtaining a result of each target in the current tile after reasoning for each image tile. Each result records a target type id, a target central point abscissa X, a target central point ordinate Y, a target width ratio W and a target height ratio H. Simultaneously recording the Cartesian coordinates (x) of the upper left corner of the current tile in the whole image 0 ,y 0 )。
Calculating four vertex Cartesian coordinates of each target, specifically:
coordinates of the upper left corner:
ltx pixel ＝X*s-0.5*W+x 0 ；
lty pixel ＝Y*s-0.5*H+y 0 ；
lower left corner coordinates:
ldx pixel ＝ltx pixel ；
ldy pixel ＝lty pixel +H；
coordinates of the upper right corner:
rtx pixel ＝ltx pexel +W；
rty pixel ＝lty pixel ；
lower right corner coordinates:
rdx pixel ＝ltx pixel +W；
rdy pixel ＝lty pixel +H；
and generating a vector result file according to the Cartesian coordinates of the four vertexes of each target and the affine transformation information of the input image. Wherein the affine transformation information of the image comprises a projection abscissa x0 of the upper left corner of the image map Line rotation, pixel width x width Projection ordinate y0 of the upper left corner map Column rotation, pixel height y height . The projected coordinates of each coordinate point of each target are:
x map ＝x0 map +x pixel *x width ；
y map ＝y0 map +y pixel *y width ；
the reasoning process of the change detection model is similar to that of the semantic recognition model, but the input image is in two stages. And similarly, a sliding window mechanism is used for image cropping and splicing, and finally, a vector result file is output.
And finally, matching and comparing the vector result of each deep learning model with corresponding target vector tile data, counting the condition that whether each pixel is predicted correctly or not, and calculating a confusion matrix to obtain TP, FP, FN and TN.
Calculating precision: k is the number of classes
Accuracy = (TP + TN)/(TP + TN + FP + FN);
precision = TP/(TP + FP);
recall = TP/(TP + FN);
F1＝2*(Precision*Recall)/(Precision+Recall)；
cross-over ratio IoU = TP/(TP + FP + FN);
average cross-over ratio
And recording the precision numerical value in a txt file to form a precision evaluation result.
The invention provides a sample library construction method suitable for a multi-application scene deep learning algorithm, which can store and generate samples with unified standards, and can enhance data through an overlap region cutting mechanism, so that the characteristic loss caused during sample cutting is reduced, a model reasoning result can be converted into a result file required by production, and the method is suitable for domestic and foreign mainstream satellite images and unmanned aerial vehicle images, and can provide a basic support effect for the application of remote sensing in the field of natural resources.
The sample library establishing method eliminates a sample storage and generation mode of semantic segmentation, target identification and change detection, unifies sample standards, saves the repetitive development work and time required by sample generation and result file conversion, performs data enhancement through an overlapping region cutting and splicing mechanism, and improves model precision.
Example two:
the embodiment of the present invention further provides a natural resource sample library establishing apparatus, where the natural resource sample library establishing apparatus is configured to execute the natural resource sample library establishing method provided in the foregoing content of the embodiment of the present invention, and the following is a detailed description of the apparatus provided in the embodiment of the present invention.
As shown in fig. 2, fig. 2 is a schematic diagram of the natural resource sample library creating apparatus, where the natural resource sample library creating apparatus includes:
the acquiring unit 10 is configured to acquire a sample remote sensing image, where the sample remote sensing image includes: the remote sensing image is used for performing semantic segmentation, target recognition and change detection;
the interpretation unit 20 is used for carrying out artificial visual interpretation on the sample remote sensing image to obtain a target sample remote sensing image;
the dividing unit 30 is configured to perform tile division on the target sample remote sensing image by using the vector labeling data of the sample remote sensing image to obtain a tile data set, and configure a label file for the tile data in the tile data set;
and the constructing unit 40 is configured to construct a natural resource sample library based on the tile data set and the tag file.
In the embodiment of the present invention, a sample remote sensing image is obtained, where the sample remote sensing image includes: the remote sensing image is used for performing semantic segmentation, target recognition and change detection; carrying out artificial visual interpretation on the sample remote sensing image to obtain a target sample remote sensing image; performing tile segmentation on the target sample remote sensing image by using vector labeling data of the sample remote sensing image to obtain a tile data set, and configuring a label file for tile data in the tile data set; based on the tile data set and the label file, a natural resource sample library is constructed, the purpose of providing training samples with the same standard for various deep learning models is achieved, and the technical problem that samples in different batches are often processed and converted in different modes to be input into the deep learning models for training, so that the training efficiency of the deep learning models is low is solved, and the technical effect of improving the training efficiency of the deep learning models is achieved.
Example three:
an embodiment of the present invention further provides an electronic device, which includes a memory and a processor, where the memory is used to store a program that supports the processor to execute the method in the first embodiment, and the processor is configured to execute the program stored in the memory.
Referring to fig. 3, an embodiment of the present invention further provides an electronic device 100, including: the device comprises a processor 50, a memory 51, a bus 52 and a communication interface 53, wherein the processor 50, the communication interface 53 and the memory 51 are connected through the bus 52; the processor 50 is arranged to execute executable modules, such as computer programs, stored in the memory 51.
The Memory 51 may include a high-speed Random Access Memory (RAM) and may also include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The communication connection between the network element of the system and at least one other network element is realized through at least one communication interface 53 (which may be wired or wireless), and the internet, a wide area network, a local network, a metropolitan area network, and the like may be used.
The bus 52 may be an ISA bus, PCI bus, EISA bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one double-headed arrow is shown in FIG. 3, but this does not indicate only one bus or one type of bus.
The memory 51 is used for storing a program, the processor 50 executes the program after receiving an execution instruction, and the method executed by the apparatus defined by the flow process disclosed in any of the foregoing embodiments of the present invention may be applied to the processor 50, or implemented by the processor 50.
The processor 50 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuits of hardware or instructions in the form of software in the processor 50. The Processor 50 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in the memory 51, and the processor 50 reads the information in the memory 51 and completes the steps of the method in combination with the hardware thereof.
Example four:
the embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program performs the steps of the method in the first embodiment.
In addition, in the description of the embodiments of the present invention, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art.
In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the device or element being referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the following descriptions are only illustrative and not restrictive, and that the scope of the present invention is not limited to the above embodiments: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A natural resource sample library establishing method is characterized by comprising the following steps:
obtaining a sample remote sensing image, wherein the sample remote sensing image comprises: the remote sensing image is used for performing semantic segmentation, target recognition and change detection;
carrying out artificial visual interpretation on the sample remote sensing image to obtain a target sample remote sensing image;
carrying out tile segmentation on the target sample remote sensing image by using vector labeling data of the sample remote sensing image to obtain a tile data set, and configuring a label file for tile data in the tile data set;
and constructing a natural resource sample library based on the tile data set and the label file.
2. The method of claim 1, wherein if the sample remote sensing image is a remote sensing image for semantic segmentation, tile segmentation is performed on the target sample remote sensing image by using vector labeling data of the sample remote sensing image to obtain a tile dataset, and a label file is configured for tile data in the tile dataset, including:
determining whether an overlapping area exists between the sample remote sensing image and the vector marking data;
if the tile data exists, the sample remote sensing image and the vector marking data are respectively segmented according to a preset size and a preset overlap size to obtain first image tile data of the sample remote sensing image and first vector tile data of the vector marking data, wherein the preset overlap size is the size of an overlap area between any two adjacent tiles;
determining first target image tile data in the image tile data and first target vector tile data in the vector tile data, wherein the first target image tile data and the first target vector tile data coincide with each other;
constructing a first tile data set based on the first target image tile data;
and generating a first tile label sample picture based on the target vector tile data, and determining the first tile label sample picture as a first label file, wherein the pixel values of background pixels and various ground object pixels in the first tile label sample picture are different.
3. The method according to claim 2, wherein if the sample remote sensing image is a remote sensing image for target identification, tile segmentation is performed on the target sample remote sensing image by using vector labeling data of the sample remote sensing image to obtain a tile data set, and a label file is configured for tile data in the tile data set, including:
determining whether an overlapping area exists between the sample remote sensing image and the vector marking data;
if the tile data exists, the sample remote sensing image and the vector marking data are respectively segmented according to a preset size and a preset overlap size to obtain second image tile data of the sample remote sensing image and second vector tile data of the vector marking data, wherein the preset overlap size is the size of an overlap area between any two adjacent tiles;
determining second target image tile data in the image tile data and second target vector tile data in the vector tile data, wherein the second target image tile data and the second target vector tile data coincide with each other;
constructing a second tile data set based on the second target image tile data;
and generating a text file based on the circumscribed rectangle of each target ground object in the second target vector tile data, and determining the text file as a second label file, wherein the text file comprises the position information and the category information of the target ground object.
4. The method according to claim 3, wherein if the sample remote sensing image is a remote sensing image for change detection, the sample remote sensing image comprises a front time phase image and a time phase image;
utilizing the vector labeling data of the sample remote sensing image to perform tile segmentation on the target sample remote sensing image to obtain a tile data set, and configuring a label file for the tile data in the tile data set, wherein the tile segmentation comprises the following steps:
determining whether an overlapping area exists between the sample remote sensing image and the vector marking data;
if the tile data exists, the sample remote sensing image and the vector marking data are respectively segmented according to a preset size and a preset overlap size to obtain third image tile data of the sample remote sensing image and third vector tile data of the vector marking data, wherein the preset overlap size is the size of an overlap area between any two adjacent tiles;
determining third target image tile data in the image tile data and third target vector tile data in the vector tile data, wherein the third target image tile data and the third target vector tile data coincide with each other;
constructing a third tile data set based on the third target image tile data;
and generating a second tile label sample picture based on the third target vector tile data, and determining the tile label sample picture as a second label file, wherein the pixel values of background pixels and various ground object pixels in the second tile label sample picture are different.
5. The method of claim 4, further comprising:
inputting the sample remote sensing images into corresponding deep learning models respectively to obtain vector results of the deep learning models, wherein the deep learning models comprise: a semantic segmentation model, a target recognition model and a change detection model;
and determining a precision evaluation result based on the vector result of each deep learning model, the first target vector tile data, the second target vector tile data and the third target vector tile data, wherein the precision evaluation result is used for representing the performance of each deep learning model.
6. A natural resource sample library creation apparatus, comprising:
the acquisition unit is used for acquiring a sample remote sensing image, wherein the sample remote sensing image comprises: the remote sensing image is used for performing semantic segmentation, target recognition and change detection;
the interpretation unit is used for carrying out manual visual interpretation on the sample remote sensing image to obtain a target sample remote sensing image;
the segmentation unit is used for performing tile segmentation on the target sample remote sensing image by using the vector labeling data of the sample remote sensing image to obtain a tile data set, and configuring a label file for the tile data in the tile data set;
and the construction unit is used for constructing a natural resource sample library based on the tile data set and the label file.
7. The apparatus according to claim 6, wherein if the sample remote sensing image is a remote sensing image for semantic segmentation, the segmentation unit is configured to:
determining whether an overlapping area exists between the sample remote sensing image and the vector marking data;
if the tile data exists, the sample remote sensing image and the vector marking data are respectively segmented according to a preset size and a preset overlap size to obtain first image tile data of the sample remote sensing image and first vector tile data of the vector marking data, wherein the preset overlap size is the size of an overlap area between any two adjacent tiles;
determining first target image tile data in the image tile data and first target vector tile data in the vector tile data, wherein the first target image tile data and the first target vector tile data are mutually overlapped;
constructing a first tile data set based on the first target image tile data;
and generating a first tile label sample picture based on the target vector tile data, and determining the first tile label sample picture as a first label file, wherein the pixel values of background pixels and various ground object pixels in the first tile label sample picture are different.
8. The apparatus according to claim 7, wherein if the sample remote sensing image is a remote sensing image for object recognition, the segmentation unit is configured to:
determining whether an overlapping area exists between the sample remote sensing image and the vector marking data;
if the tile data exists, the sample remote sensing image and the vector marking data are respectively segmented according to a preset size and a preset overlap size to obtain second image tile data of the sample remote sensing image and second vector tile data of the vector marking data, wherein the preset overlap size is the size of an overlap area between any two adjacent tiles;
determining second target image tile data in the image tile data and second target vector tile data in the vector tile data, wherein the second target image tile data and the second target vector tile data coincide with each other;
constructing a second tile data set based on the second target image tile data;
generating a text file based on the circumscribed rectangle of each target ground object in the second target vector tile data, and determining the text file as a second label file, wherein the text file comprises the position information and the category information of the target ground object.
9. An electronic device comprising a memory for storing a program that enables a processor to perform the method of any of claims 1 to 5 and a processor configured to execute the program stored in the memory.
10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of the claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211410724.4A CN115909059A (en) | 2022-11-11 | 2022-11-11 | Natural resource sample library establishing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211410724.4A CN115909059A (en) | 2022-11-11 | 2022-11-11 | Natural resource sample library establishing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115909059A true CN115909059A (en) | 2023-04-04 |
Family
ID=86470504
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211410724.4A Pending CN115909059A (en) | 2022-11-11 | 2022-11-11 | Natural resource sample library establishing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115909059A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116434009A (en) * | 2023-04-19 | 2023-07-14 | 应急管理部国家减灾中心(应急管理部卫星减灾应用中心) | Construction method and system for deep learning sample set of damaged building |
| CN117218485A (en) * | 2023-09-05 | 2023-12-12 | 安徽省第二测绘院 | Deep learning model-based multi-source remote sensing image interpretation sample library creation method |
| CN117349462A (en) * | 2023-12-06 | 2024-01-05 | 自然资源陕西省卫星应用技术中心 | Remote sensing intelligent interpretation sample data set generation method |
| CN117392486A (en) * | 2023-12-12 | 2024-01-12 | 湖北珞珈实验室 | Method, device, equipment and storage medium for constructing natural resource element sample library |
| CN117788982A (en) * | 2024-02-26 | 2024-03-29 | 中国铁路设计集团有限公司 | Large-scale deep learning data set manufacturing method based on railway engineering topography result |
-
2022
- 2022-11-11 CN CN202211410724.4A patent/CN115909059A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116434009A (en) * | 2023-04-19 | 2023-07-14 | 应急管理部国家减灾中心(应急管理部卫星减灾应用中心) | Construction method and system for deep learning sample set of damaged building |
| CN116434009B (en) * | 2023-04-19 | 2023-10-24 | 应急管理部国家减灾中心(应急管理部卫星减灾应用中心) | Construction method and system for deep learning sample set of damaged building |
| CN117218485A (en) * | 2023-09-05 | 2023-12-12 | 安徽省第二测绘院 | Deep learning model-based multi-source remote sensing image interpretation sample library creation method |
| CN117349462A (en) * | 2023-12-06 | 2024-01-05 | 自然资源陕西省卫星应用技术中心 | Remote sensing intelligent interpretation sample data set generation method |
| CN117349462B (en) * | 2023-12-06 | 2024-03-12 | 自然资源陕西省卫星应用技术中心 | Remote sensing intelligent interpretation sample data set generation method |
| CN117392486A (en) * | 2023-12-12 | 2024-01-12 | 湖北珞珈实验室 | Method, device, equipment and storage medium for constructing natural resource element sample library |
| CN117788982A (en) * | 2024-02-26 | 2024-03-29 | 中国铁路设计集团有限公司 | Large-scale deep learning data set manufacturing method based on railway engineering topography result |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN115909059A (en) | Natural resource sample library establishing method and device | |
| CN110348294B (en) | Method and device for positioning chart in PDF document and computer equipment | |
| EP3806064B1 (en) | Method and apparatus for detecting parking space usage condition, electronic device, and storage medium | |
| Wang et al. | Photovoltaic panel extraction from very high-resolution aerial imagery using region–line primitive association analysis and template matching | |
| CN109977997B (en) | Image target detection and segmentation method based on convolutional neural network rapid robustness | |
| CN110443212B (en) | Positive sample acquisition method, device, equipment and storage medium for target detection | |
| CN113298169A (en) | Convolutional neural network-based rotating target detection method and device | |
| CN113238578B (en) | Routing planning method and system for power tower unmanned aerial vehicle inspection route | |
| CN114972191A (en) | Method and device for detecting farmland change | |
| CN112419202B (en) | Automatic wild animal image recognition system based on big data and deep learning | |
| CN113033516A (en) | Object identification statistical method and device, electronic equipment and storage medium | |
| CN110910445B (en) | Object size detection method, device, detection equipment and storage medium | |
| WO2021248269A1 (en) | High-precision identification method and system for transformer device | |
| CN116168351B (en) | Inspection method and device for power equipment | |
| CN111091101A (en) | High-precision pedestrian detection method, system and device based on one-step method | |
| Geng et al. | An improved helmet detection method for YOLOv3 on an unbalanced dataset | |
| CN113221947A (en) | Industrial quality inspection method and system based on image recognition technology | |
| CN112102296A (en) | Power equipment target identification method based on human concept | |
| CN111931721B (en) | Method and device for detecting color and number of annual inspection label and electronic equipment | |
| CN110298302A (en) | A kind of human body target detection method and relevant device | |
| CN114067339A (en) | Image recognition method and device, electronic equipment and computer readable storage medium | |
| CN111079752A (en) | Method and device for identifying circuit breaker in infrared image and readable storage medium | |
| CN116246161A (en) | Method and device for identifying target fine type of remote sensing image under guidance of domain knowledge | |
| Zhang et al. | A YOLOv3-Based Industrial Instrument Classification and Reading Recognition Method | |
| CN114943720A (en) | Electric power image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |