CN109657728B

CN109657728B - Sample production method and model training method

Info

Publication number: CN109657728B
Application number: CN201811598543.2A
Authority: CN
Inventors: 刘昱君; 李明巨; 石善球; 王丹; 许磊磊; 张璐; 李福洪; 朱映
Original assignee: PROVINCIAL GEOMATICS CENTRE OF JIANGSU
Current assignee: PROVINCIAL GEOMATICS CENTRE OF JIANGSU
Priority date: 2018-12-26
Filing date: 2018-12-26
Publication date: 2021-03-30
Anticipated expiration: 2038-12-26
Also published as: CN109657728A

Abstract

The embodiment of the invention provides a sample production method and a model training method, wherein the sample production method comprises the following steps: acquiring identification codes of feature elements to be processed from candidate vector data matched with the image data, wherein the candidate vector data records identification codes of various feature elements and feature boundary information corresponding to the identification codes; preprocessing the ground feature boundary information corresponding to the identification code according to a first preset condition to obtain target element data; and cutting the image data by using the target element data to obtain a plurality of target samples, wherein the target samples are used for constructing a training model in a deep learning algorithm. By the method, the target samples can be obtained by using the vector data to be selected matched and associated with the image data, and the problems of few samples and low sample production efficiency in the prior art are solved.

Description

Sample production method and model training method

Technical Field

The invention relates to the field of data processing, in particular to a sample production method and a model training method.

Background

Since the application of deep learning technology in speech recognition, image processing and the like, the field of remote sensing image interpretation also starts to utilize the deep learning technology to participate in part of the interpretation process. However, the current remote sensing image sample data has two problems: firstly, the sample types are few, and the data volume is insufficient; secondly, the samples need to be manually selected and manually marked, and the workload is large, so that the production efficiency of the samples is low. The two problems restrict the development of the deep learning technology in the field of remote sensing image interpretation.

Disclosure of Invention

In order to overcome the problems in the prior art, the embodiments of the present invention provide a sample production method and a model training method.

In a first aspect, an embodiment of the present invention provides a sample production method, where the method includes:

acquiring identification codes of feature elements to be processed from candidate vector data matched with the image data, wherein the candidate vector data records identification codes of various feature elements and feature boundary information corresponding to the identification codes;

preprocessing the ground feature boundary information corresponding to the identification code according to a first preset condition to obtain target element data;

and cutting the image data by using the target element data to obtain a plurality of target samples, wherein the target samples are used for constructing a training model in a deep learning algorithm.

By the method, the vector data to be selected which is matched and associated with the image data can be used as a data source for producing the sample, and elements represented by certain identification codes in the vector data to be selected are further processed, for example, surface feature boundary information of certain elements is preprocessed to obtain target element data which meets the production requirements of the sample, wherein different preprocessing can be performed on the elements in the vector data to be selected corresponding to different target sample requirements. And then, cutting the image data according to the target element data to obtain a plurality of target samples, wherein the target samples can be related to the same feature element or various feature elements. The method can be used for cutting the image data in batches, and the sample production efficiency is improved. The target samples can be used as training samples in the deep learning process and can be used for constructing a training model in a deep learning algorithm.

With reference to the first aspect, in a possible design, the preprocessing the feature boundary information corresponding to the identification code according to a first preset condition to obtain target element data includes:

acquiring feature boundary information corresponding to the identification code, wherein the feature boundary information comprises a feature area;

and filtering the feature elements represented by the identification codes according to the feature areas to obtain target element data.

By the method, before the single target identification sample and the single element segmentation sample are produced, the elements which cannot meet the production requirement of the sample are filtered, and the data of the target elements which meet the production requirement of the sample are obtained. The sample production requirement may be the floor space of a feature, or the pixel amount of a feature in the image data. For example, it may be directly determined whether the area of the feature is smaller than a threshold, and if so, deleting the element with the area smaller than the threshold to filter the first element to obtain the target element data; or calculating the pixel quantity of the corresponding element in the image data according to the acquired area of the ground object, comparing the calculated pixel quantity with a pixel threshold value, and filtering out the element of which the pixel quantity is lower than the pixel threshold value.

With reference to the first aspect, in one possible design, the target sample includes a single-element segmentation sample, where the single-element segmentation sample includes a first reference sample and a first reference label, and the segmenting the image data by using the target element data to obtain a plurality of target samples includes:

acquiring surface feature boundary information of each element in the target element data to obtain an envelope rectangle of each element;

cutting the image data according to the envelope rectangle to extract pixels cut by the envelope rectangle from the image data, and performing monochrome filling on a non-target element part in the envelope rectangle to obtain the first reference sample;

and performing monochrome filling on a target element part in the envelope rectangle to obtain the first reference label, wherein the filling color of the target element part is different from that of the non-target element part.

The method can produce single element segmentation samples of specified types, and the elements corresponding to the specified types can be elements appearing in pieces such as farmlands, dry lands, vegetation and the like. These elements have small pitch to element.

With reference to the first aspect, in one possible design, the target sample includes a single-element segmentation sample, the single-element segmentation sample includes a second reference sample and a second reference label, and the segmenting the image data by using the target element data to obtain a plurality of target samples includes:

determining a cutting rectangle of each element according to the feature boundary information of each element in the target element data;

cutting the image data by using the cutting rectangle to extract pixels cut by the cutting rectangle from the image data to obtain the second reference sample;

and filling the target element part and the non-target element part in the same cutting rectangle with two colors in a single color respectively to obtain the second reference label, wherein the identification code of the target element part is different from that of the non-target element part.

The clipping rectangle may be an envelope rectangle determined by the boundary of each element, or may be a clipping grid determined by the total boundary of all elements and the average area of a single element. Whether the envelope rectangle or the clipping mesh is the clipping rectangle, the image data may be clipped as the clipping rectangle, and a single-element segmentation sample may be further generated. By the method, different types of single-element segmentation samples can be produced according to specific requirements. Each single element segmentation sample comprises a reference sample and a reference label associated with the reference sample. And the production process of each single-element segmentation sample can be realized in batch, and a plurality of single-element segmentation samples can be obtained at one time. If the single-element segmentation sample is used as training data to obtain a training model, a user can be helped to quickly distinguish the single elements and identify the outline of the independent elements.

With reference to the first aspect, in a possible design, the target sample includes a multi-element segmentation sample, and the preprocessing the feature boundary information corresponding to the identification code according to a first preset condition to obtain target element data includes:

and rendering each surface feature element recorded in the vector data to be selected by using a preset color configuration table by taking each identification code in the vector data to be selected as an element distinguishing mark to obtain target element data.

The method can lay a foundation for producing the multi-element segmentation sample, and particularly lay a foundation for obtaining the label in the multi-element segmentation sample. By rendering each surface feature element, a plurality of elements can be directly distinguished by colors.

With reference to the first aspect, in one possible design, the multi-element segmentation sample includes a multi-element segmentation reference sample and a multi-element segmentation reference tag, and the obtaining a plurality of target samples by segmenting the image data with the target element data includes:

determining a standard grid according to the feature boundary information of each element in the target element data;

cutting the image data by using the standard grid to obtain the multi-element segmentation reference sample;

and rasterizing a plurality of elements in the standard grid to obtain the multi-element division reference label.

By the method, a large number of multi-element segmentation samples can be obtained, and the reliability of the multi-element segmentation samples is high. If the multi-element segmentation sample is used as training data to obtain a training model, a user can be helped to quickly distinguish a plurality of elements and identify the outlines of the elements.

obtaining a buffer interval;

and performing area buffer updating on the feature boundary corresponding to the identification code according to the buffer interval to obtain target element data, wherein updated feature boundary information is recorded in the target element data.

By the method, some elements which do not meet the production requirement of the sample can be preprocessed, and the elements which do not meet the production requirement of the sample can be updated in a buffering way, so that new element boundaries of the elements can be obtained. In practice, these factors that do not meet the production requirements of the sample may be factors that result in low pixel count in the image data due to too small a footprint; it is also possible that the information is insufficient at the time of initial recording (possibly described only in dots, lines), making it difficult to determine the elements of the actual area.

With reference to the first aspect, in one possible design, the manner of obtaining the buffer interval includes:

responding to a buffering instruction input by a user, obtaining a buffering range in the buffering instruction, and taking the buffering range as a buffering interval.

The buffer range in the buffer command may be a buffer interval for a single element or a batch buffer interval for a certain type of element.

With reference to the first aspect, in a possible design, the obtaining the buffer interval further includes: and taking a preset buffer range as a buffer interval of batch buffering.

With reference to the first aspect, in one possible design, the target samples include single target identification samples, and the obtaining multiple target samples by segmenting the image data by using the target element data includes:

and cutting the image data by using the boundary in the updated ground feature boundary information to obtain the single-target identification sample.

By the method, a plurality of single target identification samples can be obtained, and if the single target identification samples are used as training data to obtain a training model, target classification or target identification can be realized in the remote sensing image interpretation process.

With reference to the first aspect, in one possible design, before the segmenting the image data by using the target element data to obtain a plurality of target samples, the method further includes:

and setting a region attribute for each element in the target element data.

By the method, the regional information and the element information can be associated to produce the target sample with the regional attribute.

With reference to the first aspect, in one possible design, the method further includes:

and recording the region attribute of the target sample, the element category associated with the target sample and the picture pixel attribute.

By the method, the subsequent management query is facilitated after the relevant attributes of the target sample are recorded, and the target sample is conveniently maintained.

In a second aspect, an embodiment of the present invention further provides a model training method, where the method includes:

obtaining a plurality of target samples obtained by the sample production method of the first aspect;

and inputting the target samples as training data into a model to be trained for training to obtain an interpretation model.

The interpretation model obtained by the method has higher reliability and wider application range, and solves the problem that the deep learning technology is difficult to develop due to the lack of training samples in the field of remote sensing image interpretation.

In a third aspect, an embodiment of the present invention further provides a sample production apparatus, where the apparatus includes:

the system comprises a data source acquisition module, a processing module and a processing module, wherein the data source acquisition module is used for acquiring identification codes of feature elements to be processed from vector data to be selected related to image data, and the vector data to be selected records identification codes of a plurality of feature elements and feature boundary information corresponding to the identification codes;

the preprocessing module is used for preprocessing the feature boundary information corresponding to the identification code according to a first preset condition to obtain target element data;

and the generating module is used for cutting the image data by using the target element data to obtain a plurality of target samples, wherein the target samples are used for constructing a training model in a deep learning algorithm.

In a fourth aspect, an embodiment of the present invention further provides an electronic device, including: a processor, a memory and a bus, the memory storing machine-readable instructions executable by the processor, the processor and the memory communicating via the bus when the electronic device is operating, the machine-readable instructions when executed by the processor performing the steps of the method of the first aspect.

In a fifth aspect, the present invention provides a readable storage medium, on which a computer program is stored, and the computer program runs the steps in the method according to the first aspect when being executed by a processor.

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.

Fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Fig. 2 is a flowchart of a sample production method according to an embodiment of the present invention.

Fig. 3 is a diagram of a paddy field sample in one example provided by the embodiment of the invention.

Fig. 4 is a diagram of a single element division sample for paddy field according to another embodiment of the present invention.

Fig. 5 is an illustration of a sample of single-element segmentation about a pit in an example provided by an embodiment of the present invention.

Fig. 6 is an illustration of a single-element division sample of a parking lot according to an embodiment of the present invention.

Fig. 7 is a diagram of a multi-element segmentation sample in an example provided by an embodiment of the present invention.

Fig. 8 is a functional block diagram of a sample production apparatus according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures. Meanwhile, in the description of the present invention, the terms "first", "second", and the like are used only for distinguishing the description, and are not to be construed as indicating or implying relative importance.

The applicant finds that most of the current remote sensing image sample data sets come from necessary maps and Google map images, sample categories are few, manual selection and manual marking are mostly adopted, workload is high, the mode of producing samples is only to roughly frame ground features to achieve marking, and the application direction of deep learning in a remote sensing interpretation model is limited to a target classification and target detection level by the rough label samples.

Therefore, the applicant designs a set of sample production modes aiming at the problems that the source data of the existing deep learning-oriented learning sample in the remote sensing interpretation field is few and the sample production is difficult, fully analyzes the existing basic mapping data, the acquisition standard of the geographical national condition general survey and monitoring data and the acquisition mode of the geographical national condition general survey data, and designs a method capable of producing reliable samples in batches according to different application directions of the deep learning in the remote sensing interpretation field and various requirements on the samples.

Because the basic mapping data of each period are obtained according to the mapping requirements strictly, the monitoring data of the geographical national situation census is obtained through a complete quality control system, and the two kinds of data are reliable enough and have higher authenticity. In the actual data acquisition process, the remote sensing image data is acquired by basic mapping data and geographical national situation census monitoring data of each period through aviation/satellite photography, image acquisition elements are interpreted by professional field personnel, a series of basic requirements on data acquisition are formulated, and finally a database is established through field inspection and internal compilation. The remote sensing image data corresponding to each period can be subjected to preliminary data processing and on-site painting, and finally quality inspection to obtain geographical national condition vector data and basic mapping vector data. The production process of each stage of basic surveying and mapping data and the geographic national condition census monitoring number is basically the same as the process of manual production of samples, a complete quality control system is provided, particularly, earth surface coverage data in the geographic national condition census covers each region, different types of surface features and even different types of surface features of the same type of surface features have clear outline boundaries with high precision, each pixel in each stage of remote sensing image is distributed to a certain object class, semantic segmentation in deep learning is that pixel-level classification needs to be carried out on targets, the precision of the two types of data for collecting the surface features based on the pixel level can meet the sample requirement of example segmentation, and the data precision of the geographic national condition vector data and the basic surveying and mapping vector data can be guaranteed.

After the applicant researches and discovers the above content, the geographical national situation vector data matched and associated with the geographical national situation census data (monitoring, surface coverage) and the basic mapping vector data matched and associated with the basic mapping data are obtained, and the sample production method of the application is designed.

Some terms in the embodiments of the present invention will be explained below.

Geographical national conditions vector data: vector data obtained by the general survey data of the geographical national conditions can describe earth surface coverage data and monitoring data. In the geographic national situation and surface coverage data, the data are mainly collected based on surface elements. The surface coverage classification information reflects the natural surface of the surface, the natural attributes or conditions of natural creatures. The geographical national situation vector data can be obtained by aerial remote sensing images (aerial photographs) or satellite remote sensing images (satellite photographs). The image resolution of the aerial photo can be 0.3 meter and 0.5 meter, and the image resolution of the satellite photo can be 1 meter.

Basic mapping vector data: the vector data obtained by the basic mapping data can describe natural and artificial ground features with distinct features in the mapping data. The basic mapping data comprises 4D data products such as DLG, DOM, DEM, DRG and the like. The basic mapping vector data can be obtained by aerial remote sensing images (aerial photographs) or by satellite remote sensing images (satellite photographs). The image resolution of the aerial photo can be 0.3 meter and 0.5 meter, and the image resolution of the satellite photo can be 1 meter. It should be noted that the following embodiments refer to the case where the base mapping data is produced by using DLG type samples, and in other possible embodiments, those skilled in the art may produce samples using other types of base mapping vector data (e.g., DOM, DEM, DRG, etc.).

Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the features of the embodiments can be combined with each other without conflict.

Fig. 1 is a block diagram of an electronic device 100 according to an embodiment of the present invention. The electronic device 100 may be a server, a Personal Computer (PC), a tablet PC, a smart phone, a Personal Digital Assistant (PDA), or the like.

The electronic device 100 comprises a processor 110, a memory 120 and a bus, the memory 120 storing machine-readable instructions executable by the processor 110, the processor 110 and the memory 120 communicating via the bus when the electronic device 100 is running, the machine-readable instructions when executed by the processor 110 performing the steps of the sample production method as follows.

In a specific implementation process, in order to perform communication connection with other terminal devices, the electronic device 100 may further include components such as a communication interface, a communication and network expansion card, and the like; in order to display each stage result in the sample production method or provide an interactive interface for the user, the electronic device 100 may further include a display unit; the electronic device 100 may further include an input and output unit for receiving data input by a user or for outputting data required by the user, which will not be described herein.

First embodiment

Please refer to fig. 2, which is a flowchart illustrating a sample manufacturing method according to an embodiment of the present invention. Since the present application relates to the production of multiple samples, in one embodiment, multiple sample requirements can be obtained in advance through research, and a corresponding sample is produced for each sample; in another embodiment, after the user inputs the sample requirement, the sample requirement of the user can be read and recognized, which sample or samples need to be generated is further determined, and then the appropriate sample is generated.

The specific process shown in FIG. 2, including S21-S23, is described in detail below.

And S21, acquiring the identification codes of the feature elements to be processed from the vector data to be selected matched with the image data, wherein the vector data to be selected records the identification codes of various feature elements and feature boundary information corresponding to the identification codes.

And S22, preprocessing the feature boundary information corresponding to the identification code according to a first preset condition to obtain target element data. In order to meet different target sample requirements as much as possible, different pre-processing can be performed on elements in the vector data to be selected.

And S23, cutting the image data by using the target element data to obtain a plurality of target samples, wherein the target samples are used for constructing a training model in the deep learning algorithm.

In S21, the Image data represents Remote Sensing Image (RS) data, which is films or photos recording electromagnetic waves of various features, and is mainly classified into aerial photos and satellite photos, which are referred to as aerial photos and satellite photos, respectively.

The candidate vector data matched and associated with the image data can be geographical national condition vector data or basic mapping vector data. No matter which kind of candidate vector data is matched and associated with the remote sensing image. By using the two types of candidate vector data as data sources for producing the samples, the samples with higher accuracy can be obtained. The skilled person in the art can select the vector data to be selected as the data source according to actual needs, and further produce a plurality of target samples based on the selected data source, and the produced target samples can be applied to element segmentation, and also can be applied to element classification or identification.

In one example, the candidate vector data is geographic national conditions vector data. The vector data to be selected may be presented in a table form, where the identification code of the feature element is used as a primary key, each line of data represents an element, for example, one line of vector data represents a surface element, if one line of vector data is selected, it is equivalent to selecting one element in one map layer in a summarized map set, the selected element has a closed boundary, and an area within the boundary is represented as an area value in the vector data to be selected.

Similarly, each row of data in the candidate vector data may also represent a point element or a line element, for example, when the candidate vector data is the basic mapping vector data, the associated data corresponding to each identification code may represent a point or a line in the basic mapping data.

The feature boundary information in the candidate vector data may include other boundary information of the element corresponding to the identification code, such as coordinates, in addition to the area value. However, these coordinates can only show absolute coordinate information such as longitude, latitude, and altitude, and if a specific position attribute of an element is to be obtained, an administrative area attribute needs to be additionally set.

After acquiring some or all of the identification codes in the candidate vector data, the feature elements corresponding to the identification codes may be preprocessed, and step S22 is executed.

Regarding S22, different rules are set as the first preset conditions according to different sample requirements. The first preset condition defines processing items required to process the elements, and the processing items comprise: minimum pixel amount of boundary area/element required for producing the sample, region attribute setting, color setting, boundary buffering and other processing items. After the first preset condition is obtained, corresponding processing items are further executed according to the first preset condition so as to preprocess the surface feature elements corresponding to the identification codes, and target element data can be obtained after preprocessing.

In S23, after the target element data is obtained, the image data may be further cut in batch based on the element boundaries in the target element data to obtain a plurality of target samples, and these target samples may be used as a training set to construct an image interpretation training model. If the vector data to be selected is the geographical national situation vector data, the remote sensing image data related to the geographical national situation vector data is cut, and if the vector data to be selected is the basic mapping vector data, the remote sensing image data related to the basic mapping vector data is cut.

Because the existing remote sensing image samples are few, and the existing samples are manually selected and marked, the workload is large, errors are easy to occur, the generated samples are rough, and the application direction of the samples is limited. By the method in the embodiment, a large number of target samples can be automatically generated. Firstly, vector data to be selected, which mainly comprises geographical national condition vector data and is assisted by basic mapping vector data, is matched and associated with remote sensing image data, elements in the vector data to be selected are preprocessed, and then the remote sensing image data is cut by utilizing the preprocessed target element data, so that a plurality of target samples can be produced in batches, and the obtained samples are more, reliable in data and high in precision, and can meet the application requirements of deep learning.

Secondly, according to different application directions of deep learning in the remote sensing interpretation field, a plurality of processing items are set in the first preset condition to preprocess vector data to be selected, and further, a plurality of target samples are generated according to preprocessed target element data so as to meet different application requirements.

Therefore, the method takes the earth surface coverage data as the main part, supplements natural and artificial ground features with distinct characteristics in the basic mapping data, automatically produces label samples applied to different directions for the remote sensing image data corresponding to each period, and solves the problem of insufficient samples in the field of remote sensing interpretation of the deep learning technology.

In order to meet different application directions of the deep learning technology in the field of remote sensing interpretation, three target samples are set in the embodiment, including a single-element segmentation sample, a multi-element segmentation sample and a single target identification sample.

The single target recognition samples can be used for element recognition or classification, and the training model generated by taking a plurality of single target recognition samples as training data can recognize the element types in the image and judge the ground feature of a certain element.

The single element segmentation example may be used to perform single element segmentation, and a training model generated using a plurality of single element segmentation examples as training data may segment a single element in a certain image, for example, an outline that can represent a specified element in a certain figure is identified, so as to distinguish the specified element from other parts in the image, and obtain the outline of the single element. The single-element segmentation sample further comprises a plurality of subdivision samples, and each single-element segmentation sample comprises a reference sample obtained by segmenting image data and a reference label obtained according to vector data.

Similarly, the multi-element segmentation sample can be used for multi-element segmentation, and the training model generated by using the multi-element segmentation samples as training data can segment the plurality of elements in a certain image, for example, an outline capable of representing a designated element in a certain figure is marked, so that the plurality of designated elements in the same image are distinguished to obtain the outlines of the plurality of elements. Each multi-element segmentation sample comprises a reference sample obtained by segmenting image data and a reference label obtained according to vector data.

In the process of generating the target sample, since some elements may have too small actual occupied area and the pixel quantity of the elements in the image data is insufficient, the image data can be cut after the vector data to be selected is filtered. In order to implement this function, the feature boundary information in S21 may include a feature area, and S22 of the method may include the following sub-steps: S221-S222.

S221: and acquiring the feature boundary information corresponding to the identification code, wherein the feature boundary information comprises a feature area. For example, the same type of elements can be extracted according to the identification code, and then the area of the feature corresponding to each element can be acquired; and the identification code and the ground feature area can be simultaneously acquired directly according to the vector data to be selected.

S222: and filtering the feature elements represented by the identification codes according to the feature areas to obtain target element data. As an embodiment, it may be directly determined whether the area of the feature is smaller than the area threshold, and if so, the element having the area smaller than the area threshold is deleted to filter the first element to obtain the target element data. As another embodiment, the pixel amount of the element in the image data may be calculated according to the acquired feature area, and it is determined whether the calculated pixel amount is lower than a pixel threshold, and if so, the element having the pixel amount lower than the pixel threshold may be deleted to filter the first element to obtain the target element data. A person skilled in the art may set a corresponding area threshold/pixel threshold for each element in the vector data to be selected according to actual needs, and the specific setting manner and specific value of the area threshold and the pixel threshold should not be construed as a limitation to the present application.

In one example, the geographical national condition vector data is used as the candidate vector data, the first elements are land and object elements such as planting land, forest and grass, farmland dry land, vegetation and the like, and the single-element segmentation sample and the single-target identification sample can be produced by the method. The feature of these feature elements is that they are usually in the form of pieces, and the spacing between the feature elements of the same kind is small, and the area of some feature elements is small.

If the image cutting is directly performed on all the elements corresponding to the identification codes obtained from the candidate vector data, which may result in insufficient pixels of some of the cut images, the texture features of some of the elements cannot be completely reflected (for example, the texture features of the elements in the paddy field cannot be obtained due to insufficient pixel quantity in the paddy field in fig. 3), and the produced sample does not have the distinct features of the elements. In order to solve such a problem, in the present embodiment, qualitative and quantitative analysis is performed on each feature, and the lowest pixel amount of different features, that is, the lower limit of the pixel amount reflecting the main feature of the feature, is determined and is reflected in the vector data as the area size of the feature. The target sample is reproduced after filtering out an area of elements.

It should be noted that some dot/line elements may have a small pixel amount originally, so as to avoid misoperation, if it is recognized that the elements represented by the identification code are dot and line elements, another processing method is available, that is, the elements are subjected to boundary expansion or buffering, so that the pixel amount of some dot/line elements meets the sample production requirement. For example, elements such as towers, gates, archways, chimneys, satellite stations, astronomical benches, tunnels, etc. may be represented by only point elements or line elements in the candidate vector data. For vector data in which only one element is represented by a point or a line, the element may be subjected to boundary expansion or buffering so that the boundary of the element is updated in a buffering manner, and the element after the boundary update may be processed, for example, by dividing the image data by the updated boundary. To implement this function, S22 may further include: and acquiring a buffer interval, and performing area buffer updating on the feature boundary corresponding to the identification code according to the buffer interval to obtain target element data, wherein updated feature boundary information is recorded in the target element data.

There are two ways to obtain the buffer interval: firstly, responding to a buffering instruction input by a user, obtaining a buffering range in the buffering instruction, wherein the buffering range in the buffering instruction can be a buffering interval aiming at a certain single element or a batch buffering interval aiming at a certain type of element; and secondly, acquiring a preset buffering range, and taking the preset buffering range as a buffering interval for carrying out batch buffering on element boundaries.

It should be noted that, the order of the filtering and buffer updating processes should not be construed as limiting the present invention. In the process of producing a single target identification sample, the target sample may be produced by using the point and line elements in the basic mapping vector data, and therefore, it is necessary to perform boundary buffering on the point and line elements. Whether filtering or buffer updating is actually needed can be set according to the requirements of users. In an embodiment, after receiving the user input or the selected element type and the sample requirement, a first preset condition meeting the user requirement may be generated, or first preset conditions corresponding to different requirements may be obtained according to the user requirement obtained through research in advance.

The specific generation processes of the three target examples will be described below.

First, a single element split sample. Because the difference between different types of elements is large, two types of subdivision examples are set for the single-element segmentation example, the first type of subdivision example includes a first reference example and a first reference label, and the second type of subdivision example includes a second reference example and a second reference label. There are two different ways of handling the second type of subdivision sample. In other words, each subdivision sample includes a reference sample and a reference label. In this embodiment, color filling is performed for any one of the reference labels, so that each reference label contains a positive label and a negative label.

In one embodiment of producing the single element split sample, the single element split sample includes a first reference sample, a first reference label. The above S23 includes the following substeps: s231a-S233 a.

S231 a: and acquiring the feature boundary information of each element in the target element data to obtain an envelope rectangle of each element. As an embodiment, the element boundary of each element in the feature boundary information may be obtained first. And similar to the data representation method in the candidate vector data, each surface feature element in the target element data is subjected to element distinguishing by using the identification code. In one example, the identification code is a code, the identification code of the paddy field is "0110", and all elements of the paddy field in the target element data can be extracted by the identification code "0110", and the element boundary of each paddy field is obtained. Four-to-four information of the element boundary can be determined based on the acquired element boundary, and the minimum circumscribed rectangle, namely the envelope rectangle, of the target element can be determined through the four-to-four information.

Alternatively, before S232a (and even before S231 a), a region attribute may be added to each element in the vector data to be selected or the target element data. As an embodiment, the region attribute space in the administrative division data may be associated with each element in the candidate vector data or associated with each element in the target element data. For example, it is only necessary to add a region attribute to the determined envelope rectangle.

S232 a: and cutting the image data according to the envelope rectangle to extract pixels cut by the envelope rectangle from the image data, and performing monochrome filling on a non-target element part in the envelope rectangle to obtain the first reference sample.

S233 a: and performing monochrome filling on a target element part in the envelope rectangle to obtain the first reference label, wherein the filling color of the target element part is different from that of the non-target element part. It should be noted that the identification codes corresponding to the target element part and the non-target element part are different.

Wherein, the left part in fig. 4 may represent a first reference example with respect to a paddy field in one example, and the right part in fig. 4 may represent a first reference label with respect to a paddy field in one example.

It should be noted that the sequence between S232a and S233a should not be construed as limiting the present invention, and in the actual implementation process, S232a and S233a may be executed simultaneously or step by step, for example, S233a may be executed first, and then S232a may be executed. In executing S233a, the target element portion and the non-target element portion in the envelope rectangle may be filled in monochrome with two colors, respectively.

In one example, the color filled for the non-target element portion in the envelope rectangle is black. The target element part and the non-target element part in the envelope rectangle are respectively used as positive and negative examples (or positive and negative labels) of the first reference label. Of course, in other examples, other colors may be used as long as they can be effectively distinguished from the target element portion in the envelope rectangle.

In the process of acquiring the general survey data of the geographical national conditions, the main aim is to construct a geographical national conditions database with strong current performance and high precision and full coverage, to classify all the ground features in the involved range in a full coverage mode, and not to qualitatively analyze the specific conditions of elements in the images (for example, the paddy field in the following figure 3, and the figure 3 is a schematic diagram which does not meet the requirements of the sample). If the example shown in fig. 3 is obtained, although the outline of the elements of the paddy field is obtained, the texture features of the paddy field cannot be reflected, and therefore, the first reference example is designed, the target element part is clearly distinguished from the non-target element part, and the texture features of the elements are reserved in the first reference example.

By the method, the image data can be cut in batch by utilizing the boundary information of the single elements, so that a plurality of first reference samples and a plurality of first reference labels associated with the plurality of first reference samples are obtained. The obtained single-element segmentation sample is more real and reliable. The method is favorable for producing target samples of elements such as paddy fields, dry lands and the like which need to keep the texture characteristics of the ground features.

In another embodiment of producing the single element split sample, the single element split sample includes a second reference sample, a second reference label. The above S23 includes the following substeps: s231b-S233 b.

S231 b: and determining the cutting rectangle of each element according to the feature boundary information of each element in the target element data.

S232 b: and cutting the image data by using the cutting rectangle so as to extract pixels cut by the cutting rectangle from the image data to obtain the second reference sample.

S233 b: and filling the target element part and the non-target element part in the same cutting rectangle with two colors in a single color respectively to obtain the second reference label, wherein the identification code of the target element part is different from that of the non-target element part.

It should be noted that the sequence between S232b and S233b should not be construed as limiting the present invention, and in the actual implementation process, S232b and S233b may be executed simultaneously or step by step, for example, S233b may be executed first, and then S232b may be executed.

Optionally, the clipping rectangle is an envelope rectangle or a clipping grid.

In one case, the clipping rectangle is a clipping mesh, and S231b includes: acquiring surface feature boundary information of each element in the target element data to obtain the total boundary of all elements and the average area of single elements; and determining the peripheral frame of the cutting grid according to the total boundary, and determining the size of the single grid in the cutting grid according to the average area of the single element. The above method can more flexibly determine the boundary of the clipping grid and the size of a single grid in the clipping grid.

Wherein the grid lines in the cutting grid are used for cutting off the elements. For example, for an element such as a river, after the identification codes of all the elements of the river are extracted, all the river elements may be extracted into the same layer, the average area of a single river element is calculated, the size of a single grid in the clipping grid is determined according to the average area, and the clipping grid is divided according to the total boundary of all the rivers. In one example, if the average area of the elements is 12000 square meters, the size of a single grid can be determined to be 110 m × 110 m in a manner of taking the square nearest, and if the resolution of the image is 1 m, the size of the single grid can also be expressed as 110 pixel × 110 pixel. It should be noted that the above description merely provides one way to determine a clipping grid and should not be construed as limiting the present application.

After the clipping mesh is set, a region attribute may be added to each mesh in the clipping mesh. As an embodiment, the grid data and the preset administrative division data may be subjected to superposition analysis.

Accordingly, S232b includes: and cutting the image data by utilizing the cutting grid to extract partial pixels in the image data to obtain a second reference sample.

Accordingly, S233b includes: and filling the target element part and the non-target element part in the same grid with two colors in a single color respectively to obtain a second reference label, wherein the identification code of the target element part is different from that of the non-target element part. In one example, the target element portion may be white and the non-target element portion may be black for the same grid.

In one example, since the elements such as the canal, the road, the railway, etc. have different shapes and sizes and large area differences, and the elements such as the canal, the road, the railway, etc. are mostly long and slender, if the sample is directly cut according to the upper, lower, left and right boundary ranges of the elements, the size of the obtained sample is large, and the proportion of the element ground object in the sample picture is small, so that the significance and the value of the sample as the elements are lost. In the example, the local features of the elements and the integrity of the elements are considered to be embodied as much as possible, the size of the cutting grid is made according to the average area of the same elements, the elements are broken by the boundaries (grid lines) of the cutting grid, corresponding image pixels are extracted through the grid boundaries, and the samples are produced in batches. And selecting target elements in the grid according to positions, dividing the area of the elements in the grid by the area of the grid to obtain an area ratio, comparing the area ratio with a preset value, and selecting or rejecting the elements with the small area ratio to ensure the quality of the target sample and produce the label samples (a second reference sample and a second reference label). In one example, a single-element division sample for a pit as shown in fig. 5 can be obtained, where the left part in fig. 5 represents a second reference sample and the right part in fig. 5 represents a second reference label.

By the method, the target samples can be produced by the elongated elements such as canals, roads and railways with different shapes and sizes and large area difference, the image data is cut by the grids, the target element parts and the non-target element parts in the grids are further subjected to color filling, the integrity of the elements is kept, and the obtained target samples are more real and reliable.

In another case, the clipping rectangle is an envelope rectangle, and the specific implementation process of S231b is similar to the process of S231a, and please refer to the foregoing description for the description of the element boundary, the envelope rectangle, and so on, and will not be described again here. Optionally, after determining the envelope rectangle, a region attribute may be added to the envelope rectangle.

Accordingly, if the clipping rectangle is an envelope rectangle, S232b may include: and cutting the image data by using the envelope rectangle to extract partial pixels in the image data to obtain a second reference sample.

Accordingly, S233b may include: in the enveloping rectangle, the element boundary is used as a boundary line, and the inside and the outside of the element boundary are filled with single colors respectively by two colors to obtain a second reference label. In the same enveloping rectangle, the inside of the element boundary represents the target element portion, and the outside of the element boundary represents the non-target element portion. In one example, the target component portion may be whitened, the non-target component portion may be blacked, and the second reference label may be rasterized. Wherein, the envelope rectangle can be determined according to the minimum and maximum values in the two-dimensional coordinates of each element.

By the method, the target sample can be produced for the elements with large spacing span of parking lots, airports and airport runways, the integrity of the sample can be ensured, the element characteristics can be shown, in one example, a single-element segmentation sample related to the parking lots can be obtained as shown in fig. 6, the left part in fig. 6 represents a second reference sample, and the right part in fig. 6 represents a second reference label.

Second, a multi-element division example. In one example, the vector data to be selected is collected based on the surface elements, collected different types of ground objects and even different types of ground objects of the same type have clear outline boundaries, the precision is high, and each pixel in the remote sensing image in the remote sensing shooting period is assigned to a certain object class. The semantic segmentation of deep learning is to classify the target by pixels, and the precision of collecting ground features at the pixel level can meet the sample requirement of the semantic segmentation of deep learning.

For the multi-element division sample, S22 may include: and rendering each surface feature element recorded in the vector data to be selected by using a preset color configuration table by taking each identification code in the vector data to be selected as an element distinguishing mark to obtain target element data. For example, if the geographic national condition vector data is used as the candidate vector data, the unique value rendering color can be performed on each feature element in the geographic national condition vector data.

Different RGB colors may be defined for different types of feature elements, and different color may be used to represent different feature elements to form a color arrangement table. And performing color distribution on the surface feature elements recorded in the vector data to be selected by utilizing a color configuration table to realize rendering.

After the rendering is finished, the rendered elements may be subjected to global rasterization and a cutting step may be further performed, or the cutting step may be performed first and then the content obtained by cutting may be subjected to global or local rasterization.

The multi-element division sample comprises a multi-element division reference sample and a multi-element division reference label. In order to obtain the multi-element division reference sample and the multi-element division reference tag, the step S23 may include the following sub-steps: s231c-S233 c.

S231 c: and determining a standard grid according to the feature boundary information of each element in the target element data.

For example, feature boundary information of each element in the target element data may be obtained first, and a total boundary of all elements is obtained; and determining the peripheral frame of the standard grid according to the total boundary. In one example, the size of a single grid in a standard grid may be 1024 pixels by 1024 pixels.

S232 c: and cutting the image data by using the standard grid to obtain the multi-element segmentation reference sample.

S233 c: and rasterizing a plurality of elements in the standard grid to obtain the multi-element division reference label.

It should be noted that the sequence between S232c and S233c should not be construed as limiting the present invention.

By the method, a large number of multi-element segmentation samples can be obtained, the obtained multi-element segmentation samples are high in precision, each pixel can be allocated to the object type of the element, and the precision of the ground feature elements acquired according to the pixel level can meet the requirement of deep learning semantic segmentation. The multi-element segmentation sample subjected to rasterization processing is high in reliability, visual feeling can be given to people through colors, the discrimination is high, and machine readability is high. In one example, a multi-element division sample as shown in fig. 7 may be obtained, where the left part in fig. 7 represents a multi-element division reference sample and the right part in fig. 7 represents a multi-element division reference label.

Third, single target recognition example. In order to obtain the single target recognition sample, the step S23 may include segmenting the image data by using the boundary in the updated feature boundary information to obtain the single target recognition sample.

The updated feature boundary information refers to the buffered or updated element boundary.

The single-target identification sample as the classification sample or the identification sample does not need to reach the pixel level in production precision, and the single-target identification sample only needs to fully embody the basic characteristics of the ground objects of the corresponding category and can be distinguished from other parts. Based on this requirement, the main data sources for producing the single-target identification examples are point/line elements in the basic mapping vector data and face elements of high-rise buildings in the geographic national conditions vector data. These features are characterized by small footprint and small pixels in the image. The elements may only have information of one point element and one line element in the candidate vector data, and the boundary area of the elements is small or even equal to none. In this case, it is necessary to buffer or update the boundaries of the point and line elements so that the updated element boundaries can cover the main features of the elements. And further cutting the image data by using the updated element boundary to obtain a single-target identification sample.

Wherein, the buffer mode for the point/line element can be realized as follows: and the point/line elements are subjected to buffer configuration according to the actual remote sensing image until the main characteristics of the elements can be contained. And executing the step S23 after the buffering is finished, obtaining a new peripheral rectangular outline by using the updated ground feature boundary information, filling the new peripheral rectangular outline with water, and cutting to produce the single-target identification sample.

When the single-target identification sample is produced for the surface elements of the high-rise buildings and the like, due to the fact that side viewing angles exist during shooting, ground objects (high-rise buildings) with certain heights generate displacement differences on images, the elements are difficult to match with image real objects in a matching mode, boundary buffering can be conducted on the surface elements according to a preset batch buffering interval, then new boundaries obtained after buffering are used for cutting image data, and the single-target identification sample is obtained.

In this embodiment, optionally before S23, the method further includes: a region attribute is set for each element in the target element data.

In one embodiment, the region attribute may be set directly in a range defined by an element boundary of the element. In a specific implementation, the target element data recorded with the plurality of target elements and the preset administrative division data may be used for performing overlay analysis to add regional attributes to the plurality of target elements in the target element data in batches.

As another embodiment, a region attribute may be set to an envelope rectangle of an element. The region attribute can be added by performing superposition analysis by using a plurality of envelope rectangles covering the target element and preset administrative division data to add the region attribute to the element in the envelope rectangle.

As still another embodiment, the region attribute may be set to the set standard grid. Only the standard grid data and the preset administrative division data need to be subjected to superposition analysis. The target sample produced by cutting the image by using the standard grid added with the regional attribute naturally has the regional attribute.

Optionally, in this embodiment, the method further includes: and recording the region attribute, the element category and the picture pixel attribute of the target sample.

The subsequent management and query are facilitated by recording the contents of the region attribute, the element category, the picture pixel attribute, the image name and the like of the target sample.

In summary, by the above method, geographic national condition vector data or basic mapping vector data associated with image data in a matching manner can be used as vector data to be selected, target element data meeting sample production requirements can be obtained after elements in the vector data to be selected are preprocessed, then a rectangle (or a grid) used for cutting the image data is determined based on the target element data, and finally the image data is segmented to obtain a plurality of target samples with higher reliability, and a plurality of target samples applicable to different requirements can be obtained. The method solves the problem that in the prior art, due to the reasons of few samples, difficult sample production and the like, the application of the deep learning technology in the field of remote sensing image interpretation is limited, and gets rid of the mode that the samples can only be obtained manually in the past, thereby improving the sample production efficiency.

Second embodiment

The embodiment provides a model training method which comprises two links.

In the first step, a plurality of target samples obtained by the sample production method provided in the first embodiment are obtained.

And a second step of inputting the target samples as training data into a model to be trained for training to obtain an interpretation model. For other details regarding the target sample in this embodiment, reference is further made to the related description of the foregoing embodiments, which are not repeated herein.

By the method, the multiple target samples provided by the embodiment of the invention can be effectively utilized in the deep learning technology to obtain the interpretation model. The training data source is sufficient, so that the reliability of the obtained interpretation model is higher, and the application of deep learning in the field of remote sensing image interpretation is expanded.

Third embodiment

The present embodiment provides a sample production apparatus 300, as shown in fig. 8, the sample production apparatus 300 including:

the data source obtaining module 310 is configured to obtain identification codes of feature elements to be processed from candidate vector data matched with the image data, where the candidate vector data records identification codes of multiple feature elements and feature boundary information corresponding to the identification codes;

the preprocessing module 320 is used for preprocessing the feature boundary information corresponding to the identification code according to a first preset condition to obtain target element data;

the generating module 330 is configured to cut the image data by using the target element data to obtain a plurality of target examples, where the target examples are used to construct a training model in the deep learning algorithm.

For other details of the apparatus, reference is further made to the related description in the first embodiment, and details are not repeated here.

By the aid of the device, a large number of target samples can be obtained, and the problems that in the prior art, the number of samples is small and the production efficiency of the samples is low are solved.

In addition to the above embodiments, the present application provides a readable storage medium, on which a computer program is stored, and the computer program runs the steps in the sample production method provided in the above first embodiment when executed by a processor. The storage medium includes: various media that can store program codes, such as a U disk, a removable hard disk, a memory, a magnetic disk, or an optical disk.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It is to be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method of sample production, the method comprising:

cutting the image data by using the target element data to obtain a plurality of target samples, wherein the target samples are used for constructing a training model in a deep learning algorithm;

the target examples comprise single-element segmentation examples, the single-element segmentation examples comprise first reference examples and first reference labels, the image data is segmented by using the target element data to obtain a plurality of target examples, and the method comprises the following steps:

filling a target element part in the enveloping rectangle with a single color to obtain the first reference label, wherein the filling color of the target element part is different from that of the non-target element part;

or:

the target examples comprise single-element segmentation examples, the single-element segmentation examples comprise second reference examples and second reference labels, the image data is segmented by using the target element data to obtain a plurality of target examples, and the method comprises the following steps:

filling the target element part and the non-target element part in the same cutting rectangle with two colors in a single color respectively to obtain a second reference label, wherein the identification code of the target element part is different from that of the non-target element part;

or:

the target examples comprise multi-element division examples, the multi-element division examples comprise multi-element division reference examples and multi-element division reference labels, and the target element data is used for cutting the image data to obtain a plurality of target examples, and the method comprises the following steps:

2. The method of claim 1, wherein the preprocessing the feature boundary information corresponding to the identification code according to a first preset condition to obtain target element data comprises:

3. The method of claim 1, wherein the preprocessing the feature boundary information corresponding to the identification code according to a first preset condition to obtain target element data comprises:

4. The method of claim 1, wherein the preprocessing the feature boundary information corresponding to the identification code according to a first preset condition to obtain target element data comprises:

obtaining a buffer interval;

5. The method of claim 4, wherein the target samples comprise single target identification samples, and the segmenting the image data using the target element data to obtain a plurality of target samples comprises:

6. The method according to any one of claims 1-5, wherein before said segmenting said image data into a plurality of target samples using said target element data, said method further comprises:

and setting a region attribute for each element in the target element data.

7. A method of model training, the method comprising:

obtaining a plurality of target samples obtained by the sample production method according to any one of claims 1 to 6;