CN110728295A

CN110728295A - Semi-supervised landform classification model training and landform graph construction method

Info

Publication number: CN110728295A
Application number: CN201910821645.4A
Authority: CN
Inventors: 李坚强; 陈杰; 陈壮壮; 曾崛
Original assignee: Shenzhen Zhongke Baotai Technology Co Ltd
Current assignee: Shenzhen Zhongke Baotai Aerospace Technology Co ltd
Priority date: 2019-09-02
Filing date: 2019-09-02
Publication date: 2020-01-24
Anticipated expiration: 2039-09-02
Also published as: CN110728295B

Abstract

The embodiment of the application is suitable for the technical field of artificial intelligence, and discloses a semi-supervised landform classification model training and landform image construction method, wherein the method comprises the following steps: acquiring a landform sample data set after manual labeling and corresponding label information; performing countermeasure training on the generative countermeasure network by using the label information and the manually marked geomorphic sample data set to obtain target pseudo geomorphic data output by the generative countermeasure network; the target pseudo-geomorphic data is pseudo-geomorphic data which is output when the training effect of the generative countermeasure network reaches the optimum; training the pre-trained landform classification model by using the target pseudo-landform data; and after the training is finished, constructing a landform image by using the landform classification model. The embodiment of the application is based on the generation type confrontation network, a large amount of training data are generated by using a small amount of artificially labeled data, and therefore the acquisition time of the landform sample image is shortened.

Description

Semi-supervised landform classification model training and landform graph construction method

Technical Field

The application belongs to the technical field of artificial intelligence, and particularly relates to a semi-supervised landform classification model training and landform image construction method.

Background

A landscape map refers to a map representing various landscape landmarks. It can reflect the external morphological characteristics and causes, years, development processes, developmental levels and interrelationships of various landforms.

At present, when a convolutional neural network model is adopted to construct a landform image, a large amount of manually labeled landform sample images are required to train corresponding models, and a large amount of time is required to be spent for manually labeling the large amount of landform sample images.

Disclosure of Invention

The embodiment of the application provides a semi-supervised landform classification model training and landform image construction method, which aims to solve the problem that a large amount of time is required to collect training samples in the conventional landform image construction.

In a first aspect, an embodiment of the present application provides a semi-supervised geomorphic classification model training method, including:

acquiring a landform sample data set after manual labeling and corresponding label information;

performing countermeasure training on a generative countermeasure network by using the label information and the manually marked geomorphic sample data set to obtain target pseudo geomorphic data output by the generative countermeasure network; wherein, the target pseudo-geomorphic data is the pseudo-geomorphic data output when the training effect of the generating countermeasure network reaches the optimum;

and training the pre-trained landform classification model by using the target pseudo-landform data.

With reference to the first aspect, in a possible implementation manner, the generative confrontation network is a conditional generative confrontation network, and the tag information is geomorphic coarse-grained category information;

the performing countermeasure training on the generative countermeasure network by using the tag information and the manually labeled geomorphic sample data set to obtain target pseudo geomorphic data output by the generative countermeasure network, includes:

inputting random noise and the landform coarse-grained category information into a generator in the conditional generation countermeasure network, and acquiring pseudo-landform data output by the generator;

inputting the pseudo-landform data and the corresponding landform coarse-grained type information thereof, the real landform data and the corresponding landform coarse-grained type information thereof into a discriminator in the condition generating countermeasure network, wherein the discriminator is used for judging whether the real landform data and the pseudo-landform data are consistent; wherein, the real landform data is a sample image in the manually marked landform sample data set;

and obtaining a discrimination result output by the discriminator, and continuously and iteratively training the conditional generation type countermeasure network until the discrimination result reaches the optimum, wherein the pseudo-geomorphic data output by the generator is used as the target pseudo-geomorphic data.

With reference to the first aspect, in a possible implementation manner, the generative confrontation network is a fine-grained representation information conditional generative confrontation network, and the tag information includes landform coarse-grained type information and landform fine-grained type information;

inputting random noise, the landform coarse category information and the landform fine-grained category information into a generator in the fine-grained representation information condition generation type countermeasure network, and acquiring pseudo-landform data output by the generator;

inputting the pseudo-landform data and the corresponding landform coarse-grained type information thereof, the real landform data and the corresponding landform coarse-grained type information thereof to a discriminator in the fine-grained representation information condition generation type countermeasure network, and acquiring a discrimination result output by the discriminator; wherein, the real landform data is a sample image in the manually marked landform sample data set;

inputting the pseudo-geomorphic data into a fine-grained hidden coding network in the fine-grained representation information condition generation type countermeasure network, and acquiring fine-grained hidden codes output by the fine-grained hidden coding network;

and continuously training the fine-grained representation information condition generation type countermeasure network in an iterative manner until the judgment result reaches the optimum and the fine-grained hidden code reaches the preset effect, and taking the pseudo-geomorphic data as the target pseudo-geomorphic data.

With reference to the first aspect, in a possible implementation manner, an objective function formula of the fine-grained representation information condition generation type countermeasure network is as follows:

wherein c is the information of the coarse-grained type of the landform, s is the information of the fine-grained type of the landform, z is random noise, and the fine-grained representation information condition generation type confrontation network model is

The generator model is G (z, s | c), the discriminator model is D (x | c), Q is a fine-grained hidden coding network, I [ s; g (z, s | c)]And L is the lower bound of variation of the mutual information.

With reference to the first aspect, in a possible implementation manner, before the acquiring the manually labeled geomorphic sample data set and the corresponding tag information, the method further includes:

acquiring an unlabelled geomorphic sample data set;

based on the pre-trained landform classification model, selecting a target sample image from the unlabelled landform sample data set according to the information entropy of the sample image of each landform category;

after the target sample image is manually labeled, obtaining the manually labeled target sample image so as to obtain the manually labeled landform sample data set comprising the manually labeled target sample image.

With reference to the first aspect, in a possible implementation manner, before the obtaining an unlabelled geomorphic sample data set, the method further includes:

acquiring an acquired original landform image;

cutting the relief original image into image blocks to obtain the unlabeled relief sample data set comprising the image blocks.

With reference to the first aspect, in a possible implementation manner, the pre-training process of the geomorphic classification model includes:

selecting a training sample from the unlabelled landform sample data set;

and after the training samples are manually labeled, pre-training a landform classification model by using the manually labeled training samples.

In a second aspect, an embodiment of the present application provides a method for constructing a geomorphic graph, including:

acquiring a landform image to be processed;

obtaining a landform classification result of the landform image to be processed according to the trained landform classification model and the landform image to be processed; wherein, the landform classification model is a model obtained by training through the semi-supervised landform classification model training method of any one of the first aspect;

and constructing a landform graph according to the landform classification result.

With reference to the second aspect, in a possible implementation manner, the obtaining, according to the trained landform classification model and the to-be-processed landform image, a landform classification result of the to-be-processed landform image includes:

cutting the landform image to be processed into image blocks;

and inputting the image blocks into the landform classification model to obtain the landform classification result of each image block.

With reference to the second aspect, in a possible implementation manner, the constructing a landform map according to the landform classification result includes:

labeling each image block by using a preset label corresponding to the landform classification result according to the landform category of each image block to obtain a labeled image block;

and splicing the image blocks according to the positions of the image blocks in the landform image to be processed to construct the landform image.

In a third aspect, an embodiment of the present application provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor, when executing the computer program, implements the method according to any one of the first and/or second aspects.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the method according to any one of the first and/or second aspects.

In a fifth aspect, embodiments of the present application provide a computer program product, which, when run on a terminal device, causes the terminal device to perform the method of any one of the first and/or second aspects.

According to the embodiment of the application, the landform sample data set after manual labeling is input into the generative confrontation network for confrontation training, the target pseudo-landform data of the generative confrontation network when the training effect is optimal is obtained, and the target pseudo-landform data is used as the training data of the landform classification model, namely based on the generative confrontation network, a large amount of training data are generated by using a small amount of manually labeled data, so that the acquisition time of the landform sample image is shortened.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic block diagram of a flow of a semi-supervised geomorphic classification model training method according to an embodiment of the present application;

fig. 2 is a schematic block diagram of a process of generating a manually labeled geomorphic sample data set according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a process for generating manual annotation data according to an embodiment of the present application;

FIG. 4 is a schematic diagram of an active learning model provided in an embodiment of the present application;

FIG. 5 is a schematic block flow diagram illustrating a countermeasure training process for a conditional generation countermeasure network provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of a conditional generative confrontation network model framework provided by an embodiment of the present application;

fig. 7 is a schematic block diagram of a process of countermeasure training of a fine-grained representation information condition generating countermeasure network provided in an embodiment of the present application;

fig. 8 is a schematic diagram of a fine-grained representation information condition generating type confrontation network model framework provided in an embodiment of the present application;

FIG. 9 is a schematic block flow chart of a method for constructing a landscape architecture according to an embodiment of the present application;

FIG. 10 is another schematic block flow diagram of a method for constructing a landscape architecture provided by an embodiment of the present application;

fig. 11 is a block diagram illustrating a structure of a geomorphic classification model training apparatus according to an embodiment of the present application;

fig. 12 is a block diagram illustrating a structure of a landscape architecture constructing apparatus according to an embodiment of the present application;

fig. 13 is a schematic structural diagram of a terminal device according to an embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application.

The technical scheme provided by the embodiment of the application is applied to the construction of the landform map, and the process of constructing the landform map can be roughly divided into a stage of training a landform classification model and a stage of constructing the landform map.

In the training stage of the landform classification model, a large number of landform sample images are generated through a generating type countermeasure network based on a small number of sample images after manual labeling. Compared with the prior art that a large number of landform sample images are collected to train corresponding models, the embodiment of the application can greatly reduce the collection time of the sample images. In addition, the generative confrontation network can generate sample images of various landform categories, so that the subsequently constructed landform image has better effect.

After the training of the landform classification model is completed, the landform image to be processed can be input into the landform classification model to obtain a landform classification result of the image, and then the required landform image is constructed according to the landform classification result.

It is worth pointing out that the geomorphic classification model training and the geomorphic graph construction method provided by the embodiment of the application are semi-supervised, that is, a supervisor needs to be introduced in the geomorphic classification model training process. The semi-supervision is embodied in the generation process of manual marking data, and specifically comprises the following steps: according to the magnitude of the image information entropy, after a sample image with the largest uncertainty is selected from an unlabelled geomorphic sample data set through a pre-trained geomorphic classification model, the geomorphic type of the selected sample image is manually judged, and then manual labeling is carried out, so that a manually labeled geomorphic sample data set is formed. After obtaining the manually marked landform sample data set, performing countermeasure training by using the landform sample data set to generate a large amount of pseudo-landform data, wherein in the process of countermeasure training, the generated pseudo-landform data is provided with pseudo labels by specifying landform coarse-grained type information or the landform coarse-grained type information and the landform fine-grained type information; and then training the pre-trained landform classification model by using the generated pseudo-landform data.

The technical solutions provided in the embodiments of the present application will be described below with specific embodiments.

Referring to fig. 1, a schematic flow chart of a semi-supervised geomorphic classification model training method provided in an embodiment of the present application is shown, where the method may include the following steps:

and S101, acquiring a landform sample data set after manual labeling and corresponding label information.

It should be noted that the manually labeled geomorphic sample data set includes geomorphic sample images of multiple geomorphic categories, and each geomorphic category has multiple sample images. For example, each terrain category includes 10 sample images.

And the landform sample images in the manually marked landform sample data set are all manually marked images. After selecting a corresponding sample image from the unmarked landform sample image dataset, manually judging which landform type the selected landform sample image belongs to, and then manually marking according to the landform type. For example, if the selected geomorphic sample image has a geomorphic category belonging to grass, it is labeled as grass.

Each landform sample image in the manually marked landform sample data set has corresponding label information, and the label information is landform type label information, namely the label information can represent the belonged landform type of the corresponding landform sample image. For example, if the label information of a certain landform sample image is a cement land label, it indicates that the landform type of the landform sample image is a cement land.

It is worth pointing out that the geomorphic sample image in the geomorphic sample data set after the manual labeling may be the whole image or an image block. And cutting the whole landform sample image according to a preset size to obtain a plurality of image blocks with the same size.

And selecting a corresponding sample image from the un-labeled geomorphic sample data set by human, and then performing human labeling to obtain the geomorphic sample data set after the human labeling. And selecting a corresponding sample image, and then carrying out manual labeling, thereby obtaining the manually labeled landform sample data set. Compared with the prior art, if the number of unlabeled data is large, the efficiency and accuracy of artificially selecting the sample image are lower than those of selecting the sample image in an active learning mode.

In some embodiments, the required sample image can be automatically selected from the unlabeled geomorphic sample data set through the pre-trained geomorphic classification model according to the magnitude of the image information entropy. Referring to fig. 2, which is a schematic block diagram of a process of generating a manually labeled geomorphic sample data set, before the manually labeled geomorphic sample data set and corresponding label information are obtained, the model training method may further include:

step S201, obtaining a landform sample data set which is not marked.

It should be noted that the unlabelled geomorphic sample data set includes a plurality of unlabelled geomorphic sample images, and the geomorphic sample images may be image blocks, and the image blocks may be obtained by cutting original geomorphic sample images. And the original sample image of the landform can be an image of the landform acquired by the unmanned aerial vehicle. And the unlabelled geomorphic sample data set comprises image blocks of a plurality of geomorphic categories.

In some embodiments, before the obtaining the unlabeled geomorphic sample data set, the model training method may further include: acquiring an acquired original landform image; and cutting the original relief image into image blocks to obtain an unlabeled relief sample data set comprising the image blocks.

Wherein, above-mentioned landform original image can be the landform image that unmanned aerial vehicle gathered. Cutting the acquired original landform image according to a preset size to obtain a corresponding image block; and then forming the unmarked geomorphic sample data set based on the image blocks obtained by cutting. For example, a relief original image is cut into a plurality of 64 × 64 size image blocks according to 64 × 64 size.

And S202, based on the pre-trained landform classification model, selecting a target sample image from the unlabeled landform sample data set according to the information entropy of the sample image of each landform category.

It should be noted that the above-mentioned geomorphic classification model includes, but is not limited to, a convolution layer, a pooling layer, a Flatten layer, an activation function layer, a full-link layer, and a softmax function layer, and the link relationship and the corresponding function of each layer are well known to those skilled in the art and will not be described herein again.

The terrain classification model is pre-trained, wherein the terrain classification model may be considered pre-trained when iteratively trained a predetermined number of times. For example, when the iterative training times are more than 50 times, the pre-training of the landform classification model is considered to be completed. The pre-training refers to the preliminary training of the established landform classification model by using a small amount of manually marked landform sample images. The pre-training process specifically comprises the following steps: manually selecting a small number of sample images from the unlabelled landform sample image dataset, and manually labeling the selected landform sample images; and then training a landform classification model by using a small amount of manually labeled landform sample images. The above-mentioned "a small number of sample images" can be defined according to actual needs. In the present embodiment, "a small number of sample images" may mean that 10 pieces of relief sample image data are selected for each relief category.

In some embodiments, the pre-training process of the geomorphic classification model includes: selecting a training sample from the unlabelled landform sample data set; after the training samples are manually labeled, the pre-constructed landform classification model is pre-trained by using the manually labeled training samples.

Specifically, after a small number of landform sample images are manually selected, the landform sample images can be manually labeled according to the belonged landform categories, and then training of the landform classification model is performed by using the manually labeled landform sample images.

After the landform classification model is pre-trained, the required target sample image can be selected from an unlabeled landform sample data set by using the landform classification model. The target sample image is selected based on the information entropy size of the landform sample image of each landform category, and the target sample image comprises the landform sample images of various landform categories.

Specifically, a certain number of sample images are selected from the landform sample images of various landform categories respectively based on the information entropy size of the landform sample image of each landform category.

For example, the unlabeled geomorphic sample data set comprises sample images of three geomorphic categories of grassland, texture land and cement land. Firstly, calculating the information entropy of each image, then sequencing each type of landform sample images from large to small according to the size of the information entropy, and then respectively selecting the first 10 images from each type of sample images as target sample images. At this time, the target sample images include 10 lawn-category relief sample images, 10 texture-land-category relief sample images, and 10 cement-land-category relief sample images.

More specifically, the information entropy calculation formula is specifically:

H[y|x；w]＝-∑_cp(y＝c|x,w)log p(y＝c|x,w)

where c is a landform category of the sample image, which may be defined by p (y ═ c | x, w) ═ softmax (fw (x)), fw (x) is an output of the pre-trained landform classification model, and the training sample data is selected from the unlabeled landform sample data set U through the function a (x, w) given the parameter w.

And a (x, w) ═ H [ y | x, w]In the present embodiment, x may be passed^*And (4) selecting a target sample image from the unlabeled sample data set by x ∈ U (x, w).

And selecting a target sample image according to the size of the information entropy, and selecting the sample image which reduces the uncertainty of the current model to the maximum extent. In other words, the larger the information entropy, the larger the uncertainty of the geomorphic sample image, and selecting the geomorphic sample image from large to small according to the information entropy can reduce the uncertainty of the current model as much as possible.

Step S203, after the target sample image is manually labeled, obtaining the manually labeled target sample image so as to obtain the manually labeled geomorphic sample data set including the manually labeled target sample image.

Specifically, after a target sample image with the largest uncertainty is selected from an unlabelled geomorphic sample data set through a pre-trained geomorphic classification model, the category of the target sample image is artificially judged, and the target sample image is artificially labeled. Thus, the manually labeled landform sample data set consisting of the manually labeled target sample images can be obtained. The manually labeled landform sample data set comprises sample images of various landform categories and corresponding label information.

In order to better describe the generation process of the manually labeled geomorphic sample data set, the following description is made with reference to a schematic diagram of the generation process of the manually labeled data set shown in fig. 3.

As shown in fig. 3, the acquired picture is cut into image block data, and the picture can be a landform original image acquired by the unmanned aerial vehicle; and forming an unlabelled landform sample data set by the cut image block data, selecting image block data from the unlabelled landform sample data set, then carrying out artificial labeling according to the landform category to which the image block belongs, and generating a corresponding data set after artificial labeling. For example, if the landform type of the selected image block is a grassland, the image block is marked as the grassland, and finally, a plurality of pieces of grassland image block data form a grassland data set; if the landform type of the selected image block is a texture land, marking the selected image block as the texture land, and finally forming a texture land data set by a plurality of pieces of texture map block data; and marking the selected landform type of the image block as a cement land, and finally forming a cement land data set by a plurality of pieces of cement map block data. The labeled geomorphic sample data set also consists of image block data with the same size, specifically, image block data with the size of 64 × 64.

According to the magnitude of the information entropy, a target sample image is selected from an unlabelled geomorphic sample data set through a pre-trained geomorphic classification model, and then manual labeling is carried out to generate the artificially labeled geomorphic sample data set. The following will be described with reference to the schematic diagram of the active learning model shown in fig. 4.

As shown in fig. 4, the active learning model consists of C, Q, L, S, U five parts. Wherein, C refers to the landform classification model after the pre-training is finished; l represents a landform sample data set which is manually marked; u represents an unmarked landform sample data set; s represents a supervisor, and can mark the unmarked geomorphic sample data set; q represents a query function, and the geomorphic sample data can be selected from the un-labeled geomorphic sample data set U for labeling through the query function.

Therefore, the target sample image is selected from the unlabelled landform sample data set in an active learning mode, the diversity of generated data can be ensured, and the problem that the generated mode resists the limitation of a network under a small number of samples is solved.

S102, performing countermeasure training on the generative countermeasure network by using the label information and the manually labeled geomorphic sample data set to obtain target pseudo geomorphic data output by the generative countermeasure network; the target pseudo-geomorphic data is pseudo-geomorphic data which is output when the training effect of the generative countermeasure network reaches the optimum.

It is understood that the generative confrontation network may include a generator and a discriminator, the generator may generate corresponding pseudo-geomorphic data, and the discriminator may discriminate the difference between the input pseudo-geomorphic data and the real geomorphic data. When the discriminator cannot distinguish the pseudo-geomorphic data from the real geomorphic data, namely the generative confrontation network tends to converge, the training effect of the generative confrontation network can be considered to be optimal, at the moment, the pseudo-geomorphic data generated by the generator is used as target pseudo-geomorphic data, and the target pseudo-geomorphic data is used as training data of the geomorphic classification model. Therefore, a large amount of target pseudo-geomorphic data can be generated by using less manual marking data, and a large amount of geomorphic sample images do not need to be acquired in a large amount of time.

In some embodiments, when the generated countermeasure network is specifically a conditional generation countermeasure network, the tag information is geomorphic coarse-grained type information. Referring to the schematic flow diagram of the countermeasure training process shown in fig. 5, the specific process of performing countermeasure training on the generative countermeasure network by using the tag information and the manually labeled geomorphic sample data set to obtain the target pseudo geomorphic data output by the generative countermeasure network may include:

step S501, inputting random noise and the rough-granularity type information of the landform into a generator in the conditional generation countermeasure network, and acquiring the pseudo-landform data output by the generator.

It should be noted that the random noise may be gaussian random noise, and the landform coarse-grained type information refers to information representing a landform type to which the sample image belongs, where the landform type is a coarse-grained type. Coarse-grained categories include, but are not limited to, grass, texture, and cement. Corresponding to the coarse-grained category, the fine-grained category is a further representation of the coarse-grained category, e.g. the coarse-grained category is grass, and the corresponding fine-grained category comprises bald grass, sparse grass, dense grass, etc.

Specifically, Gaussian random noise and coarse-grained type label information are input into a generator, and after multilayer deconvolution processing is performed on the generator, pseudo-geomorphic data corresponding to the specified geomorphic coarse type are generated. For example, when the input label information is a grass label, the generator outputs a grass sample image, when the label information is a texture label, the generator outputs a texture sample image, and when the label information is a cement label, the generator outputs a cement sample image.

Step S502, inputting the pseudo-geomorphic data and the corresponding geomorphic coarse-grained type information thereof, the real geomorphic data and the corresponding geomorphic coarse-grained type information thereof into a discriminator in a condition generating countermeasure network, wherein the discriminator is used for judging whether the sample image and the pseudo-geomorphic data are consistent; the real landform data is a sample image in the landform sample data set after manual labeling.

It should be noted that the real geomorphic data is a geomorphic sample image corresponding to a geomorphic sample data set after manual labeling. The landform coarse-grained type information corresponding to the real landform data refers to information characterizing the landform type of the landform sample image. For example, when the input real landform data is a grassland sample image, the corresponding rough-grained category information of the landform is a grassland category. Similarly, the geomorphic coarse-grained type information corresponding to the pseudo geomorphic data refers to information characterizing the geomorphic type of the sample image generated by the generator.

And S503, acquiring the judgment result output by the discriminator, continuously iterating the training condition generation type countermeasure network according to the judgment result, and taking the pseudo-geomorphic data output by the generator as target pseudo-geomorphic data until the judgment result is optimal.

Specifically, after generating the pseudo-landform data of the corresponding landform category, the generated pseudo-landform data, coarse-grained category information corresponding to the pseudo-landform data, a corresponding real sample image, and coarse-grained category information corresponding to the real sample image are input to a discriminator, and a discrimination result output by the discriminator is obtained. For example, when the pseudo-geomorphic data is a grassland sample image, the grassland sample image in the manually labeled geomorphic sample data set is used as real geomorphic data, and corresponding data is input into the discriminator to obtain a discrimination result output by the discriminator, wherein the discrimination result can represent the probability that the currently generated geomorphic data belongs to the real geomorphic data.

And when the judgment result is consistent, the fact that the real geomorphic data and the pseudo geomorphic data cannot be distinguished by the judger is shown, the training result is shown to be optimal, and at the moment, the pseudo geomorphic data output by the generator can be used as target pseudo geomorphic data.

It should be understood that the confrontation training process of the generative confrontation network is an iterative training process, and the above steps S501 to S503 are only described for one training process. And (4) performing iterative training according to the processes of the steps S501 to S503 until the output result reaches the optimal value, and considering that the training is finished.

The objective function formula of the condition generating type countermeasure network is specifically as follows:

z represents random noise, x specifically refers to an input image block, y refers to extra information which can be any information, and information is signed in the index; the conditional generation network countermeasure model is V (G, D), the generator is G (z | y), and the discriminator is D (x | y).

To better describe the countermeasure training process of the conditional generation countermeasure network and the constituent architecture thereof, the following description will be made in conjunction with a schematic diagram of a conditional generation countermeasure network model framework shown in fig. 6.

As shown in fig. 6, the conditional generation countermeasure network includes G (generator) to which Z (random noise) and C (coarse-grained classification information) are input and D (discriminator) to which G (generator) outputs X (pseudo-geomorphic data); inputting X (false landform data), X (real landform data) and C (coarse-grained type information) corresponding to each to D (discriminator); and D (a discriminator) outputs discrimination results of the real landform data and the pseudo landform data.

It can be seen that sample images of different landform categories can be generated through the conditional generation type countermeasure network to achieve the purpose of expanding the training sample data set, so that a better landform image building effect can be achieved under the condition that as few manual labeling samples as possible are used.

In practical application, a large amount of training data can be generated by using a conditional generation type countermeasure network, but the generated training data may have insufficient diversity of generated samples, so that the generated samples are not abundant enough, and the subsequent landform mapping effect is influenced. To overcome this problem, fine-grained representation information conditional generation may be used to generate better quality training data against the network.

In other embodiments, when the generated countermeasure network is a fine-grained representation information conditional generation countermeasure network, the tag information includes geomorphic coarse-grained type information and geomorphic fine-grained type information. Referring to the schematic block diagram of the flow of the countermeasure training process shown in fig. 7, the specific process of performing the countermeasure training on the generative countermeasure network by using the tag information and the manually labeled geomorphic sample data set to obtain the target pseudo geomorphic data output by the generative countermeasure network may include:

step S701, inputting the random noise, the landform rough category information and the landform fine-grained category information into a generator in the fine-grained representation information condition generation type countermeasure network, and acquiring the pseudo-landform data output by the generator.

It can be understood that the random noise may be gaussian random noise, and the fine-grained morphology category information may represent a morphology category to which the morphology sample image belongs, where the morphology category is a fine-grained morphology category. Where the fine-grained categories are further representations of coarse-grained categories, e.g., grasses, and the corresponding fine-grained categories include bald grasses, sparse grasses, and dense grasses.

Step S702, inputting the pseudo-landform data and the corresponding landform coarse-grained type information thereof, the real landform data and the corresponding landform coarse-grained type information thereof into a discriminator in a fine-grained representation information condition generation type countermeasure network, and acquiring a discrimination result output by the discriminator; the real landform data is a sample image in the landform sample data set after manual labeling.

Step S703, inputting the pseudo-geomorphic data to a fine-grained hidden coding network in the fine-grained represented information condition generating countermeasure network, and obtaining a fine-grained hidden code output by the fine-grained hidden coding network.

It should be noted that, the fine-grained information encoding network may enable the fine-grained semantic representation in the generated pseudo-geomorphic data to approach the specified fine-grained category information.

Step S704, continuously and iteratively training a fine-grained representation information condition generation type countermeasure network, and taking the pseudo-geomorphic data as the target pseudo-geomorphic data until the judgment result reaches the optimal value and the fine-grained hidden code reaches the preset effect.

It should be noted that, when the discriminator cannot distinguish between the real geomorphic data and the pseudo geomorphic data, that is, when the generative countermeasure network tends to converge, the discrimination result can be considered to be optimal. Further, mutual information of the pseudo-geomorphic data and the fine-grained categories is calculated, the larger the mutual information is, the larger the correlation between the generated image and the fine-grained categories is. When the mutual information reaches a certain value, the fine-grained hidden code can be considered to achieve a preset effect. At this time, the pseudo-geomorphic data output by the generator may be used as the target pseudo-geomorphic data.

The mutual information calculation formula is specifically as follows:

I[s；G((z,s)|c)]＝H[s]-H[s|G((z,s)|c)]

the specific development is as follows:

the objective function formula of the fine-grained representation information condition generation type countermeasure network is specifically as follows:

wherein, the fine-grained representation information condition generation type countermeasure network model is

The generator is G (z, s | c), the discriminator model is D (x | c), c is the landform coarse-grained type information, s is the landform fine-grained type information, Q is the fine-grained hidden coding network, I [ s; g (z, s | c)]Are mutual information. L is the lower bound of variation of mutual information. In order to better describe the model of the fine-grained representation information condition generating type confrontation network and the corresponding training process, the following description will be made in conjunction with a fine-grained representation information condition generating type confrontation network model framework diagram shown in fig. 8.

As shown in fig. 8, the fine-grained representation information condition generation type confrontation network model includes G (generator), D (discriminator), and Qs (fine-grained hidden coding network). Inputting Z (random noise), S (fine-grained hidden coding) and C (coarse-grained category information) into G (generator), and outputting corresponding X (pseudo-geomorphic data) by the G (generator); inputting X (false landform data), X (real landform data) and C (coarse grain class information) corresponding to the X (false landform data) into D (discriminator), and outputting discrimination results of the real landform data and the false landform data by the D (discriminator); and inputting X (pseudo landform data) into a Qs (fine-grained hidden coding network) to obtain an output S (fine-grained hidden coding). Therefore, the target pseudo-geomorphic data required by the condition generating type confrontation network generation of the fine-grained representation information can be used for learning additional fine-grained representation information on the basis of the condition generating type confrontation network, the generation effect of the target pseudo-geomorphic data is improved, and the generator can sample better data to be used for training a geomorphic classification model so as to obtain better geomorphic mapping effect.

And S103, training the pre-trained landform classification model by using the target pseudo-landform data.

It is understood that the pre-training process of the pre-trained landform classification model can be referred to above, and is not described herein again. The target pseudo-geomorphic data is a sample image which is generated by a training finished generative confrontation network and is provided with different pseudo labels.

Specifically, sample images with different pseudo labels are respectively input, the softmax function layer predicts the landform categories of the input sample images, and then judges whether the predicted landform categories are consistent with the landform categories of the input sample images. And (4) performing iterative training for multiple times until the accuracy of predicting the landform type of the input sample image is more than or equal to 90%, and finishing training by considering the trained and pre-trained landform classification model. Wherein the accuracy can be calculated by formula

Calculating to obtain the total number of the test samples,

M_ijthe number of test samples representing class i classified into class j.

The softmax function layer firstly obtains probability values of different classes of the input sample images, and finally predicts which class the input sample images belong to. In particular according to the formula

Obtaining probability values that input image blocks belong to different categories, wherein theta is a model parameter, x is an input image block, j represents the total category number, and i represents the output several categories; obtaining probability values of different categories of the input image blocks, and then obtaining the probability values according to a formula

Predicting the landform category to which the image block belongs, x representing the picture in the prediction set D, C representing the specific landform category, C representing the total number of categories, p_c(x) Representing the probability value that picture x belongs to category c. The landform belonging to which category is specifically determined by the probability that the input image block belongs to which landform category is the largest. For example, the probability values of the classes to which the currently input sample image belongs are obtained by the softmax function layer, and are 0.5 grassland, 0.3 cement land and 0.2 texture land, and finally the input sample image is predicted to belong to the grassland data.

It should be noted that the sample image included in the target pseudo-geomorphic data may be an entire geomorphic image or an image block. When the image blocks are used for training, the images also need to be cut into image blocks for classification when the images are built by the subsequent landform.

It should be noted that the pre-training process may be regarded as a preliminary training process, and the used training sample data is less, and the classification accuracy of the pre-trained landform classification model is lower. The process of training the pre-trained landform classification model by using the generated target pseudo-landform data can be regarded as further training after the initial training, the number of used training samples is more, and the classification precision of the trained landform classification model is higher.

After the landform classification model is trained, the trained landform classification model is used for classifying the collected landform images, and then the landform image is built. The following describes how to use the trained landform classification model to perform landform mapping.

Referring to fig. 9, a schematic flow chart diagram of a method for constructing a topographic map provided in an embodiment of the present application is shown, where the method may include the following steps:

and step S901, acquiring a landform image to be processed.

It can be understood that the above-mentioned to-be-processed relief image may be a relief original image acquired by the unmanned aerial vehicle, and may also be other to-be-processed relief images.

S902, obtaining a landform classification result of the landform image to be processed according to the trained landform classification model and the landform image to be processed; the landform classification model is a model obtained by training through any one of the landform classification model training methods.

It is understood that the above-mentioned geomorphic classification model is the geomorphic classification model in the above geomorphic classification model training method. That is, the geomorphic classification model is obtained by training using any one of the geomorphic classification model training methods described above, and for the relevant description of the model training, please refer to the above, which is not described herein again.

And inputting the trained landform classification model into a landform image, and outputting a landform classification result of the landform image. The landform classification result can represent which kind of landform class the landform image corresponds to. For example, when the input landform image is a grassland image shot by an unmanned aerial vehicle, the output landform classification result is a grassland category.

When the training data sets are different, the output of the landform classification model is correspondingly different. For example, when the training data set is a coarse-grained landform sample image, the trained landform classification model can only recognize the landform of a coarse-grained category. For example, pending landform image is the bald meadow image that unmanned aerial vehicle shot, and the training data set of the landform classification model that trains is coarse grained's landform sample image, and at this moment, with bald meadow image input to the landform classification model in, the classification result of landform classification model output is the meadow, and not the bald meadow. Similarly, when the training data set is a fine-grained landform sample image, the trained landform classification model can identify the fine-grained landform.

It should be noted that the input of the geomorphic classification model may be the whole to-be-processed geomorphic image, or the whole to-be-processed geomorphic image may be cut into image blocks, that is, the image blocks are input to the geomorphic classification model. In contrast, the entire to-be-processed relief image may not have the features of the entire picture, or it is difficult to determine the features representing the entire picture, resulting in a poor effect of constructing the relief image from the entire picture. Preferably, after the geomorphic image to be processed is cut into a plurality of image blocks, the geomorphic image is built through the image blocks.

And step S903, constructing a landform graph according to the landform classification result.

When the input of the feature classification model is an image block, the classification result of each image block is output. And after the landform classification result of each image block is obtained, splicing the image blocks together again according to the landform classification result to form a complete landform image.

The process of mapping the terrain by means of image blocks will be described below.

Referring to FIG. 10, another flow diagram schematic of a topographical map construction method is shown, which may include the following steps;

and step S1001, acquiring a landform image to be processed.

And step S1002, cutting the landform image to be processed into image blocks.

Specifically, the landform image to be processed is cut into a plurality of image blocks with the same size according to a preset size. For example, the relief image to be processed is cut into a plurality of 64 × 64 image blocks.

And step S1003, inputting the image blocks into the landform classification model to obtain the landform classification result of each image block.

It will be appreciated that each image block may correspond to a different landscape, for example, one image block being a grass land, one image block being a texture land, one image block being a cement land, etc. And inputting the image blocks into the landform classification model, so that a landform classification result of each image block can be obtained.

Step S1004, labeling each image block with a preset label corresponding to the landform classification result according to the landform category to which each image block belongs, to obtain a labeled image block.

Note that, labeling the image blocks with preset labels is to facilitate distinguishing different types of landforms. For example, a first landscape uses a first type of label and a second landscape uses a second type of label. And presetting the corresponding relation between the landform type and the label, and then directly marking by using the corresponding label.

The preset labels can be any marks as long as different types of landforms can be distinguished. For example, the labeling is performed according to preset colors, that is, different colors are used to distinguish different types of landforms. Specifically, the grassland corresponds to green, the cement land corresponds to blue, and the texture land corresponds to orange, when the landform classification result of a certain image block is the grassland, the image block is filled with green, and when the landform classification result is the cement land, the image block is filled with blue.

Step S1005, splicing the image blocks according to the positions of the image blocks in the geomorphic image to be processed, so as to construct a geomorphic image.

Specifically, after the image blocks are labeled, the labeled image blocks may be re-spliced together. For example, when a geomorphic image to be processed is cut into four image blocks, the positions of the four image blocks in the original image are the upper left corner, the upper right corner, the lower left corner and the lower right corner respectively; the landform classification results of the four image blocks are obtained through the landform classification model respectively, and after the corresponding labels are used for labeling, the image blocks are spliced together according to the positions of the image blocks in the original image, specifically, the image blocks originally positioned in the upper left corner are placed in the upper left corner and spliced together according to the positions to form a complete landform image. After the complete landform image is formed by splicing, the landform category of the corresponding area can be known through different labels.

According to the embodiment of the application, the training samples are efficiently selected in an active learning mode, and meanwhile, a large amount of training data are generated by utilizing the generation type countermeasure network, so that the data acquisition time is shortened. In addition, sample images of different landform categories can be generated through the conditional generation type confrontation network so as to expand training sample data, and a better landform drawing establishing effect can be achieved under the condition of less manually marked samples. And the fine-grained representation information condition generation type countermeasure network is adopted, and on the basis of the condition generation type countermeasure network, additional fine-grained representation information is learned, so that training sample data with better quality is generated, and the topographic effect is better.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Corresponding to the training method of the geomorphic classification model described in the above embodiments, a corresponding training apparatus of the geomorphic classification model will be described below.

Referring to fig. 11, a block diagram of a geomorphic classification model training apparatus provided in the embodiment of the present application is shown, where the apparatus may include:

the first obtaining module 111 is configured to obtain a landform sample data set after manual labeling and corresponding tag information;

the confrontation training module 112 is configured to perform confrontation training on the generative confrontation network by using the tag information and the manually labeled geomorphic sample data set, so as to obtain target pseudo geomorphic data output by the generative confrontation network; the target pseudo-geomorphic data is pseudo-geomorphic data which is output when the training effect of the generative countermeasure network reaches the optimum;

and the training module 113 is configured to train the pre-trained landform classification model by using the target pseudo-landform data.

In a possible implementation manner, the generative confrontation network is a conditional generative confrontation network, and the tag information is geomorphic coarse-grained type information;

the confrontation training module may be specifically configured to:

inputting random noise and geomorphic coarse-grained type information into a generator in a condition generating countermeasure network, and acquiring pseudo geomorphic data output by the generator;

inputting the pseudo-landform data, the landform coarse-grained type information corresponding to the pseudo-landform data, the real landform data and the landform coarse-grained type information corresponding to the real landform data into a discriminator in the condition generating countermeasure network, wherein the discriminator is used for judging whether the sample image and the pseudo-landform data are consistent; wherein, the real landform data is a sample image in the landform sample data set after manual marking;

and acquiring a discrimination result output by the discriminator, continuously iterating the training condition to generate the confrontation network, and taking the pseudo-geomorphic data output by the generator as target pseudo-geomorphic data when the discrimination result is optimal.

In a possible implementation manner, the generative confrontation network is a condition generative confrontation network of fine-grained representation information, and the tag information includes geomorphic coarse-grained category information and geomorphic fine-grained category information;

the confrontation training module may be specifically configured to:

inputting random noise, landform coarse category information and landform fine-grained category information into a generator in a fine-grained representation information condition generation type countermeasure network, and acquiring pseudo-landform data output by the generator;

inputting the pseudo-landform data, the landform coarse-grained type information corresponding to the pseudo-landform data, the real landform data and the landform coarse-grained type information corresponding to the real landform data into a discriminator in a fine-grained representation information condition generation type countermeasure network, and acquiring a discrimination result output by the discriminator; wherein, the real landform data is a sample image in the landform sample data set after manual marking;

inputting the pseudo-geomorphic data into a fine-grained hidden coding network in a fine-grained representation information condition generation type countermeasure network, and acquiring fine-grained hidden codes output by the fine-grained hidden coding network;

and continuously and iteratively training the fine-grained representation information condition generation type countermeasure network until the judgment result reaches the optimal state and the fine-grained hidden code reaches the preset effect, and taking the pseudo-geomorphic data as the target pseudo-geomorphic data.

In one possible implementation, the objective function formula of the fine-grained representation information condition generation type countermeasure network is as follows:

wherein c is the information of the coarse-grained type of the landform, s is the information of the fine-grained type of the landform, z is random noise, and the fine-grained representation information condition generation type confrontation network model isThe generator model is G (z, s | c), the discriminator model is D (x | c), and Q is a fine-grained hidden coding network. Is [ s; g (z, s | c)]And L is the lower bound of variation of the mutual information.

In a possible implementation manner, the apparatus may further include:

the second acquisition module is used for acquiring the unmarked landform sample data set;

the selection module is used for selecting a target sample image from the unlabeled landform sample data set based on the pre-trained landform classification model according to the information entropy of the sample image of each landform category;

and the artificial labeling module is used for obtaining the artificially labeled target sample image after artificially labeling the target sample image so as to obtain the artificially labeled landform sample data set comprising the artificially labeled target sample image.

In a possible implementation manner, the apparatus may further include:

the third acquisition module is used for acquiring the acquired original landform image;

and the cutting module is used for cutting the landform original image into image blocks so as to obtain an unlabeled landform sample data set comprising the image blocks.

In a possible implementation manner, the apparatus further includes a pre-training module, specifically configured to:

selecting a training sample from the unlabelled landform sample data set;

after the training samples are manually labeled, the pre-constructed landform classification model is pre-trained by using the manually labeled training samples.

It should be noted that the geomorphic classification model training device corresponds to the geomorphic classification model training method described above one to one, and for the related description, reference is made to the above corresponding contents, which are not described herein again.

Corresponding to the topographic map building method described in the above embodiment, a corresponding topographic map building apparatus will be described below.

Referring to fig. 12, a block diagram of a topographic map constructing apparatus provided in the embodiment of the present application is shown, where the apparatus may include:

an image obtaining module 121, configured to obtain a landform image to be processed;

the classification module 122 is configured to obtain a landform classification result of the landform image to be processed according to the trained landform classification model and the landform image to be processed; wherein, the landform classification model is a model obtained by training through the landform classification model training method of any one of the first aspect;

and the constructing module 123 is configured to construct a landform map according to the landform classification result.

In a possible implementation manner, the classification module is specifically configured to:

cutting the landform image to be processed into image blocks;

In a possible implementation manner, the building module is specifically configured to:

labeling each image block by using a preset label corresponding to a landform classification result according to the landform category of each image block to obtain a labeled image block;

and splicing the image blocks according to the positions of the image blocks in the landform image to be processed to construct a landform image.

It should be noted that the topographic map building apparatus and the topographic map building method described above correspond to each other, and for related introduction, reference is made to the above corresponding contents, which are not described herein again.

It should be noted that, for the information interaction, execution process, and other contents between the above-mentioned devices/modules, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and reference may be made to the part of the embodiment of the method specifically, and details are not described here.

Fig. 13 is a schematic structural diagram of a terminal device according to an embodiment of the present application. As shown in fig. 13, the terminal device 13 of this embodiment includes: at least one processor 130, a memory 131, and a computer program 132 stored in the memory 131 and executable on the at least one processor 130, the processor 130 implementing the steps in any of the above-described individual geomorphic classification model training method and/or geomorphologic map construction method embodiments when executing the computer program 132.

The terminal device 13 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The terminal device may include, but is not limited to, a processor 130, a memory 131. Those skilled in the art will appreciate that fig. 13 is only an example of the terminal device 13, and does not constitute a limitation to the terminal device 13, and may include more or less components than those shown, or combine some components, or different components, such as an input/output device, a network access device, and the like.

The Processor 130 may be a Central Processing Unit (CPU), and the Processor 130 may be other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The storage 131 may in some embodiments be an internal storage unit of the terminal device 13, such as a hard disk or a memory of the terminal device 13. The memory 131 may also be an external storage device of the terminal device 13 in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 13. Further, the memory 131 may also include both an internal storage unit and an external storage device of the terminal device 13. The memory 131 is used for storing an operating system, an application program, a BootLoader (BootLoader), data, and other programs, such as program codes of the computer programs. The memory 131 may also be used to temporarily store data that has been output or is to be output.

An embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored, and when being executed by a processor, the computer program implements the steps in any of the above-mentioned geomorphic classification model training method and/or geomorphic graph construction method embodiments.

When the computer program product runs on the terminal device, the steps in the above-mentioned individual geomorphic classification model training method and/or geomorphic graph construction method embodiments are implemented when the terminal device executes the computer program product.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims

1. A semi-supervised landform classification model training method is characterized by comprising the following steps:

2. The semi-supervised geomorphic classification model training method of claim 1, wherein the generative confrontation network is a conditional generative confrontation network, and the label information is geomorphic coarse-grained type information;

3. The semi-supervised geomorphic classification model training method of claim 1, wherein the generative confrontation network is a fine-grained representation information conditional generative confrontation network, and the label information comprises geomorphic coarse-grained type information and geomorphic fine-grained type information;

4. The semi-supervised geomorphic classification model training method of claim 3, wherein the objective function formula of the fine-grained representation information condition generation countermeasure network is as follows:

5. The semi-supervised geomorphic classification model training method of any one of claims 1 to 4, wherein before the obtaining of the manually labeled geomorphic sample data set and corresponding label information, further comprising:

acquiring an unlabelled geomorphic sample data set;

6. The semi-supervised geomorphic classification model training method of claim 5, wherein before the obtaining of the unlabelled geomorphic sample data set, further comprising:

acquiring an acquired original landform image;

7. The semi-supervised geomorphic classification model training method of claim 5, wherein the pre-training process of the geomorphic classification model comprises:

selecting a training sample from the unlabelled landform sample data set;

8. A method for constructing a topographic map is characterized by comprising the following steps:

acquiring a landform image to be processed;

obtaining a landform classification result of the landform image to be processed according to the trained landform classification model and the landform image to be processed; wherein, the landform classification model is a model obtained by training through the semi-supervised landform classification model training method of any one of the above claims 1 to 7;

9. The method for constructing a geomorphic graph according to claim 8, wherein the obtaining of the geomorphic classification result of the geomorphic image to be processed according to the trained geomorphic classification model and the geomorphic image to be processed comprises:

cutting the landform image to be processed into image blocks;

10. The method for constructing a geomorphic graph according to claim 9, wherein the constructing a geomorphic graph according to the geomorphic classification result comprises:

11. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 7 and/or 8 to 10 when executing the computer program.

12. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7 and/or 8 to 10.