CN116612385B - Remote sensing image multiclass information extraction method and system based on depth high-resolution relation graph convolution - Google Patents

Abstract

The invention discloses a method and system for extracting multi-category information from remote sensing images based on deep high-resolution relationship graph convolution and a computer storage medium. The method includes: S1: Divide the feature maps of different dimensions according to the SLIC superpixel segmentation results of the original RGB image of the corresponding size; S2: Segment the feature map to obtain K categories, and each pixel has a corresponding SLIC category; S3 : Learning by constructing resolution images of different sizes into a heterogeneous graph with two relationships as edges and SLIC classification as nodes; S4: Restore graph neural networks of different dimensions into feature maps, and cooperate with the fully connected layer to Complete the classification of pixels to extract the target category. This invention utilizes high-resolution features and relationship information in the generated graph structure to explicitly combine the scenario of multi-label classification tasks with graph learning, and has better modeling for heterogeneous graphs with multiple classes. method.

Description

Multi-category information extraction method for remote sensing images based on deep high-resolution relationship graph convolution with system

Technical field

The invention relates to the field of remote sensing, the field of deep learning computer vision, and the field of graph learning, and more specifically to a method and system for extracting multi-category information from remote sensing images based on deep high-resolution relationship graph convolution.

Background technique

In recent years, semantic segmentation based on deep learning has become a mainstream method. Semantic segmentation is a method of classifying pixels in a picture one by one to extract the desired information category. When facing remote sensing images, traditional information extraction methods mainly rely on the traditional convolutional neural network framework to extract features. However, convolution has a limited field of view and mainly explores the feature relationships of local small-scale pixels, thus limiting multi-label classification. Under the task of capturing the long-range dependent features of different labels in high-resolution remote sensing images.

Fully Convolutional Networks (FCNs) [Fully Convolutional Networks for Semantic Segmentation] first removes the fully connected layer and introduces an end-to-end training mode for semantic segmentation. However, the downsampled feature maps of FCNs destroy spatial information. Segmentation results easily lose boundary information. SegNet [SegNet: A DeepConvolutional Encoder-Decoder Architecture for Image Segmentation] utilizes an encoder-decoder structure. It utilizes max pooling position index to upsample the feature map. Therefore, the spatial information and high-frequency features lost due to max pooling can be recovered. U-Net [U-Net: Convolutional Networks for BiomedicalImage Segmentation] utilizes learnable transposed convolutions instead of interpolation to upsample feature maps. Skip connections are used so that the decoder knows the information lost due to max pooling at each stage. The Deeplab [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs] series of work mainly focuses on the research of atrous convolution. Atrous convolution can expand the receptive field of the convolution kernel while maintaining the spatial resolution of the image. The Atrous Spatial Pyramid Pooling (ASPP) module proposed in [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, AtrousConvolution, and Fully Connected CRFs] can effectively capture multi-scale contextual semantic information. However, this method is not effective in small-scale target segmentation and easily loses boundary information. In order to solve the problem of spatial information loss, Lin et al. proposed RefineNet [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation], which inputs each layer of the encoder's feature map into a refine block. The thinning block can fully integrate spatial information of different resolutions. Therefore, RefineNet achieves the task of image semantic segmentation with high accuracy.

However, convolutional neural networks (CNNs) can only focus on the small-scale impact on local information in the image. The above methods are difficult to capture the long-range dependence characteristics of high-resolution remote sensing images. To overcome this problem, Liang [A Deep NeuralNetwork Combined CNN and GCN for Remote Sensing Scene Classification] et al. introduced superpixel nodes in images to generate graphs, but did not consider graph models with multiple labels. In remote sensing image scene classification, Li [Multi-Label Remote Sensing Image Scene Classification by Combining aConvolutional Neural Network and a Graph Neural Network] et al. use superpixels to generate image graphics structures. Combining remote high-level semantic information with graph convolutional networks (GNNs) has achieved good results. However, it does not explicitly combine the scenario of multi-label classification tasks with graph learning. There are better ways to model heterogeneous graphs with multiple classes.

In this invention, inspired by High-Resolution Network (HRNet) [Deep High-Resolution Representation Learning for Human Pose Estimation] and Relational Graph Convolutional Network (R-GCN) [ModelingRelational Data with Graph Convolutional Networks], a A multi-category information extraction method for remote sensing images based on deep high-resolution relationship graph convolution, which utilizes high-resolution features and relationship information in the generated graph structure.

Contents of the invention

In view of the above problems, the present invention provides a method for extracting multi-category information from remote sensing images based on deep high-resolution relationship graph convolution, which aims to solve the existing problem of not clarifying the scene and graphics learning of multi-label classification tasks. .

In order to achieve the above objectives, according to one aspect of the present invention, a method for extracting multi-category information from remote sensing images based on depth and high-resolution relationship graph convolution is provided. The method includes the following steps:

S1: Divide the feature maps of different sizes and dimensions according to the SLIC superpixel segmentation results of the original RGB image of the corresponding size; the specific method is: for images of different sizes, after manually setting the number of clusters K, assume that the original image The number of pixels is N. The image is first divided into blocks of the same size, and the size of each block is S; here Among the divided blocks, cluster centers are chosen randomly. Here, in order to avoid the sampling point being at the edge or in the image noise part, it is necessary to manually adjust the points adjacent to the pixel gradient in the area near a sampling point, select the smallest gradient as the cluster center, and calculate the relationship between the pixel point and the pixel within the range of 2S*2S. The color distance d _c and spatial distance d _s of the cluster center; among them, the value of point i in RGB coordinates is (l _i , a _i , b _i ), and the value of point j in RGB coordinates is (l _j , a _j , b _j ); the value of point i in the distance coordinate is (x _i , y _i ), and the value of point j in the distance coordinate is (x _j , y _j ); then, the calculation formulas of dc and ds are as follows:

After calculating the distance, each pixel will update the image block to which it belongs, average the pixels of the same image block to obtain a new cluster center, and then repeat the previous steps until the distance between the two cluster centers is less than a set threshold.

S2: K categories can be obtained by segmenting the feature map, and each pixel has a corresponding SLIC category; at this time, the K categories are regarded as K nodes, and the characteristics of the nodes are the characteristics of the feature map position where the pixel is located. The number of channels is the same as the number of feature dimensions of the node. Add edges between adjacent categories. There are two types of edges, which are determined by calculating the similarity between nodes. The specific calculation method is as follows:

Among them, a and b are two n-dimensional vectors, a _i represents the i-th dimension feature of node a, b _i represents the i-th dimension feature of node b, and the value of n depends on the number of channels of the feature map at this time. After normalization, if the value is greater than 0.5, it is classified as Class 1 edge, otherwise it is classified as Class 0, so as to learn similar pixel sets and different pixel sets.

S3: For images of different resolutions, by calculating the similarity of features corresponding to the SLIC category, it is divided into two types of relationships: similar and dissimilar, and a heterogeneous graph with the two relationships as edges and the SLIC classification as nodes is constructed for learning; since we follow Node similarity divides edges into two categories. We will learn different feature transformation matrices according to different edge types. The specific formula is as follows:

in, is the embedding of node i in layer l,/> is the embedding of node i in layer l+1,/> Represents the set of neighbor nodes of node i under the r-th relationship, c _{i, r} are constants,/> Represents the feature transformation matrix under the relationship r of the lth layer, when R=0, that is/> It represents the relationship feature transformation matrix of the node to the next layer of itself; R represents the type of relationship, the relationship with R=1 represents "similarity", and the relationship with R=2 represents "dissimilarity".

S4: Restore graph neural networks of different dimensions into feature maps, and use the fully connected layer to classify each pixel to extract the target category.

On the other hand, the present invention is a remote sensing image multi-category information extraction system based on deep high-resolution relationship graph convolution. The system specifically includes:

The segmentation module is used to divide feature maps of different sizes and dimensions according to the SLIC superpixel segmentation results of the original RGB image of the corresponding size; the specific method is: for images of different sizes, after manually setting the number of clusters K, assuming that the original Some images have N pixels. The image is first divided into blocks of the same size, and the size of each block is S;

The classification module is used to divide the feature map into K categories according to the segmentation module, and each pixel has a corresponding SLIC category;

The learning module is used to calculate the similarity of features corresponding to SLIC categories for images of different resolutions, divide them into two types of relationships, similar and dissimilar, and construct a heterogeneous graph with the two relationships as edges and the SLIC classification as nodes for learning;

The extraction module is used to restore graph neural networks of different dimensions into feature maps, and cooperates with the fully connected layer to classify each pixel, thereby extracting the target category.

Another aspect of the present invention is a computer storage medium. Computer program instructions are stored in the computer storage medium. When the computer program instructions are executed by a processor, any one of the above-mentioned depth-based high-resolution relationship graphs can be implemented. Multi-category information extraction method for accumulated remote sensing images.

The proposed method was tested on Potsdam and Vaihingen public datasets and compared with state-of-the-art methods. Under two accuracy metrics: F1 (F1-score) and IoU (Intersection over Union), the method outperformed for all comparison methods.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only They are used for the purpose of illustrating preferred embodiments and are not to be construed as limitations of the invention. Also throughout the drawings, the same reference characters are used to designate the same components. In the attached picture:

Figure 1 shows a schematic diagram of multi-category information extraction from remote sensing images based on deep high-resolution relationship graph convolution;

Figure 2 shows the class map obtained by segmenting the feature map.

Detailed ways

In order to make the above objects, features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only for the purpose of this application. Some embodiments of the invention are disclosed, but not all the embodiments. Based on the embodiments of the present invention, all other embodiments by those skilled in the art without creative efforts shall fall within the scope of protection of the present invention.

As shown in Figure 1, HRNet was originally proposed to solve the human pose estimation problem. The network contains four parallel subnetworks representing four different resolutions. At the same time, as the depth increases, high-resolution subnets are gradually added to low-resolution subnets, and multi-resolution subnets are connected in parallel. Unlike the encoder-decoder architecture, this network does not require upsampling from downsampled feature maps. Since HRNet always maintains high-resolution feature representation, it intuitively brings richer semantic features to the semantic segmentation task. In the past practice, feature maps of four sizes were finally sampled into the same dimension for prediction. We divide the feature maps of different size dimensions according to the SLIC superpixel segmentation results of the original RGB image of the corresponding size.

The specific method is: for images of different sizes, after manually setting the number of clusters K, assuming that the number of pixels in the original image is N, we first divide the image into blocks of the same size, and the size of each block is S. here Among the divided blocks, cluster centers are chosen randomly. Here, in order to avoid the sampling point being at the edge or in the image noise part, we need to manually adjust the points adjacent to the pixel gradient in the area near a sampling point, select the smallest gradient as the cluster center, and calculate the pixel point and The color distance d _c and spatial distance d _s of the cluster center; among them, the value of point i in RGB coordinates is (l _i , a _i , b _i ), and the value of point j in RGB coordinates is (l _j , a _j , b _j ); the value of point i in the distance coordinate is (x _i , y _i ), and the value of point j in the distance coordinate is (x _j , y _j ); then, the calculation formulas of dc and ds are as follows:

After calculating the distance, each pixel will update the image block to which it belongs, average the pixels of the same image block to obtain a new cluster center, and then repeat the previous steps until the distance between the two cluster centers is less than a set threshold.

From this, the segmentation feature map can obtain K categories as shown in Figure 2. Each pixel therefore has a corresponding SLIC class. At this time, K categories are regarded as K nodes. The characteristics of the node are the characteristics of the feature map position where the pixel is located. The number of channels of the feature map is the same as the number of feature dimensions of the node. Add edges between adjacent categories. There are two types of edges, which are determined by calculating the similarity between nodes. The specific calculation method is as follows:

Among them, a and b are two n-dimensional vectors, a _i represents the i-th dimension feature of node a, b _i represents the i-th dimension feature of node b, and the value of n depends on the number of channels of the feature map at this time. After normalization, if the value is greater than 0.5, it is classified as Class 1 edge, otherwise it is classified as Class 0, so as to learn similar pixel sets and different pixel sets. Learning by constructing resolution images (Images) of different sizes into a heterogeneous graph (Graph) with two relationships as edges and slic classification as nodes has two advantages: 1) The graph structure will expand the characteristics of different resolutions The scale of graph information update is not limited to the small area of local rules. Because at this time, the objects of convolution are no longer pixels in a fixed area, but nodes obtained by slic classification with a larger field of view. Each node obtained by slic classification corresponds to a piece of pixels with similar characteristics, and the distribution shape of the pixels is also Can be irregular. When performing a convolution operation on a graph structure, feature learning is actually performed on multiple large-scale pixels. 2) Heterogeneous graphs based on feature similarity can aggregate pixels into category sets to improve the classification accuracy of target pixels. After completing the constructed graph, we follow the multi-relationship graph neural network learning steps. Since we divide edges into two categories according to node similarity, we will learn different feature transformation matrices according to different edge types. The specific formula is as follows:

in, is the embedding of node i in layer l,/> is the embedding of node i in layer l+1,/> Represents the set of neighbor nodes of node i under the r-th relationship, c _{i, r} are constants,/> Represents the feature transformation matrix under the relationship r of the lth layer, when R=0, that is/> It represents the relationship feature transformation matrix of the node to the next layer of itself; R represents the type of relationship, the relationship with R=1 represents "similarity", and the relationship with R=2 represents "dissimilarity".

Finally, the graph neural network of different dimensions is restored into a feature map, and the fully connected layer is used to classify each pixel, and the target category can be extracted.

On the other hand, the present invention is a remote sensing image multi-category information extraction system based on deep high-resolution relationship graph convolution. The system specifically includes:

The segmentation module is used to divide feature maps of different sizes and dimensions according to the SLIC superpixel segmentation results of the original RGB image of the corresponding size; the specific method is: for images of different sizes, after manually setting the number of clusters K, assuming that the original Some images have N pixels. The image is first divided into blocks of the same size, and the size of each block is S; here Among the divided blocks, cluster centers are chosen randomly. Among the divided blocks, cluster centers are chosen randomly. Here, in order to avoid the sampling point being at the edge or in the image noise part, it is necessary to manually adjust the points adjacent to the pixel gradient in the area near a sampling point, select the smallest gradient as the cluster center, and calculate the relationship between the pixel point and the pixel within the range of 2S*2S. The color distance d _c and spatial distance d _s of the cluster center; among them, the value of point i in RGB coordinates is (l _i , a _i , b _i ), and the value of point j in RGB coordinates is (l _j , a _j , b _j ); the value of point i in the distance coordinate is (x _i , y _i ), and the value of point j in the distance coordinate is (x _j , y _j ); then, the calculation formulas of dc and ds are as follows:

After calculating the distance, each pixel will update the image block to which it belongs, average the pixels of the same image block to obtain a new cluster center, and then repeat the previous steps until the distance between the two cluster centers is less than a set threshold.

The classification module is used to divide the feature map into K categories according to the segmentation module. Each pixel has a corresponding SLIC category; at this time, the K categories are regarded as K nodes, and the characteristics of the nodes are the positions of the feature maps where the pixels are located. The number of channels of the feature map is the same as the number of feature dimensions of the node. Add edges between adjacent categories. There are two types of edges, which are determined by calculating the similarity between nodes. The specific calculation method is as follows:

Among them, a and b are two n-dimensional vectors, a _i represents the i-th dimension feature of node a, b _i represents the i-th dimension feature of node b, and the value of n depends on the number of channels of the feature map at this time. After normalization, if the value is greater than 0.5, it is classified as Class 1 edge, otherwise it is classified as Class 0, so as to learn similar pixel sets and different pixel sets.

The learning module is used to calculate the similarity of features corresponding to SLIC categories for images of different resolutions, divide them into two types of relationships, similar and dissimilar, and construct a heterogeneous graph with the two relationships as edges and the SLIC classification as nodes for learning; K categories are regarded as K nodes. The characteristics of the node are the characteristics of the feature map position where the pixel is located. The number of channels of the feature map is the same as the number of feature dimensions of the node. Since we divide edges into two categories according to node similarity, we will learn different feature transformation matrices according to different edge types. The specific formula is as follows:

in, is the embedding of node i in layer l,/> is the embedding of node i in layer l+1,/> Represents the set of neighbor nodes of node i under the r-th relationship, c _{i, r} are constants,/> Represents the feature transformation matrix under the relationship r of the lth layer, when R=0, that is/> It represents the relationship feature transformation matrix of the node to the next layer of itself; R represents the type of relationship, the relationship with R=1 represents "similarity", and the relationship with R=2 represents "dissimilarity".

The extraction module is used to restore graph neural networks of different dimensions into feature maps, and cooperate with the fully connected layer to classify each pixel, thereby extracting the target category.

Embodiments of the present application also provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, it implements the above-mentioned multi-class remote sensing image based on deep high-resolution relationship graph convolution. Each process of the information extraction method embodiment can achieve the same technical effect. To avoid duplication, it will not be described again here.

Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on its differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other.

Those skilled in the art should understand that embodiments of the embodiments of the present application may be provided as methods, devices, or computer storage media. Therefore, embodiments of the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment that combines software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

Embodiments of the present application are described with reference to flowcharts and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the present application. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine such that the instructions are executed by the processor of the computer or other programmable data processing terminal device. Means are generated for implementing the functions specified in the process or processes of the flowchart diagrams and/or the block or blocks of the block diagrams.

These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing terminal equipment to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the The instruction means implements the functions specified in a process or processes of the flowchart and/or a block or blocks of the block diagram.

These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, so that a series of operating steps are performed on the computer or other programmable terminal equipment to produce computer-implemented processing, thereby causing the computer or other programmable terminal equipment to perform a computer-implemented process. The instructions executed on provide steps for implementing the functions specified in a process or processes of the flow diagrams and/or a block or blocks of the block diagrams.

Although preferred embodiments of the embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once the basic inventive concepts are understood. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

Claims (9)

1. A remote sensing image multiclass information extraction method based on depth high resolution relation graph convolution is characterized by comprising the following steps:

s1: dividing feature images with different dimension according to the SLIC super-pixel segmentation result of the original RGB image with corresponding size; the specific method comprises the following steps: after the clustering number K is manually set for the images with different sizes, the original image pixel number is assumed to be N, the images are firstly divided into blocks with the same size, and the size of each block is S;

s2: dividing the feature map to obtain K categories, wherein each pixel has a corresponding SLIC category;

s3: aiming at images with different resolutions, the corresponding feature similarity of SLIC categories is calculated and divided into two similar and dissimilar relations, and the two relations are used as edges and SLIC categories are used as heterogeneous graphs of nodes to learn;

s4: restoring the image neural network with different dimensions into a feature image, and completing classification of each pixel by matching with a full-connection layer so as to extract target types;

the method is characterized by comprising the following steps of constructing a heterogeneous graph with two relations as edges and SLIC classification as nodes to learn, wherein the heterogeneous graph specifically comprises the following steps:

edges are added between the connected categories, the categories of the edges are two, the edges are judged by calculating the similarity between the nodes, and the specific calculation mode is as follows:

wherein a and b are two n-dimensional vectors, a _i Representing the ith dimension characteristic of node a, b _i The value of n representing the i-th dimensional feature of node b depends on the number of channels of the feature map at this time; after normalization, if the value is greater than 0.5, determining the value as 1 class edge, otherwise determining the value as 0 class, and learning the similar pixel set and the different pixel sets;

the method comprises the steps of,

according to the multi-relation graph neural network learning step, the edges are divided into two types according to the node similarity, and different feature transformation matrixes are learned according to different edge types, wherein the specific formula is as follows:

wherein,is the embedding of layer i node +.>Is the embedding of layer i of the l+1 node, of->Represents the neighbor node set of the node i under the r-th relation, c _i,r Is constant, & lt>Represents the feature transformation matrix under the relation R of the first layer, when r=0, i.e. +.>Representing the relation feature transformation matrix of the node to the next layer; r represents the kind of relationship, r=1 represents "similar", and r=2 represents "dissimilar".

2. The method for extracting information from multiple classes of remote sensing images based on depth high resolution relationship graph convolution as defined in claim 1, wherein S1 specifically comprises: here, theAnd selecting a cluster center from the divided blocks.

3. The remote sensing image multiclass information extraction method based on depth high resolution relation graph convolution as defined in claim 2, wherein selecting a clustering center specifically comprises: in order to avoid the sampling point being at the edge or the image noise part, the point adjacent to the gradient of the pixel in the area near one sampling point needs to be manually adjusted, the smallest gradient is selected as the clustering center, and the color distance d between the pixel point and the clustering center is calculated in the range of 2S by 2S _c And a spatial distance d _s The method comprises the steps of carrying out a first treatment on the surface of the Wherein the value of the i point in the RGB coordinates is (l) _i ,a _i ,b _i ) The value of j point in RGB coordinates is (l) _j ,a _j ,b _j ) The method comprises the steps of carrying out a first treatment on the surface of the The value of the i point in the distance coordinate is (x _i ，y _i ) The value of j point in the distance coordinate is (x _j ，y _j ) The method comprises the steps of carrying out a first treatment on the surface of the Then, dc and ds are calculatedThe formula is as follows:

after the distance is calculated, each pixel point updates the image block to which the pixel point belongs, averages the pixel points of the same image block to obtain a new clustering center, and then repeats the previous steps until the distance between the two clustering centers is smaller than a set threshold value.

4. The remote sensing image multiclass information extraction method based on depth high resolution relation graph convolution according to claim 1 or 2, wherein K categories are taken as K nodes, the characteristics of the nodes are the characteristics of the pixels in the position of the characteristic graph, and the number of channels of the characteristic graph is the same as the number of characteristic dimensions of the nodes.

5. A remote sensing image multiclass information extraction system based on depth high resolution relation graph convolution is characterized by comprising the following steps:

the segmentation module is used for dividing the feature images with different dimension according to the SLIC super-pixel segmentation result of the original RGB image with the corresponding size; the specific method comprises the following steps: after the clustering number K is manually set for the images with different sizes, the original image pixel number is assumed to be N, the images are firstly divided into blocks with the same size, and the size of each block is S;

the classification module is used for dividing the feature map into K categories according to the segmentation module, and each pixel has a corresponding SLIC category;

the learning module is used for dividing images with different resolutions into two similar and dissimilar relations by calculating the similarity of corresponding features of SLIC categories, and constructing a heterogeneous graph with the two relations as edges and SLIC categories as nodes for learning;

the extraction module is used for restoring the graph neural network with different dimensions into a feature graph, and completing classification of each pixel by matching with the full-connection layer so as to extract the target type;

wherein the learning configured to learn from the heterogeneous graph with two relationships as edges and SLIC classification as nodes further comprises:

edges are added between the connected categories, the categories of the edges are two, the edges are judged by calculating the similarity between the nodes, and the specific calculation mode is as follows:

wherein a and b are two n-dimensional vectors, a _i Representing the ith dimension characteristic of node a, b _i The value of n representing the i-th dimensional feature of node b depends on the number of channels of the feature map at this time; after normalization, if the value is greater than 0.5, determining the value as 1 class edge, otherwise determining the value as 0 class, and learning the similar pixel set and the different pixel sets;

according to the multi-relation graph neural network learning step, the edges are divided into two types according to the node similarity, and different feature transformation matrixes are learned according to different edge types, wherein the specific formula is as follows:

wherein,is the embedding of layer i node +.>Is the embedding of layer i of the l+1 node, of->Represents the neighbor node set of the node i under the r-th relation, c _i,r Is constant, & lt>Represents the feature transformation matrix under the relation R of the first layer, when r=0, i.e. +.>Representing the relation feature transformation matrix of the node to the next layer; r represents the kind of relationship, r=1 represents "similar", and r=2 represents "dissimilar".

6. The remote sensing image multiclass information extraction system based on depth high resolution relational graph convolution of claim 5, whereinAnd selecting a cluster center from the divided blocks.

7. The remote sensing image multiclass information extraction system based on depth high resolution relation graph convolution of claim 6, wherein selecting a clustering center specifically comprises: in order to avoid the sampling point being at the edge or the image noise part, the point adjacent to the gradient of the pixel in the area near one sampling point needs to be manually adjusted, the smallest gradient is selected as the clustering center, and the color distance d between the pixel point and the clustering center is calculated in the range of 2S by 2S _c And a spatial distance d _s The method comprises the steps of carrying out a first treatment on the surface of the Wherein the value of the i point in the RGB coordinates is (l) _i ,a _i ,b _i ) The value of j point in RGB coordinates is (l) _j ,a _j ,b _j ) The method comprises the steps of carrying out a first treatment on the surface of the The value of the i point in the distance coordinate is (x _i ，y _i ) The value of j point in the distance coordinate is (x _j ，y _j ) The method comprises the steps of carrying out a first treatment on the surface of the Then, dc and ds are calculated as follows:

after the distance is calculated, each pixel point updates the image block to which the pixel point belongs, averages the pixel points of the same image block to obtain a new clustering center, and then repeats the previous steps until the distance between the two clustering centers is smaller than a set threshold value.

8. The remote sensing image multi-class information extraction system based on depth high resolution relation graph convolution according to claim 5 or 6, wherein K classes are used as K nodes, the characteristics of the nodes are the characteristics of the pixel in the position of the characteristic graph, and the number of channels of the characteristic graph is the same as the number of characteristic dimensions of the nodes.

9. A computer storage medium, wherein computer program instructions are stored in the computer storage medium, and when the computer program instructions are executed by a processor, the method for extracting multi-class information of remote sensing images based on deep high resolution relation graph convolution is realized according to any one of claims 1-4.