CN116310350B - Urban scene semantic segmentation method based on graph convolution and semi-supervised learning network - Google Patents

Abstract

The invention discloses a method for semantic segmentation of urban scenes based on graph convolution and semi-supervised learning network, comprising the following steps: S1, pre-training the graph convolution network to obtain initialization parameters; S2, inputting the original point set once , the output feature vector ; S3, for the original point set Compute eigenvectors from each point's neighborhood ; S4, calculate the eigenvector and The distance is used as a loss function to adjust the parameters of the graph convolutional network; S5, using labeled data to assign pseudo-labels to unlabeled data; S6, assigning pseudo-labels Do semantic segmentation and predict the category of each point. The method of the invention can realize the semantic segmentation of the urban road scene with only a small amount of labeled data.

Description

Urban scene semantic segmentation method based on graph convolution and semi-supervised learning network

Technical Field

The present invention relates to the field of computer graphics, and in particular to a method for semantic segmentation of urban scenes based on graph convolution and semi-supervised learning networks.

Background Art

As a kind of unstructured three-dimensional data, point cloud represents objects more accurately, flexibly and diversely than data formats such as voxels and grids, and has a wide range of applications in the field of smart cities. For example, in urban construction planning, point cloud is used to generate digital traffic maps to assist in the planning of traffic routes, urban construction, etc., so as to improve planning efficiency and accuracy; in environmental monitoring and analysis, point cloud data can be used to perform three-dimensional modeling of actual scenes, which can be used to analyze landforms, hydrogeology, and building damage, etc., to facilitate urban management and maintenance.

In actual smart city applications, point cloud acquisition and application can usually be divided into the following five steps: (1) point cloud acquisition; (2) point cloud preprocessing; (3) point cloud feature extraction; (4) point cloud semantic segmentation; and (5) downstream model deployment and application.

One difficulty in the above steps is that feature extraction and semantic segmentation require a large amount of labeled data for model training. In feature extraction and semantic segmentation, the traditional method is to use manually designed feature descriptors to extract features or use deep learning methods to automatically extract features using neural networks, thereby achieving semantic segmentation. However, the training process of these methods is usually supervised learning, that is, a large amount of labeled data is required for model training. However, the point cloud of urban scenes is huge in scale. If all points are manually labeled, the process will be cumbersome and expensive.

Summary of the invention

The purpose of the present invention is to overcome the difficulty that the urban scene semantic segmentation algorithm requires a large amount of labeled data, and provide an urban scene semantic segmentation method based on graph convolution and semi-supervised learning network.

The urban scene semantic segmentation method based on graph convolution and semi-supervised learning network includes the following steps:

S1. Use the public and annotated urban road dataset to pre-train the graph convolutional network to obtain the initialization parameters of each layer in the graph convolutional network;

S2: Input the original point set into the initialized graph convolutional network once , The points in the graph only contain coordinates xyz and color rgb information, and the graph convolutional network is used to output the feature vector ;

S3, the original point set in step S2 Use k-NN for each point, find k neighboring points to form a neighborhood, and calculate the feature vector based on the neighborhood of each point ;

S4. Calculate the feature vector and The distance between the two images is used as a loss function, and the loss function is used to adjust the parameters of the graph convolutional network in step S2;

S5. The original point set As a target semantic segmentation dataset , which contains labeled data and unlabeled data, wherein the amount of labeled data accounts for the original point set The ratio is 1%~10%, and then the labeled data is used in the semi-supervised learning network to assign pseudo labels to the unlabeled data;

S6: assign pseudo labels to the Used for network inference, semantic segmentation and predicting the category of each point.

Furthermore, the step S2 is specifically as follows:

S21, using the encoder of the graph convolutional network to encode the original point set Encode to get the encoding features ;

S22, then use the decoder of the graph convolutional network to encode the features Decode to get the decoding features ;

S23, decode the features through MLP Mapping output as feature vector ; Among them, the eigenvector The dimension of each point in is expressed as , and They represent the encoded coordinates and color features respectively. The subscript of r represents the feature channel. The superscript of r represents the mean and 2 represents the variance.

Furthermore, the step S3 is specifically as follows:

For the original point set in step S2 For each point in the graph, k-NN is used to find k neighboring points to form a neighborhood, and the feature vector is calculated based on the neighborhood of each point. , where the dimension of each point is expressed as: ;

The calculation process is:

in, represents the mean of the neighborhood coordinate channel of each point, represents the mean of the neighborhood color channels of each point, represents the variance of the neighborhood color channel of each point, Take 1, 2, 3, the self-learning process is set and Each feature channel corresponds to the feature distance calculation, and n represents the index of the k neighboring points of each point.

Furthermore, the step S4 is specifically as follows:

Suppose the original point set input to the graph convolutional network in step S2 is contain points, the coordinate distance is calculated as Euclidean distance, and the color distance is calculated as Manhattan distance;

Coordinate distance loss function for:

Color distance loss function for:

Finally, the loss function is:

in, The original point set The index of each point in and are two hyperparameters. In this graph convolutional network and They are set to 1/3 and 2/3 respectively; the loss function is used to train the graph convolutional network in step S2 to further adjust the parameters of its encoder and decoder.

Furthermore, the step S5 is specifically as follows:

S51, the original point set As the target semantic segmentation dataset ,but For a group containing points, let the original point set The point set with labeled data is , the points are , the point set of unlabeled data is , the points are , then and ;

S52, using the encoder and decoder trained and adjusted in step S4, outputting the The MLP of dimension is replaced by the output dimensional MLP, and the output dimensional vector is denoted by ;

S53, will The features corresponding to the points with labels in are expressed as , the feature corresponding to the unlabeled point is expressed as ;but ;

in, and Both dimensional vectors, and use subscripts to distinguish different points, and Contains categories, 0＜ ≤ , for The actual number of categories that need to be semantically segmented in ;

S54, select the data belonging to the category from the known labeled data point cloud, and calculate the feature average of these points to obtain the average feature vector of this category :

in, Indicates that the category is The number of points, express The corresponding point categories are , and then, the input Calculate the average feature vector of b categories in points ,in ;for The remaining categories that do not exist in It is recorded as zero vector;

S55. Calculate the points of unlabeled data The eigenvector of and The similarity matrix :

in, The Euclidean distance between the average feature vector of the class and the vector corresponding to the unlabeled point, The superscript indicates the category. The subscript of represents a point, and , It represents the base of natural logarithm, and the exponent in brackets is: The dimension is ;

S56: The feature vector in step S53 is Then use MLP to map it into a vector , as the prediction result, The feature dimension is ;

for There are labeled points. The Softmax classifier and cross entropy loss function are used directly to realize category prediction. The loss function calculated for these points is ;

for unlabeled points, generate pseudo labels first, and then use them to Contrast, specifically: first select the similarity matrix category by category The most confident points, assuming that a total of points ( ≤ ≤ ), and then select the category with the highest confidence for each of the selected points, and then update this The maximum confidence of the pseudo-label of each point and its corresponding label value;

Will The prediction loss function of unlabeled points is designed as:

Among them, the subscript express Any point among the points, s represents The indices of the unlabeled points, for The number of categories contained in, m represents the index of the number of categories, Represents the probability value of the final predicted label, Represents the pseudo label category. When the pseudo label category and the predicted category are the same, m takes 1, otherwise it takes 0. Indicates that when the point is Take 1 if it is correct, otherwise take 0;

S57. Loss function of the entire graph convolutional network for:

Among them, the weight for:

Among them, epoch represents the current training round, max-epoch represents the maximum training round, and the starting time is Use smaller weights.

Furthermore, the step S6 is specifically as follows:

Iterate the process of assigning pseudo labels in steps S51-S57 to train the network until the target data set When the trained network converges, the similarity matrix is removed during the final prediction. The calculation of Use Softmax classifier on all points in , and then read The remaining points in the target data set can be tested by iterating this test process. Semantic segmentation and category prediction for all points.

After adopting the above technical solution, the present invention has the following advantages compared with the background technology:

1. This paper adopts the idea of transfer learning, makes full use of the similar characteristics of different urban scenes, and uses public annotated data sets to obtain the initialization parameters of the graph convolutional network, which helps to improve the stability of the neural network performance on different data sets;

2. The present invention adopts a self-learning pre-training task, making full use of the local and color characteristics of objects in urban scenes. It can also learn the prior distribution of data to fine-tune network parameters without using labeled data;

3. The present invention uses semi-supervised learning to reduce the dependence on labeled data, so that only a small amount of labeled data is needed to generate high-quality pseudo-labels, improve the effect of semi-supervised learning, and realize semantic segmentation of the target data set, greatly reducing the dependence on manually labeled data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of fine-tuning parameters of a pre-trained graph convolutional network according to the present invention;

FIG2 is a flow chart of the training process of generating pseudo labels by the semi-supervised learning network of the present invention.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

Example

The urban scene semantic segmentation method based on graph convolution and semi-supervised learning network includes the following steps:

(I) Pre-train the graph convolutional network and obtain the initialization parameters (achieved by following the steps S1 below)

S1. Use a public and annotated dataset to pre-train the graph convolutional network to obtain the initialization parameters of each layer in the graph convolutional network;

The encoder of the graph convolution network described in the present invention is composed of a multi-layer perceptron layer and four graph convolution modules, and the four graph convolution modules are numbered 1, 2, 3, and 4 in sequence; the input feature of the graph convolution module is represented as The output features are ,in , that is, the output of the previous graph convolution module is the input of the next graph convolution module, and the number of input points of the current module is , the input feature dimension is , the feature dimension after convolution is , and then a random downsampling is performed to reduce the number of points to .

This example uses public and annotated urban road data as a pre-training dataset for pre-training. The public dataset Toronto3D uses a vehicle-mounted LiDAR to obtain a 1km high-quality urban scene point cloud. The dataset has more than 80 million points manually annotated, covering a total of 8 common urban scene categories: roads, zebra crossings, buildings, power lines, power towers, cars, and fences, all of which contain coordinate and color information. Select a point cloud in Toronto3D As the input of the pre-trained graph convolutional network, in this embodiment The number of points is set to 65536, then The dimension is [65536, 6].

First, use the encoder to get the encoded features , where MLP (multi-layer perceptron) is used to map [65536, 6] to [65536,16] and then input into four graph convolution modules to extract features; then the decoder is used to obtain the decoded features , whose dimension is [65536,16]; then, a fully connected network and Softmax classifier are used to implement The category prediction of each point in the fully connected network is set to (16, 64, 64, 8) for each point’s feature dimension change; then, cross entropy is used as the loss function and stochastic gradient descent is used for optimization, the network is pre-trained, and the parameters of each layer of the network are updated; finally, the above process is repeated until the network converges. The convergence condition is set to a fixed training of 100 rounds. If the prediction accuracy does not improve after 20 consecutive rounds, the training can also be stopped. The network of this embodiment uses pre-trained parameter initialization instead of random initialization.

(ii) Perform self-learning training tasks and achieve parameter fine-tuning (achieved through the following steps S2-S4)

S2: Input the original point set into the initialized graph convolutional network once , The points in the graph only contain coordinates xyz and color rgb information, and the graph convolutional network is used to output the feature vector ;

S3, the original point set in step S2 Use k-NN for each point, find k neighboring points to form a neighborhood, and calculate the feature vector based on the neighborhood of each point ;

S4. Calculate the feature vector and The distance between the two images is used as a loss function, and the loss function is used to adjust the parameters of the graph convolutional network in step S2;

Wherein, the step S2 is specifically as follows:

Since step S1 has initialized the parameters of each layer of the graph convolutional network, step S2 only uses the encoder and decoder parts. First, modify the fully connected network and Softmax classifier in step S1 to MLP layers, and then perform the following steps.

S21, using the encoder of the graph convolutional network to encode the original point set Encode to get the encoding features ;

In particular, for the original point set input into the graph convolutional network at one time , where the points only contain the 6-dimensional features of coordinates xyz and color rgb, recorded as features ,Will After being mapped to 16 dimensions by a multi-layer perceptron, the output is the feature , and used as the input of the first graph convolution module, and the output of the previous graph convolution module is the input of the next graph convolution module. After four graph convolution modules, the output feature , which is the final encoding feature .

S22, then use the decoder of the graph convolutional network to encode the features Decode to get the decoding features ;

The encoding features obtained in S21 Use MLP to perform same-dimensional mapping to obtain features , and used as decoder input to decode features After neighboring point upsampling, MLP dimension reduction and encoder skip connection, the output features are decoded. . The decoding features use the subscript and superscript Different from the characteristics of the encoder, Take 4, 3, 2, and 1 in turn. The encoder and decoder are skipped, that is, the encoded features with the same dimension With decoding features After addition, they are used as input features for subsequent layers. Decoded features Decoding feature .

S23, decode the features through MLP Mapping output as feature vector ; Among them, the eigenvector The dimension of each point in is expressed as , and They represent the encoded coordinates and color features respectively. The subscript of r represents the feature channel. The superscript of r represents the mean and 2 represents the variance.

The step S3 is specifically as follows:

First, the original point set in step S2 For each point, use k-NN to find k neighboring points to form a neighborhood;

Specifically, it is the original point set input to the network Each point in Use k-NN to find a set of nearest neighbors , and then embed the coordinate information:

= LBR ( , , , )

Among them, the coordinate feature By point Its neighboring points The spatial position relationship is obtained, specifically the absolute coordinates connecting the two points and , offset , and spatial distance ; The symbol LBR means that the connected feature vector passes through the Linear layer, BatchNorm layer and ReLU layer in sequence, and the graph convolution module The dimension of is mapped to the same dimension as the input point set feature.

Then, click With neighboring points The relationship is represented as an edge relationship :

= R ( g ( ))

in, , the input Point set features of graph convolution With its coordinate characteristics After the connection, the learnable weight g is used for weighting. g can be implemented using MLP and 1D-CNN, etc. ReLU layer. Finally, Max-Pooling is used to aggregate each point channel by channel. Neighborhood features, and use random sampling to reduce the number of points to get the output features .

Then the eigenvector is calculated based on the neighborhood of each point , where the dimension of each point is expressed as: ;

The calculation process is:

in, represents the mean of the neighborhood coordinate channel of each point, represents the mean of the neighborhood color channels of each point, represents the variance of the neighborhood color channel of each point, Take 1, 2, 3, the self-learning process is set and Each feature channel corresponds to the feature distance calculation, and n represents the index of the k neighboring points of each point. Since the point cloud of urban street scenes is sparsely distributed and uneven, the variance of the neighborhood coordinates is large, so the network only uses the mean of the coordinates. Objects such as vegetation (green) and road surface (gray) have significant color characteristics, and local color changes are usually smooth, so the color uses the mean and variance as the two features.

The step S4 is specifically as follows:

Assume that the original point set input to the graph convolutional network in step S2 is contain points, the coordinate distance is calculated as Euclidean distance, and the color distance is calculated as Manhattan distance;

Coordinate distance loss function for:

Color distance loss function for:

Finally, the loss function is:

in, The original point set The index of each point in and are two hyperparameters. In this graph convolutional network and They are set to 1/3 and 2/3 respectively; the loss function is used to train the graph convolutional network in step S2 to further adjust the parameters of its encoder and decoder.

The above steps S2-S4 implement fine-tuning of the parameters of the pre-trained graph convolutional network. Specifically, in this embodiment,

(1) First, fix the pre-trained encoder and decoder, and change the subsequent fully connected layer of the decoder to a multi-layer perceptron (MLP). The feature dimension of each point in the MLP is set to (16, 32, 9). The target semantic segmentation dataset is Build a point cloud (The construction method is the same as in the previous step of pre-training, only the data set is changed), the point cloud is passed through the network output features of Figure 1 , whose dimension is [65536, 9].

(2) At the same time In the example, k-NN is used to construct the neighborhood for each point, and the number of neighborhood points k is set to 16. The feature calculation method of a point is:

in, Represents the mean of the neighborhood coordinate channel of the point, represents the mean of the neighborhood color channels of the point, represents the variance of the neighborhood color channel of the point. i takes 1, 2, or 3, and the self-learning process is set and Each feature channel corresponds to the feature distance calculation, and n represents the index of the k neighboring points of each point. The feature representation of the point is obtained as:

but Constructed All point features are , whose dimension is [65536, 9].

(3) Calculation and The distance is used as the loss function of the network in Figure 1 and then trained. The features related to the coordinates are calculated as the Euclidean distance, and the feature distance related to the color is calculated as the Manhattan distance. Coordinate distance loss function for:

Color distance loss function for:

Finally, the loss function is:

in, The original point set The index of each point in and are two hyperparameters, set to 1/3 and 2/3.

The training is optimized using stochastic gradient descent and fixed for 30 rounds. This pre-training is used to fine-tune the parameters of the encoder and decoder to make it suitable for The dataset encoding.

(III) Generate pseudo labels and semantic segmentation using a semi-supervised learning network (implemented by following steps S5 and S6)

S5. The original point set As a target semantic segmentation dataset , which contains a small amount of labeled data and a large amount of unlabeled data, where the amount of labeled data accounts for The ratio is 1%~10%, and then the labeled data is used in the semi-supervised learning network to assign pseudo labels to the unlabeled data;

S6: assign pseudo labels to the Used for network inference, semantic segmentation and predicting the category of each point.

The step S5 is specifically as follows:

S51, the original point set As the target semantic segmentation dataset ,but For a group containing points, let the original point set The point set with labeled data is , the points are , the point set of unlabeled data is , the points are , then and ;

S52, using the encoder and decoder trained and adjusted in step S4, outputting the The MLP of dimension is replaced by the output dimensional MLP, and the output dimensional vector is denoted by ;

S53, will The features corresponding to the points with labels in are expressed as , the feature corresponding to the unlabeled point is expressed as ;but ;

in, and Both dimensional vectors, and use subscripts to distinguish different points, and Contains categories, 0＜ ≤ , for The actual number of categories that need to be semantically segmented in ;

S54, select the data belonging to the category from the known labeled data point cloud, and calculate the feature average of these points to obtain the average feature vector of this category :

in, Indicates that the category is The number of points, express The corresponding point categories are , and then, the input Calculate the average feature vector of b categories in points ,in ;for The remaining categories that do not exist in It is recorded as zero vector;

S55. Calculate the points of unlabeled data The eigenvector of and The similarity matrix :

in, The Euclidean distance between the average feature vector of the class and the vector corresponding to the unlabeled point, The superscript indicates the category. The subscript of represents a point, and , It represents the base of natural logarithm, and the exponent in brackets is: The dimension is ;

S56: The feature vector in step S53 is Then use MLP to map it into a vector , as the prediction result, The feature dimension is ;

for There are labeled points. The Softmax classifier and cross entropy loss function are used directly to realize category prediction. The loss function calculated for these points is ;

for unlabeled points, generate pseudo labels first, and then use them to In contrast, if the pseudo-label confidence level is low, the segmentation result will have a large error. Therefore, we can first select the similarity matrix by category. The most confident points, and then select the category with the highest confidence for each of the selected points.

Specifically: First select the similarity matrix category by category The most confident points, assuming that a total of points ( ≤ ≤ ), and then select the category with the highest confidence for each of the selected points, and then update this The maximum confidence of the pseudo-label of each point and its corresponding label value;

Will The prediction loss function of unlabeled points is designed as:

Among them, the subscript express Any point among the points, s represents The indices of the unlabeled points, for The number of categories contained in, m represents the index of the number of categories, Represents the probability value of the final predicted label, Represents the pseudo label category. When the pseudo label category and the predicted category are the same, m takes 1, otherwise it takes 0. Indicates that when the point is Take 1 if it is correct, otherwise take 0;

S57. Loss function of the entire graph convolutional network for:

Among them, the weight for:

Among them, epoch represents the current training round, max-epoch represents the maximum training round, and the starting time is Use smaller weights.

The step S6 is specifically as follows:

Iterate the process of assigning pseudo labels in steps S51-S57 to train the network until the target data set When the trained network converges, the similarity matrix is removed during the final prediction. The calculation of Use Softmax classifier on all points in , and then read The remaining points in the target data set can be tested by iterating this test process. Semantic segmentation and category prediction for all points.

Specifically, in this embodiment, the subsequent layers of the decoder in FIG1 are modified into two MLPs. The first MLP sets the feature dimension change of each point to (16, 32, 32), and outputs the feature , whose dimension is [65536, 32]; the second MLP sets the feature dimension change of each point to (32,32,8), and outputs the feature , whose dimension is [65536, 8]. The modified network architecture is shown in Figure 2.

Target semantic segmentation dataset It needs to contain a small amount of labeled data and a large amount of unlabeled data. In step S5, the labeled data is used to assign pseudo labels to the unlabeled data. Build a point cloud (The construction method is the same as that in the previous self-learning pre-training task). 1%~10% of the points need to have labeled information. For example, in one round of training, there are 4096 labeled points in 65536 points, with a total of 5 categories, and the labeled information accounts for 6.25%. , calculate the average feature value of each category corresponding to these labeled points, and obtain the average feature vector of the category :

in, express A marked point is The corresponding features in are of the following categories: Then, we input 4096 points one by one. Calculate the average feature vector for each category ,in .for The remaining 3 categories that do not exist in Recorded as zero vector.

Next, calculate the unmarked points The eigenvector of and The similarity matrix :

in, , The dimension is .

Next, the feature vector Then use MLP to map it into a vector , as the prediction result, The feature dimension is Among them, 4096 labeled points directly use Softmax classifier and cross entropy loss function to achieve category prediction. There are unlabeled points, and pseudo labels need to be generated for First, select the similarity matrix by category. The most confident points, and then select the category with the highest confidence for each of the selected points. points ( ≤ ≤ ), then update this The maximum confidence of the pseudo-label of the points and its corresponding label value. Among them, Set it to 50% of the number of each category. Set the number of training rounds to 100, and select Adam as the optimization method.

Finally, using the trained network, remove the similarity matrix Calculation, that is, removing the to simi and simi to Then, during the test, the input to the network is All points in the classifier are Softmax, and the Until read All the points in can be used to predict label values for all points and achieve semantic segmentation.

The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (4)

1. A method for semantic segmentation of urban scenes based on graph convolution and semi-supervised learning network, characterized in that it comprises the following steps: S1. Use the public and annotated urban road dataset to pre-train the graph convolutional network to obtain the initialization parameters of each layer in the graph convolutional network; S2: Input the original point set into the initialized graph convolutional network once , The points in the graph only contain coordinates xyz and color rgb information, and the graph convolutional network is used to output the feature vector ; The step S2 is specifically as follows: S21, using the encoder of the graph convolutional network to encode the original point set Encode to get the encoding features ; For the original point set input into the graph convolutional network at one time , where the points only contain the 6-dimensional features of coordinates xyz and color rgb, recorded as features ,Will After an MLP mapping to 16 dimensions, the output is the feature , and used as the input of the first graph convolution module, and the output of the previous graph convolution module is the input of the next graph convolution module. After four graph convolution modules, the output feature , which is the final encoding feature ; S22, then use the decoder of the graph convolutional network to encode the features Decode to get the decoding features ; S23, decode the features through MLP Mapping output as feature vector ; Among them, the eigenvector The dimension of each point in is expressed as , and Represent the encoded coordinates and color features respectively. The subscript of r represents the feature channel. The superscript of r is 1 for the mean and 2 for the variance. S3, the original point set in step S2 Use k-NN for each point, find k neighboring points to form a neighborhood, and calculate the feature vector based on the neighborhood of each point ; S4. Calculate the feature vector and The distance between the two images is used as a loss function, and the loss function is used to adjust the parameters of the graph convolutional network in step S2; The step S4 is specifically as follows: Assume that the original point set input to the graph convolutional network in step S2 is contain points, the coordinate distance is calculated as Euclidean distance, and the color distance is calculated as Manhattan distance; Coordinate distance loss function for: Color distance loss function for: Finally, the loss function is: Among them, ɑ is the original point set The index of each point in and are two hyperparameters. In this graph convolutional network and are set to 1/3 and 2/3 respectively; use this loss function to train the graph convolutional network in step S2 and further adjust the parameters of its encoder and decoder; S5. The original point set As a target semantic segmentation dataset , which contains labeled data and unlabeled data, wherein the amount of labeled data accounts for the original point set The ratio is 1%~10%, and then the labeled data is used in the semi-supervised learning network to assign pseudo labels to the unlabeled data; S6: assign pseudo labels to the Used for network inference, semantic segmentation and predicting the category of each point. 2. The urban scene semantic segmentation method based on graph convolution and semi-supervised learning network according to claim 1, characterized in that: the step S3 is specifically: For the original point set in step S2 For each point in the graph, k-NN is used to find k neighboring points to form a neighborhood, and the feature vector is calculated based on the neighborhood of each point. , where the dimension of each point is expressed as: ; The calculation process is: in, represents the mean of the neighborhood coordinate channel of each point, represents the mean of the neighborhood color channels of each point, represents the variance of the neighborhood color channel of each point, Take 1, 2, 3, self-learning process settings and Corresponding to each feature channel, n represents the index of k neighboring points of each point. 3. The urban scene semantic segmentation method based on graph convolution and semi-supervised learning network according to claim 2, characterized in that: the step S5 is specifically: S51, the original point set As the target semantic segmentation dataset ,but For a group containing points, let the original point set The point set with labeled data is , the points are , the point set of unlabeled data is , the points are , then and ; S52, using the encoder and decoder trained and adjusted in step S4, outputting the The MLP of dimension is replaced by the output dimensional MLP, and the output dimensional vector is denoted by ; S53, will The features corresponding to the points with labels in are expressed as , the feature corresponding to the unlabeled point is expressed as ;but ; in, and Both dimensional vectors, and use subscripts to distinguish different points, and Contains categories, 0＜ ≤ , for The actual number of categories that need to be semantically segmented in ; S54, select the data belonging to the category from the known labeled data point cloud, and calculate the feature average of these points to obtain the average feature vector of this category : in, Indicates that the category is The number of points, express The corresponding point categories are , and then, the input Point Calculate the average feature vector for each category ,in ;for The remaining categories that do not exist in It is recorded as zero vector; S55. Calculate the points of unlabeled data The eigenvector of and The similarity matrix : in, The Euclidean distance between the average feature vector of the class and the vector corresponding to the unlabeled point, The superscript indicates the category. Subscript express Any point among the points, and , It represents the base of natural logarithm, and the exponent in brackets is: The dimension is ; S56: The feature vector in step S53 is Then use MLP to map it into a vector , as the prediction result, The feature dimension is ; for There are labeled points. The Softmax classifier and cross entropy loss function are used directly to realize category prediction. The loss function calculated for these points is ; for unlabeled points, generate pseudo labels first, and then use them to Contrast, specifically: first select the similarity matrix category by category The most confident points, assuming that a total of Points, ≤ ≤ , and then select the category with the highest confidence for each of the selected points, and then update this The maximum confidence of the pseudo-label of each point and its corresponding label value; Will The prediction loss function of unlabeled points is designed as: Among them, the subscript express Any point among the points, s represents The indices of the unlabeled points, for The number of categories contained in, m represents the index of the number of categories, Represents the probability value of the final predicted label, Represents the pseudo label category. When the pseudo label category and the predicted category are the same, m takes 1, otherwise it takes 0. Indicates that when the point is Take 1 if it is correct, otherwise take 0; S57. Loss function of the entire graph convolutional network for: Among them, the weight for: Among them, epoch represents the current training round, max-epoch represents the maximum training round, and the starting time is Use smaller weights. 4. The urban scene semantic segmentation method based on graph convolution and semi-supervised learning network according to claim 3, characterized in that: the step S6 is specifically: Iterate the process of assigning pseudo labels in steps S51-S57 to train the network until the target data set When the trained network converges, the similarity matrix is removed during the final prediction. The calculation of Use Softmax classifier on all points in , and then read The remaining points in the target dataset are tested by iterating this test process. Semantic segmentation and category prediction for all points.