CN116543153A - Semi-supervised point cloud semantic segmentation method based on selective active learning - Google Patents
Semi-supervised point cloud semantic segmentation method based on selective active learning Download PDFInfo
- Publication number
- CN116543153A CN116543153A CN202310497117.4A CN202310497117A CN116543153A CN 116543153 A CN116543153 A CN 116543153A CN 202310497117 A CN202310497117 A CN 202310497117A CN 116543153 A CN116543153 A CN 116543153A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- semantic segmentation
- point
- data set
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000011218 segmentation Effects 0.000 title claims abstract description 76
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000012549 training Methods 0.000 claims abstract description 36
- 238000013528 artificial neural network Methods 0.000 claims abstract description 23
- 230000008569 process Effects 0.000 claims description 15
- 239000004973 liquid crystal related substance Substances 0.000 claims description 10
- 230000001629 suppression Effects 0.000 claims description 8
- 238000012546 transfer Methods 0.000 claims description 5
- 230000005764 inhibitory process Effects 0.000 claims description 3
- 230000002401 inhibitory effect Effects 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 5
- 238000012545 processing Methods 0.000 abstract description 2
- 238000005070 sampling Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 239000003550 marker Substances 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000007786 learning performance Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/0895—Weakly supervised learning, e.g. semi-supervised or self-supervised learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the field of point cloud data processing, and particularly relates to a semi-supervised point cloud semantic segmentation method based on selective active learning; the segmentation method comprises the steps of obtaining target point cloud data; inputting the target point cloud data into a trained point cloud semantic segmentation neural network, and outputting a segmentation result of the target point cloud data; the marking cost of the point cloud data is reduced through the semi-supervised training point cloud semantic segmentation neural network; by actively learning the selection points, uncertain point selection is performed from an unlabeled point cloud data set through a specific sampling strategy, so that the selected point cloud data is important, contains rich information and can also be non-redundant. The method can improve the semi-supervised learning effect and the model capacity, and finally improve the segmentation performance of the three-dimensional point cloud data.
Description
Technical Field
The invention belongs to the field of point cloud data processing, and particularly relates to a semi-supervised point cloud semantic segmentation method based on selective active learning.
Background
The purpose of the point cloud semantic segmentation is to assign a class label to each 3D point, which can be applied to various scenes such as robots, autopilots and augmented reality. Recently, deep learning-based approaches have achieved impressive performance. These high performance methods typically rely on large amounts of data with point-level tags. However, this also causes problems of high cost and mark redundancy.
To reduce the labor and cost of annotation, many methods have been proposed to explore semi-supervised ways to learn three-dimensional segmentation models of finite marker points, which are typically randomly sampled. These approaches attempt to develop efficient strategies to propagate tag information to untagged points. While these semi-supervised approaches can greatly reduce tag costs, performance may be limited. Since the marked points are randomly selected, the tag may contain redundant information, while important points may be omitted.
Active learning has recently been studied as an alternative learning strategy to alleviate the limitations of three-dimensional segmentation. Lin et al divide the entire point cloud into segments, each segment being the basic query unit of sample selection. The ReDAL suggests selecting those informative and diverse sub-scene regions for tag acquisition. Entropy, color discontinuities and structural complexity are used to measure information of sub-scene areas. Subsequently, the SSDR-AL groups the original point cloud into super points and progressively selects the most informative regions for annotation. However, the performance of region-based active learning is largely dependent on region-partitioning strategies. Furthermore, since the point cloud exhibits strong semantic similarity in the local region, selecting all points in the local region creates a redundant label budget.
In summary, although the independent use of the semi-supervised training and the active learning in the point cloud semantic segmentation task also achieves good results, how to effectively combine the semi-supervised training and the active learning and fully exert the advantages of the semi-supervised training and the active learning is a problem to be solved by the technicians in the field.
Disclosure of Invention
Based on the problems existing in the prior art, the invention provides a semi-supervised point cloud semantic segmentation method based on selective active learning, which comprises the steps of acquiring target point cloud data; inputting the target point cloud data into a trained point cloud semantic segmentation neural network, and outputting a segmentation result of the target point cloud data;
the training process of the point cloud semantic segmentation neural network specifically comprises the following steps:
101. acquiring a point cloud data set; the point cloud data set comprises a marked data set and an unmarked data set, the marked data set and the unmarked data set have the same distribution, each point cloud data in the marked data set has a label, and each point cloud data in the unmarked data set does not have a label;
102. training a point cloud semantic segmentation model in a semi-supervised mode by adopting a marked data set and an unmarked data set, and predicting to obtain a pseudo tag of unmarked data;
103. calculating the minimum margin score of the point cloud according to the point cloud data labels in the point cloud data set by adopting the layered minimum margin uncertainty measure; gradually capturing local contexts on different scales according to multiple downsampling, and calculating the context uncertainty score of the point cloud;
104. calculating an uncertainty result of the point cloud by adopting the minimum marginal score and the context uncertainty score of the point cloud;
105. and selecting a plurality of point clouds based on the uncertainty result of the point clouds, marking the selected plurality of point clouds, sending the marked point clouds into a marked data set, removing the marked point clouds from the unmarked data set, and returning to the step 101 until the training of the point cloud semantic segmentation model is completed.
The invention has the beneficial effects that:
the invention realizes the integration of a semi-supervised and improved active learning method, solves the problem that the learning performance of a semi-supervised model is limited by random point selection under the original semi-supervised framework, and provides a novel hierarchical-characteristic distance inhibition point selection active learning point selection strategy, which solves the problems of information redundancy and dependence on region division.
Drawings
FIG. 1 is a flow chart of a semi-supervised point cloud semantic segmentation method based on selective active learning according to an embodiment of the invention;
FIG. 2 is a schematic diagram of a point cloud semantic segmentation neural network according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a training process of a point cloud semantic segmentation neural network according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a semi-supervised point cloud semantic segmentation effect according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Fig. 1 is a schematic flow chart of a semi-supervised point cloud semantic segmentation method based on selective active learning, and as shown in fig. 1, the flow chart of the segmentation method operation of the invention is shown, and specifically comprises the following steps:
s1, acquiring cloud data of a target point;
in the embodiment of the invention, the target point cloud data is a point cloud data set to be segmented, the category to which each point cloud data in the set belongs needs to be marked, and the target point cloud data is segmented according to the category. The point cloud data set to be segmented may be acquired, for example, by an acquisition device on an autonomous vehicle.
S2, inputting the target point cloud data into a trained point cloud semantic segmentation neural network, and outputting a segmentation result of the target point cloud data;
in the embodiment of the invention, the problems of high cost and mark redundancy are brought about by relying on a large amount of data with point-level labels in the existing semantic segmentation neural network, so the label cost is reduced by adopting the semi-supervised point cloud semantic segmentation neural network, the semi-supervised point cloud semantic segmentation neural network is trained in advance by using a deep learning technology, the prediction type and the prediction probability of each point cloud data can be predicted by using the semi-supervised point cloud semantic segmentation neural network, wherein the prediction probability can represent the confidence of the corresponding prediction type, and the segmentation condition of the target point cloud data can be accurately obtained through the prediction probability and the prediction type.
In order to better explain the semi-supervised point cloud semantic segmentation neural network, the semi-supervised point cloud semantic segmentation neural network and the corresponding training process thereof are detailed.
Fig. 2 is a schematic diagram of a point cloud semantic segmentation neural Network according to an embodiment of the present invention, as shown in fig. 2, where a Teacher-Student model is adopted in the semi-supervised point cloud semantic segmentation neural Network, specifically, two segmentation networks are constructed by using a minkowski space according to an embodiment of the present invention, and are respectively represented as a Teacher model (Teacher Network) and a Student model (Student Network). Inputting point cloud data of a point cloud data set into a teacher model, inputting enhancement data of the point cloud data set into a student model, and training the teacher model and the student model by using consistency constraint between the teacher model and the student model and adopting cross entropy loss; optimizing a student network through gradient descent; and transferring the parameters of the student model to the teacher model by using the index moving average index.
The formula for transferring the parameters of the student model to the teacher model by using the index moving average index is expressed as:
wherein, the liquid crystal display device comprises a liquid crystal display device,is the parameters of the teacher model in the jth training iteration,>is the parameters of the teacher model in the j-1 th training iteration process, < ->Is a parameter of the student model in the jth training iteration process; alpha is a super parameter that determines the rate of parameter transfer, typically approaching 1. In each iteration process, the teacher model outputs a predicted label of the marked data and a pseudo label of the unmarked data, and then the label output result of the model can be used for sampling and selecting the unmarked points so as to learn the better marked data to optimize the point cloud semantic segmentation neural network in an active learning mode.
Fig. 3 is a schematic diagram of a training process of a point cloud semantic segmentation neural network according to an embodiment of the present invention, as shown in fig. 3, where the training process includes:
101. acquiring a point cloud data set;
in consideration of the cost problem and redundancy problem of the point cloud data marking, the method and the device select better point cloud data to mark in an active learning mode to ensure the precision of the point cloud semantic segmentation neural network, wherein the point cloud data set comprises a marked data set and an unmarked data set, the marked data set and the unmarked data set have the same distribution, each point cloud data in the marked data set is provided with a label, and each point cloud data in the unmarked data set is not provided with a label;
102. training a point cloud semantic segmentation model in a semi-supervised mode by adopting a marked data set and an unmarked data set, and predicting to obtain a pseudo tag of unmarked data;
in the embodiment of the invention, the point cloud data in the marked data set and the unmarked data set are respectively input into a teacher model, the enhancement data of the point cloud data in the marked data set and the unmarked data set are input into a student model, and the teacher model and the student model are trained by using consistency constraint between the teacher model and the student model and adopting cross entropy loss; optimizing a student network through gradient descent; and transferring the parameters of the student model to the teacher model by adopting an index moving average index, wherein in each training process, the teacher model outputs a predicted label of marked data and a pseudo label of unmarked data, and because the marked data has a real label, the unmarked data does not have a real label, and the real label of the marked data has higher effectiveness, the more important unmarked data is selected by utilizing the real label of the marked data and the pseudo label of the unmarked data in the subsequent process, and the next round of training iteration is carried out after the selected unmarked data is manually marked until the corresponding training requirement is met.
In some embodiments of the invention, semi-supervised learning refers to machine learning methods that utilize labeled and unlabeled data for training. The whole semi-supervised training process of this embodiment can be reported as follows:
1. preparing a data set: and respectively preparing the tagged point cloud data and the untagged point cloud data, and dividing the tagged point cloud data and the untagged point cloud data into a training set and a verification set according to a certain proportion.
2. Building a Teacher network: and training a deep neural network by using the tagged point cloud data to serve as a Teacher network. The training goal of this network is to minimize the loss function of tagged data, i.e., to minimize the gap between predicted and true tag values.
3. Building a Student network: and training a deep neural network by using the tagged point cloud data and the untagged point cloud data to serve as a Student network. When training the Student network, the output result of the Teacher network is used as the label of the label-free data, so that the label-free data is utilized.
4. Training Student network: and iteratively adjusting parameters of the Student network to reduce the loss function of all data as much as possible, wherein the loss function comprises tagged point cloud data and untagged point cloud data. Specifically, in each iteration period, the parameters of the Student network are updated by using the labeled point cloud data, and then the parameters are updated by using the unlabeled point cloud data. The loss function of the unlabeled point cloud data is formed by the difference between the prediction result of the Student network and the output result of the Teacher network.
5. Evaluating model performance: after each iteration cycle is completed, the validation set is used to evaluate the performance of the Student network. If the performance meets the expected requirement, stopping training; otherwise, continuing to adjust parameters of the Student network until the performance requirement is met.
Through the steps, the Teacher-Student model can fully utilize the unlabeled point cloud data to improve the learning effect, so that better performance is obtained under the same labeled point cloud data, and the segmentation performance of the point cloud semantic segmentation is improved in the subsequent process.
103. Calculating the minimum margin score of the point cloud according to the point cloud data labels in the point cloud data set by adopting the layered minimum margin uncertainty measure; gradually capturing local contexts on different scales according to multiple downsampling, and calculating the context uncertainty score of the point cloud;
in the embodiment of the invention, the existing mode of performing the point cloud segmentation task by using active learning is generally based on the region, and the performance of the active learning based on the region depends on the region division strategy to a great extent. Furthermore, since the point cloud exhibits strong semantic similarity in the local region, selecting all points in the local region creates a redundant label budget. Therefore, on the active learning adopted by the invention, the invention predicts uncertainty of the point itself and neighbor points around the point through the hierarchical minimum marginal uncertainty measurement, and gradually captures local contexts on different scales after a plurality of downsampling. And accumulating the information and uncertainty of the neighbor points every time of downsampling to obtain a final uncertainty result.
For the hierarchical minimum marginal uncertainty metric, in particular, a point without surrounding context information cannot reflect the true importance of that point. The hierarchical minimum marginal uncertainty metric calculates an uncertainty score for each point by grouping points on multiple scales and progressively perceiving the context information of the unlabeled points over a larger range of levels. The minimum margin score U of each unlabeled point containing only self information can be calculated by the following formula x 。
U x ＝h(x u ；p 1 (x u ))-h(x u ；p 2 (x u ))
Wherein U is x Representing a point cloud x u X, the minimum margin fraction of (2) u Representing candidate point clouds that can be selected for tagging; p is p 1 (x u ) Tag class probability p representing highest score obtained by predicting point cloud semantic segmentation model 2 (x u ) The method comprises the steps that a point cloud semantic segmentation model is represented to predict to obtain label category probability with high score; h (·) represents the segmentation predictor of the point cloud semantic segmentation model.
In addition to the minimum mark fraction, we need to obtain point cloud context uncertainty, we group wider neighbors by downsampling. The flexible maximum number transfer parameter (softmax) predictions for each downsampled point are obtained by averaging the flexible maximum number transfer parameter (softmax) predictions for neighboring points in the original point cloud. The flexible maximum transfer parameter (softmax) label for each downsampled point represents the predicted distribution of a local region, we useAnd (3) representing. We do voxel downsampling of N layers, to-be-marked point x u Can be represented by a downsampled prediction average of the i-th layer, as shown in the following formula:
wherein, the liquid crystal display device comprises a liquid crystal display device,representing point x u Local context information at the i-th downsampling layer; />Indicated at unlabeled point x u A j-th neighbor point at the i-th downsampling layer; k (K) i Is the total number of neighbor points in the voxel radius in the ith downsampling layer, p (·) represents the predictive probability. For downsampling of each layer, the point x is not marked u Context uncertainty score of +.>Can be obtained by the following formula:
wherein, the liquid crystal display device comprises a liquid crystal display device,representing a point cloud x u A context uncertainty score at the ith downsampling; />Predicting the point cloud semantic segmentation model to obtain local context information corresponding to the ith downsampling score highest, and performing +_f>And predicting the point cloud semantic segmentation model to obtain local context information corresponding to the ith downsampling score.
105. Calculating an uncertainty result of the point cloud by adopting the minimum marginal score and the context uncertainty score of the point cloud;
in the embodiment of the invention, the unlabeled point x is integrated u Point level uncertainty score U x And its contextual uncertainty scoreObtaining a final uncertainty fraction v u Expressed as:
wherein v is u Representing a point cloud x u Is the uncertainty result of (U) x Representing a point cloud x u Is the minimum marginal fraction of (2)，Representing a point cloud x u Context uncertainty score, x, at ith downsampling u Representing candidate point clouds that can be selected for tagging; w (w) i Indicating the super-parameter of the ith downsampling, N being the number of downsampling times, for example, w when n=3 i E {0.1,0.01,0.001} is a superparameter whose value from left to right represents the 1 st, 2 nd, 3 rd downsampling is w 1 ，w 2 ，w 3 Is a value of (a).
106. And selecting a plurality of point clouds based on the uncertainty result of the point clouds, marking the selected plurality of point clouds, sending the marked point clouds into a marked data set, removing the marked point clouds from the unmarked data set, and returning to the step 101 until the training of the point cloud semantic segmentation model is completed.
In the embodiment of the invention, after the layering minimum marginal uncertainty measurement is carried out, we obtain a ranking of final uncertainty, and at this time, the first K points can be directly selected for marking according to the ranking condition of the uncertainty result of the point cloud.
In the preferred embodiment of the present invention, we have devised a feature distance suppression method to ensure that the final selected marker points are evenly distributed, considering that this would result in redundancy of information since some of the top K points would be from the same local area. The characteristic distance suppression mode comprises the steps of determining a distance suppression radius and a characteristic similarity threshold; judging whether other selected point clouds exist in the distance inhibition radius of the candidate point cloud, and if the other selected point clouds do not exist, taking the candidate point cloud as the selected point cloud; and if other selected point clouds exist, the similarity distance between the candidate point clouds and the other selected point clouds is set, if the similarity distance exceeds a characteristic similarity threshold, the candidate point clouds are used as redundant points, the redundant points are restrained, and if the similarity distance does not exceed the characteristic similarity threshold, the candidate point clouds are used as the selected point clouds.
Specifically, for each selected point x i We give a distance-suppressing radius r and aA threshold τ of feature similarity, first, we determine if there are other selected points within the distance-suppressing radius of the selected point, D i Is a set initialized as an empty set for storing all x i Distance from point x in the radius of suppression which has been selected j The formula is as follows:
wherein d is ij Representative is x i To x j Is a distance of (3).
If it is judged that D i Still empty set, illustrated at x i No other selected point in the distance-suppressed radius, then x i This local range will be represented and marked as such in the next step. Of course, if D i Not empty, then we need to judge D i Points x and x in (a) i The formula of the similarity is as follows:
wherein f i And f j Respectively represent x i And x j If Sim (x) i ,x j )>τ,x i Will be considered redundant points and will be removed from the set of points that need to be marked.
Finally, when we go through feature distance suppression, we output the points that we need to mark finally, then we can mark these points manually or otherwise, the marked points are marked points, and these marked points will be replaced into the dataset and update the marked dataset portion and the unmarked dataset portion. The next iteration may then be performed until a preset termination condition is met.
Fig. 4 is a schematic diagram of a semi-supervised point cloud semantic segmentation effect according to an embodiment of the present invention, as shown in fig. 4, and is an effect diagram comparing the segmentation result of the full-supervision method on the test set of the S3DIS region-5. The method of the present invention gives comparable or even better results than the fully supervised baseline (MinkowskiNet) when only 0.43% of the marker data is used.
Those of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be implemented by a program to instruct related hardware, the program may be stored in a computer readable storage medium, and the storage medium may include: ROM, RAM, magnetic or optical disks, etc.
Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
Claims (8)
1. The semi-supervised point cloud semantic segmentation method based on selective active learning is characterized by comprising the steps of acquiring target point cloud data; inputting the target point cloud data into a trained point cloud semantic segmentation neural network, and outputting a segmentation result of the target point cloud data;
the training process of the point cloud semantic segmentation neural network specifically comprises the following steps:
101. acquiring a point cloud data set; the point cloud data set comprises a marked data set and an unmarked data set, the marked data set and the unmarked data set have the same distribution, each point cloud data in the marked data set has a label, and each point cloud data in the unmarked data set does not have a label;
102. training a point cloud semantic segmentation model in a semi-supervised mode by adopting a marked data set and an unmarked data set, and predicting to obtain a pseudo tag of unmarked data;
103. calculating the minimum margin score of the point cloud according to the point cloud data labels in the point cloud data set by adopting the layered minimum margin uncertainty measure; gradually capturing local contexts on different scales according to multiple downsampling, and calculating the context uncertainty score of the point cloud;
104. calculating an uncertainty result of the point cloud by adopting the minimum marginal score and the context uncertainty score of the point cloud;
105. and selecting a plurality of point clouds based on the uncertainty result of the point clouds, marking the selected plurality of point clouds, sending the marked point clouds into a marked data set, removing the marked point clouds from the unmarked data set, and returning to the step 101 until the training of the point cloud semantic segmentation model is completed.
2. The semi-supervised point cloud semantic segmentation method based on selective active learning of claim 1, wherein the point cloud semantic segmentation neural network comprises two point cloud semantic segmentation networks which are respectively represented as a teacher model and a student model and are constructed through a minkowski space; inputting point cloud data of a point cloud data set into a teacher model, inputting enhancement data of the point cloud data set into a student model, and training the teacher model and the student model by using consistency constraint between the teacher model and the student model and adopting cross entropy loss; optimizing a student network through gradient descent; and transferring the parameters of the student model to the teacher model by using the index moving average index.
3. The semi-supervised point cloud semantic segmentation method based on selective active learning of claim 2, wherein the formula for transferring the parameters of the student model to the teacher model by using the index moving average index is expressed as:
wherein, the liquid crystal display device comprises a liquid crystal display device,is the parameters of the teacher model in the jth training iteration,>is the parameters of the teacher model in the j-1 th training iteration process, < ->Is a parameter of the student model in the jth training iteration process; alpha is a super parameter that determines the parameter transfer rate.
4. The semi-supervised point cloud semantic segmentation method based on selective active learning of claim 1, wherein the minimum marginal score of the point cloud is represented as:
U x ＝h(x u ；p 1 (x u ))-h(x u ；p 2 (x u ))
wherein U is x Representing a point cloud x u X, the minimum margin fraction of (2) u Representing candidate point clouds that can be selected for tagging; p is p 1 (x u ) Tag class probability p representing highest score obtained by predicting point cloud semantic segmentation model 2 (x u ) The method comprises the steps that a point cloud semantic segmentation model is represented to predict to obtain label category probability with high score; h (·) represents the segmentation predictor of the point cloud semantic segmentation model.
5. The semi-supervised point cloud semantic segmentation method based on selective active learning of claim 1, wherein the point cloud context uncertainty score is expressed as:
wherein, the liquid crystal display device comprises a liquid crystal display device,representing a point cloud x u Context uncertainty score, x, at ith downsampling u Representing candidate point clouds that can be selected for tagging; />Predicting the point cloud semantic segmentation model to obtain local context information corresponding to the ith downsampling score highest, and performing +_f>The method comprises the steps of representing a point cloud semantic segmentation model to predict and obtain local context information corresponding to the ith downsampling score; h (·) represents the segmentation predictor of the point cloud semantic segmentation model.
6. The semi-supervised point cloud semantic segmentation method based on selective active learning of claim 1, wherein the uncertainty result of the point cloud is expressed as:
wherein v is u Representing a point cloud x u Is the uncertainty result of (U) x Representing a point cloud x u Is set to be a minimum margin score of (c),representing a point cloud x u Context uncertainty score, x, at ith downsampling u Representing candidate point clouds that can be selected for tagging; w (w) i Indicating the super-parameter of the ith downsampling, N is the downsampling frequency.
7. The semi-supervised point cloud semantic segmentation method based on selective active learning of claim 1, wherein in step 105, the selecting a plurality of point clouds based on the uncertainty result of the point clouds includes selecting a plurality of top-ranked point clouds according to the ranking of the uncertainty result of the point clouds; or selecting a plurality of point clouds with different similarity by adopting a characteristic distance suppression mode.
8. The semi-supervised point cloud semantic segmentation method based on selective active learning of claim 7, wherein the feature distance suppression mode comprises determining a distance suppression radius and a feature similarity threshold; judging whether other selected point clouds exist in the distance inhibition radius of the candidate point cloud, and if the other selected point clouds do not exist, taking the candidate point cloud as the selected point cloud; and if other selected point clouds exist, calculating the similarity distance between the candidate point clouds and the other selected point clouds, if the similarity distance exceeds a characteristic similarity threshold, taking the candidate point clouds as redundant points, and inhibiting the redundant points, and if the similarity distance does not exceed the characteristic similarity threshold, taking the candidate point clouds as the selected point clouds.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310497117.4A CN116543153A (en) | 2023-05-05 | 2023-05-05 | Semi-supervised point cloud semantic segmentation method based on selective active learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310497117.4A CN116543153A (en) | 2023-05-05 | 2023-05-05 | Semi-supervised point cloud semantic segmentation method based on selective active learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116543153A true CN116543153A (en) | 2023-08-04 |
Family
ID=87455452
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310497117.4A Pending CN116543153A (en) | 2023-05-05 | 2023-05-05 | Semi-supervised point cloud semantic segmentation method based on selective active learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116543153A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117994527A (en) * | 2024-04-03 | 2024-05-07 | 中国空气动力研究与发展中心低速空气动力研究所 | Point cloud segmentation method and system based on region growth |
-
2023
- 2023-05-05 CN CN202310497117.4A patent/CN116543153A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117994527A (en) * | 2024-04-03 | 2024-05-07 | 中国空气动力研究与发展中心低速空气动力研究所 | Point cloud segmentation method and system based on region growth |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210390458A1 (en) | Data analytics methods for spatial data, and related systems and devices | |
| CN111052128B (en) | Descriptor learning method for detecting and locating objects in video | |
| CN108629367A (en) | A method of clothes Attribute Recognition precision is enhanced based on depth network | |
| CN111047078B (en) | Traffic characteristic prediction method, system and storage medium | |
| CN112990222B (en) | Image boundary knowledge migration-based guided semantic segmentation method | |
| CN110378911B (en) | Weak supervision image semantic segmentation method based on candidate region and neighborhood classifier | |
| CN109447096B (en) | Glance path prediction method and device based on machine learning | |
| CN113065013B (en) | Image annotation model training and image annotation method, system, equipment and medium | |
| Zheng et al. | A continual learning framework for uncertainty-aware interactive image segmentation | |
| CN113109816B (en) | Echo block tracking method, device and storage medium of radar echo image | |
| CN114898121A (en) | Concrete dam defect image description automatic generation method based on graph attention network | |
| CN115410059B (en) | Remote sensing image part supervision change detection method and device based on contrast loss | |
| CN113569852A (en) | Training method and device of semantic segmentation model, electronic equipment and storage medium | |
| Demertzis et al. | Geo-AI to aid disaster response by memory-augmented deep reservoir computing | |
| CN118196410A (en) | Remote sensing image semantic segmentation method, system, equipment and storage medium | |
| CN116310647A (en) | Labor insurance object target detection method and system based on incremental learning | |
| Apeagyei et al. | Evaluation of deep learning models for classification of asphalt pavement distresses | |
| US11989656B2 (en) | Search space exploration for deep learning | |
| CN110705631B (en) | SVM-based bulk cargo ship equipment state detection method | |
| CN115239034B (en) | Method and system for predicting early defects of wind driven generator blade | |
| Ridhovan et al. | Disease detection in banana leaf plants using densenet and inception method | |
| CN114821248B (en) | Point cloud understanding-oriented data active screening and labeling method and device | |
| CN116543153A (en) | Semi-supervised point cloud semantic segmentation method based on selective active learning | |
| CN116189130A (en) | Lane line segmentation method and device based on image annotation model | |
| CN115984640A (en) | Target detection method, system and storage medium based on combined distillation technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |