CN117572457A - Cross-scene multispectral point cloud classification method based on pseudo tag learning - Google Patents
Cross-scene multispectral point cloud classification method based on pseudo tag learning Download PDFInfo
- Publication number
- CN117572457A CN117572457A CN202410061674.6A CN202410061674A CN117572457A CN 117572457 A CN117572457 A CN 117572457A CN 202410061674 A CN202410061674 A CN 202410061674A CN 117572457 A CN117572457 A CN 117572457A
- Authority
- CN
- China
- Prior art keywords
- scene
- target domain
- multispectral
- domain
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 39
- 239000011159 matrix material Substances 0.000 claims abstract description 66
- 238000001228 spectrum Methods 0.000 claims abstract description 22
- 238000012549 training Methods 0.000 claims description 18
- 230000009466 transformation Effects 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 6
- 230000003595 spectral effect Effects 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 2
- 230000017105 transposition Effects 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 3
- 238000012800 visualization Methods 0.000 description 3
- RTAQQCXQSZGOHL-UHFFFAOYSA-N Titanium Chemical compound [Ti] RTAQQCXQSZGOHL-UHFFFAOYSA-N 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
- G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
- G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
- G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/58—Extraction of image or video features relating to hyperspectral data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
- G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biomedical Technology (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Remote Sensing (AREA)
- Molecular Biology (AREA)
- Radar, Positioning & Navigation (AREA)
- Computer Networks & Wireless Communication (AREA)
- Databases & Information Systems (AREA)
- Electromagnetism (AREA)
- Biodiversity & Conservation Biology (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Radar Systems Or Details Thereof (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a cross-scene multispectral point cloud classification method based on pseudo tag learning, and belongs to the technical field of multispectral laser radar point clouds. The method comprises the following steps: 1) Respectively characterizing the multispectral laser radar point clouds of the source domain scene and the target domain scenePre-alignment of row features; 2) Respectively extracting graph features of two scenes; 3) Calculating loss; 4) Iteratively performing 3) updating the source domain-target domain alignment network parameters until the model converges to obtain a pseudo tag of the target domain and the confidence coefficient thereof; 5) Setting threshold value for pseudo tags in descending orderαBefore selectingα% of pseudo tags; 6) Splicing the adjacent matrix and the feature matrix in the target domain to obtain a new feature matrix; 7) Calculating loss according to the pseudo tag obtained in the step 5) and the feature matrix obtained in the step 6); 8) And 7) updating parameters in the step until the model converges, and finally obtaining a target domain multispectral point cloud data classification result. The method can realize high-precision classification of multi-spectrum point cloud of the cross-scene.
Description
Technical Field
The invention relates to a cross-scene multispectral point cloud classification method based on pseudo tag learning, and belongs to the technical field of multispectral laser radar point clouds.
Background
The multispectral LiDAR system can synchronously acquire three-dimensional space distribution information and spectrum information in a scene, and can provide richer characteristic information for remote sensing scene interpretation tasks. In the related processing task of multispectral LiDAR, most classification methods, especially those based on deep learning, require a large number of training data sets to achieve optimal performance. However, collecting and marking a large number of point clouds is often laborious and time consuming. On the other hand, they are only applicable to fixed scenarios, i.e. the training samples and the test samples are independent and co-distributed. The performance may be significantly degraded when applied to strange scenes. Thus, these methods cannot be directly transferred to other scenarios, nor can they be tested on unlabeled data collected in real-time. This has been the primary limiting factor in multispectral LiDAR data interpretation.
When multispectral LiDAR carries out data acquisition to remote sensing scene, multiple factors such as laser pulse emission angle, ground object spatial distribution, season and weather change all can influence the intensity of receiving laser pulse, produce spectral drift phenomenon promptly. Furthermore, either conventional methods or deep learning-based methods have poor scene adaptation, and performance is significantly degraded when there is a distribution difference between the training sample and the test sample. Obviously, the multispectral point cloud has space geometric information and spectrum information of ground objects, and the multispectral point cloud pseudo-labels of the target field scene are guided to be generated with high precision by learning the space geometric-spectrum consistency information of the intrinsic attribute of the Xi Biaozheng ground objects from the multispectral point cloud of the source field scene, and the target field pseudo-label training network is adopted, so that the performance of the multispectral point cloud ground object classification network in the target field scene can be improved, and the scene self-adaption capability of the network is improved. Therefore, how to generate high-precision target domain scene point cloud pseudo tags under the conditions of multispectral point cloud spectral drift, inconsistent ground feature distribution and the like in different scenes and realize cross-scene multispectral point cloud high-precision classification under the condition that no target domain scene real tags exist is a technical problem to be solved at present.
Disclosure of Invention
The invention aims to solve the technical problem of providing a cross-scene multispectral point cloud classification method based on pseudo tag learning, so as to cope with spectrum drift phenomenon of multispectral laser radar point clouds among different scenes, alleviate the problem of difficult cross-scene multispectral laser radar point cloud classification caused by spectrum drift phenomenon, and realize high-precision cross-scene multispectral point cloud classification under the condition of no real tags of a target domain scene.
The technical scheme of the invention is as follows: a cross-scene multispectral point cloud classification method based on pseudo tag learning comprises the following steps:
step1: respectively enabling the multispectral laser radar point cloud characteristics of the tagged source domain scene and the untagged target domain scene to be according to L 2 Performing feature pre-alignment on the norms and the Laplace matrix;
step2: respectively extracting graph features of two scenes by adopting a graph convolution neural network (Graph Convolution Neural Networks, GCN) according to the pre-aligned features;
step3: calculating source domain classification loss, maximum mean difference (Maximum Mean Discrepancy, MMD) loss and target domain shannon entropy loss according to the extracted graph characteristics and source domain labels of the two scenes;
step4: iteratively performing Step3, updating the source domain-target domain alignment network parameters, judging whether the model is converged, if yes, ending, then performing Step5, otherwise repeating Step3 to obtain a pseudo tag of the target domain and the confidence coefficient thereof;
step5: according to the confidence level, the pseudo labels are arranged in a descending order, a threshold value alpha is set, and the target domain pseudo labels with the alpha percent before are selected to be used as the true value input of the target domain classification network;
step6: splicing the adjacent matrix and the feature matrix in the target domain to obtain a new feature matrix as the feature input of the target domain classification network;
step7: calculating the classification loss of the target domain according to the pseudo tag selected by Step5 and the new feature matrix obtained by Step 6;
step8: and (3) iteratively performing Step7, updating the target domain classification network parameters, judging whether the model is converged, if yes, ending, otherwise repeating Step7, and finally obtaining a target domain multispectral point cloud data classification result.
Specifically, in Step1, the labeled source domain scene multispectral lidar point cloud data is denoted as (P s Y), the unlabeled target domain scene is denoted (P t ，) WhereinRepresenting a source domain scene contains N s Each of which has a plurality of spectral points of the label,representing the ith labeled multispectral point in the source domain scene,respectively representing that the target domain scene contains N t A single non-labeled multi-spectral point,representing the i-th unlabeled multispectral point in the target domain scene,truth value label corresponding to multispectral points of scene representing all source domains，And (5) representing a truth value label corresponding to the ith multispectral point in the source domain scene.
Specifically, in Step1, the method is described as L 2 The specific steps of feature pre-alignment of the norm and the Laplace matrix are as follows:
(1) Through L 2 The norm carries out characteristic transformation on the characteristics of the source domain and the target domain, and a specific characteristic transformation formula is as follows:
where x is the source domain, target domain characteristics,is the source domain and target domain characteristics after the characteristic transformation,is 2 norms.
(2) Obtaining source domain features of M dimensions according to the formula of the step (1)And M-dimensional target domain featuresWill beAndsplicing to obtain an overall feature matrix with M dimensionCalculating the overall feature matrix according to the K nearest neighbor algorithmFurther computing a diagonal matrix D, the elements of the diagonal matrix D，For elements in the adjacency matrix W, then the laplace matrix l=d-W, so the final overall feature matrix X is updated according to the following formula:
wherein,in order to update the feature matrix after the update, T n for matrix transpose operation s The number of the multispectral points with labels for the source domain scene is N t The number of unlabeled multispectral points in the target domain scene.
Specifically, step3 is specifically:
respectively marking the source domain scene and the target domain scene map features extracted from Step2 asAndthe source domain classification loss calculation formula is:
wherein,is the label of the i-th point in the source domain scene,is the predictive label of the i-th point in the source domain scene,is a source domain scene tag set, N s The number of the multi-spectrum points is the number of the labeled multi-spectrum points for the source field scene;
in order to measure the difference between the extracted features, the feature deviation of two scenes is calculated by using the maximum mean difference (Maximum Mean Discrepancy, MMD) loss, so as to promote the GCN extraction domain invariant feature:
wherein,is a mapping function that maps the original variables into a high-dimensional space,is a graph characteristic of the ith source domain multispectral point,is the graph characteristic of the j-th target domain multispectral point, N t The number of unlabeled multispectral points in the target domain scene;
the shannon entropy loss constraint network is adopted to obtain a target domain scene pseudo tag with higher confidence, and a specific shannon entropy loss formula is as follows:
wherein H is shannon entropy matrix,is an element in H, in particularThe calculation formula is as follows:
wherein P is a prediction probability matrix of the network to the target domain multispectral laser radar point cloud,in order to predict the probability of a probability,l is the characteristic channel number of the multispectral point cloud,pre-aligned features for the target domain node.
Specifically, the Step4 updates the source domain-target domain alignment network parameters, specifically:
(1) All parameters are optimized using a standard back propagation algorithm;
(2) In training, the overall loss is a combination of source domain classification loss, maximum mean difference (Maximum Mean Discrepancy, MMD) loss, target domain shannon entropy loss, and the overall loss of training is:
wherein,andis the balance coefficient of the balance loss.
Specifically, the adjacency matrix and the feature matrix in the splicing target domain in Step6 are specifically:
marking the target domain adjacency matrix asThe object domain of M dimension is characterizedAnd (3) withSplicing to obtain updated target domain characteristics。
Specifically, the specific formula for calculating the target domain classification loss in Step7 is as follows:
wherein,is a pseudo tag for the i-th point in the target domain scene,is the predictive label of the ith point in the target domain scene, N t For the number of unlabeled multispectral points in the target domain scene,for the extracted target domain scene graph features in Step2,is a set of target domain pseudo tags.
Specifically, the Step8 updates the target domain classification network parameters specifically as follows:
(1) All parameters were optimized using a standard back propagation algorithm.
(2) In training, the target domain classification loss in Step7 is used as a training loss.
The multispectral laser radar often has the phenomenon of alien substances or alien substances in different scenes, which can lead to lower classification precision of the target domain point cloud by using a network obtained by training only the source domain point cloud label when the target domain scene has no label for training. According to the method, the characteristics of a scene of a source domain and a scene of a target domain are aligned through design characteristic pre-alignment Ji Cao, the constant characteristics of the domain are promoted to be extracted by adopting maximum mean difference (Maximum Mean Discrepancy, MMD) loss and shannon entropy loss, and the high-quality target domain point cloud pseudo tag is obtained. And carrying out feature enhancement on the target domain features according to the target domain adjacency matrix, so as to realize high-precision classification on the unlabeled target domain multispectral point cloud by utilizing the labeled source domain scene multispectral point cloud training diagram neural network.
The beneficial effects of the invention are as follows: compared with the prior art, the method and the device for detecting the multi-spectrum point cloud spectrum drift of the multi-spectrum point cloud have the advantage that negative effects caused by the multi-spectrum point cloud spectrum drift among different scenes are relieved. The GCN is assisted to extract domain invariant features through feature domain alignment operation, and the accuracy of the target domain point cloud pseudo tag is guaranteed by adopting maximum mean difference (Maximum Mean Discrepancy, MMD) loss and shannon entropy loss. The target domain features are further enhanced according to the adjacency matrix. Under the conditions of spectrum drift, inconsistent ground feature distribution and the like of multispectral point clouds in different scenes, effective and reliable information transfer is realized so as to realize the ground feature classification of the unlabeled target domain scene. And realizing high-precision classification of multi-spectrum point cloud of the cross-scene under the condition of no real label of the scene of the target domain.
Drawings
FIG. 1 is a cross-scene multispectral point cloud classification method framework based on pseudo tag learning of the present invention;
fig. 2 is a diagram of a real ground object distribution diagram of a data set, (a) a source scene visual diagram, and (b) a target scene visual diagram in the embodiment.
Detailed Description
The invention will be further described with reference to the drawings and the specific examples.
Example 1: as shown in fig. 1, a cross-scene multispectral point cloud classification method based on pseudo tag learning includes the following steps:
step1: respectively enabling the multispectral laser radar point cloud characteristics of the tagged source domain scene and the untagged target domain scene to be according to L 2 Performing feature pre-alignment on the norms and the Laplace matrix;
in Step1, the labeled source domain scene multispectral lidar point cloud data is denoted as (P s Y), the unlabeled target domain scene is denoted (P t ，) WhereinRepresenting a source domain scene contains N s Each of which has a plurality of spectral points of the label,representing the ith labeled multispectral point in a source domain scene，Respectively representing that the target domain scene contains N t A single non-labeled multi-spectral point,representing the i-th unlabeled multispectral point in the target domain scene,true value labels corresponding to multispectral points of all source field scenes are represented,and (5) representing a truth value label corresponding to the ith multispectral point in the source domain scene.
In Step1, the process is described in terms of L 2 The specific steps of feature pre-alignment of the norm and the Laplace matrix are as follows:
(1) Through L 2 The norm carries out characteristic transformation on the characteristics of the source domain and the target domain, and a specific characteristic transformation formula is as follows:
where x is the source domain, target domain characteristics,is the source domain and target domain characteristics after the characteristic transformation,is 2 norms.
(2) Obtaining source domain features of M dimensions according to the formula of the step (1)And M-dimensional target domain featuresWill beAndsplicing to obtain an overall feature matrix with M dimensionCalculating the overall feature matrix according to the K nearest neighbor algorithmFurther computing a diagonal matrix D, the elements of the diagonal matrix D，For elements in the adjacency matrix W, then the laplace matrix l=d-W, so the final overall feature matrix X is updated according to the following formula:
wherein,in order to update the feature matrix after the update, T n for matrix transpose operation s The number of the multispectral points with labels for the source domain scene is N t The number of unlabeled multispectral points in the target domain scene.
Step2: respectively extracting graph features of two scenes by adopting a graph convolution neural network (Graph Convolution Neural Networks, GCN) according to the pre-aligned features;
step3: calculating source domain classification loss, maximum mean difference (Maximum Mean Discrepancy, MMD) loss and target domain shannon entropy loss according to the extracted graph characteristics and source domain labels of the two scenes;
respectively marking the source domain scene and the target domain scene map features extracted from Step2 asAndthe source domain classification loss calculation formula is:
wherein,is the label of the i-th point in the source domain scene,is the predictive label of the i-th point in the source domain scene,is a source domain scene tag set, N s The number of the multi-spectrum points is the number of the labeled multi-spectrum points for the source field scene;
in order to measure the difference between the extracted features, the feature deviation of two scenes is calculated by using the maximum mean difference (Maximum Mean Discrepancy, MMD) loss, so as to promote the GCN extraction domain invariant feature:
wherein,is a mapping function that maps the original variables into a high-dimensional space,is a graph characteristic of the ith source domain multispectral point,is the graph characteristic of the j-th target domain multispectral point, N t The number of unlabeled multispectral points in the target domain scene;
the shannon entropy loss constraint network is adopted to obtain a target domain scene pseudo tag with higher confidence, and a specific shannon entropy loss formula is as follows:
wherein H is shannon entropy matrix,is an element in H, in particularThe calculation formula is as follows:
wherein P is a prediction probability matrix of the network to the target domain multispectral laser radar point cloud,to predict probability, l is the number of characteristic channels of the multispectral point cloud,pre-aligned features for the target domain node.
Step4: iteratively performing Step3, updating the source domain-target domain alignment network parameters, judging whether the model is converged, if yes, ending, then performing Step5, otherwise repeating Step3 to obtain a pseudo tag of the target domain and the confidence coefficient thereof;
the Step4 updates the source domain-target domain alignment network parameters specifically as follows:
(1) All parameters are optimized using a standard back propagation algorithm;
(2) In training, the overall loss is a combination of source domain classification loss, maximum mean difference (Maximum Mean Discrepancy, MMD) loss, target domain shannon entropy loss, and the overall loss of training is:
wherein,andis the balance coefficient of the balance loss and, in the present invention,andthe value is 1.
And updating the source domain-target domain alignment network parameters by using a standard back propagation algorithm, judging whether the model is converged, if so, ending, otherwise, repeating the step S3 until the model is converged.
Step5: according to the confidence level, the pseudo labels are arranged in a descending order, a threshold value alpha is set, and the prior alpha percent of the pseudo labels in the target domain are selected as the true value input of the classification network of the target domain, wherein the value of alpha is 50 in the invention;
step6: splicing the adjacent matrix and the feature matrix in the target domain to obtain a new feature matrix as the feature input of the target domain classification network;
the adjacent matrix and the feature matrix in the splicing target domain in Step6 specifically are:
marking the target domain adjacency matrix asThe object domain of M dimension is characterizedAnd (3) withSplicing to obtain updated target domain characteristics。
Step7: calculating the classification loss of the target domain according to the pseudo tag selected by Step5 and the new feature matrix obtained by Step 6;
the specific formula for calculating the target domain classification loss in Step7 is as follows:
wherein,is a pseudo tag for the i-th point in the target domain scene,is the predictive label of the ith point in the target domain scene, N t For the number of unlabeled multispectral points in the target domain scene,for the extracted target domain scene graph features in Step2,is a set of target domain pseudo tags.
Step8: and (3) taking the target classification loss in the step (S7) as training loss, updating the target domain classification network parameters by using a standard back propagation algorithm, judging whether the model is converged, if so, ending, otherwise, repeating the step (S7) until the model is converged.
The invention is practically feasible to be explained by means of experiments on the basis of the specific implementation description below:
1. experimental data
Harbor of Tobermory dataset: the data set scene is a small harbor in tobermod in the united kingdom, three-band point cloud data are collected by an Optech Titan laser radar, the wavelengths are 1550nm, 1064nm and 532nm respectively, and the data set visualization effect is shown in fig. 2, wherein (a) is a source scene visualization map and (b) is a target scene visualization map. The study area is divided into 7 categories according to the height, material and semantic information of the land cover, namely bare land, grassland, roads, buildings, trees, power lines and automobiles.
University of Houston dataset: the data set scene is a part of the area of the houston campus, and three-band point cloud data are acquired by the Optech Titan laser radar, and the wavelengths are 1550nm, 1064nm and 532nm respectively. The study area is divided into 7 categories, namely bare land, automobiles, grasslands, roads, power lines, buildings and trees, according to the height, materials and semantic information of the land cover. The F score was used as an evaluation index. The visual effect of the two data sets is shown in fig. 2.
2. Experimental details
In the experiment, the data set is classified and verified by adopting the method and the traditional GCN method. Harbor of Tobermory data set is used as a source domain scene, university of Houston data set is used as a target domain scene, and in order to save computing resources, a super-point segmentation method is adopted to segment two scenes into 8000 super-points respectively as input. The method is adopted to carry out point cloud classification, the classification result is evaluated by adopting the evaluation index in the following formula, and the average cross-over ratio (MIoU) of the method in different ground features is shown in the table 1.
Where TP is the number of positive class points split into positive class points, FP is the number of negative class points split into positive class points, and FN is the number of positive class points split into negative class points.
TABLE 1
The method can effectively solve the problems of difficult classification of the multi-spectrum laser radar point cloud of the cross-scene caused by spectrum drift phenomenon, and the like, and can realize high-precision classification of the multi-spectrum point cloud of the cross-scene under the condition that no real label of the scene of the target domain exists.
While the present invention has been described in detail with reference to the drawings, the present invention is not limited to the above embodiments, and various changes can be made without departing from the spirit of the present invention within the knowledge of those skilled in the art.
Claims (8)
1. A cross-scene multispectral point cloud classification method based on pseudo tag learning is characterized in that: the method comprises the following steps:
step1: respectively enabling the multispectral laser radar point cloud characteristics of the tagged source domain scene and the untagged target domain scene to be according to L 2 Performing feature pre-alignment on the norms and the Laplace matrix;
step2: respectively extracting graph features of two scenes by adopting a graph convolution neural network (GCN) according to the pre-aligned features;
step3: calculating source domain classification loss, maximum mean difference MMD loss and target domain shannon entropy loss according to the extracted graph characteristics and source domain labels of the two scenes;
step4: iteratively performing Step3, updating the source domain-target domain alignment network parameters, judging whether the model is converged, if yes, ending, then performing Step5, otherwise repeating Step3 to obtain a pseudo tag of the target domain and the confidence coefficient thereof;
step5: according to the confidence level, the pseudo labels are arranged in a descending order, a threshold value alpha is set, and the target domain pseudo labels with the alpha percent before are selected to be used as the true value input of the target domain classification network;
step6: splicing the adjacent matrix and the feature matrix in the target domain to obtain a new feature matrix as the feature input of the target domain classification network;
step7: calculating the classification loss of the target domain according to the pseudo tag selected by Step5 and the new feature matrix obtained by Step 6;
step8: and (3) iteratively performing Step7, updating the target domain classification network parameters, judging whether the model is converged, if yes, ending, otherwise repeating Step7, and finally obtaining a target domain multispectral point cloud data classification result.
2. The cross-scene multispectral point cloud classification method based on pseudo tag learning of claim 1, wherein the method comprises the following steps of: in Step1, the labeled source domain scene multispectral lidar point cloud data is denoted as (P s Y), the unlabeled target domain scene is denoted (P t ，) Wherein->Representing a source domain scene contains N s Multiple spectral spots with labels->Representing the i-th labeled multispectral point in the source domain scene,>respectively representing that the target domain scene contains N t Label-free multispectral spots->Representing the i-th unlabeled multispectral point in the target domain scene,>truth-value label corresponding to multispectral points of all source field scenes>And (5) representing a truth value label corresponding to the ith multispectral point in the source domain scene.
3. The cross-scene multispectral point cloud classification method based on pseudo tag learning of claim 1, wherein the method comprises the following steps of: in Step1, the process is described in terms of L 2 The specific steps of feature pre-alignment of the norm and the Laplace matrix are as follows:
(1) Through L 2 The norm carries out characteristic transformation on the characteristics of the source domain and the target domain, and a specific characteristic transformation formula is as follows:
；
where x is the source domain, target domain characteristics,for the source domain, target domain characteristics after characteristic transformation,/->Is 2 norms;
(2) Obtaining source domain features of M dimensions according to the formula of the step (1)And M-dimensional target domain featuresWill->And->Splicing to obtain an overall feature matrix of M dimension +.>Calculating an overall feature matrix according to K nearest neighbor algorithm>Further calculating a diagonal matrix D, the elements in the diagonal matrix D being +.>，For elements in the adjacency matrix W, then the laplace matrix l=d-W, so the final overall feature matrix X is updated according to the following formula:
；
wherein,in order to update the feature matrix after the update, T for matrix transposition operationsMake N s The number of the multispectral points with labels for the source domain scene is N t The number of unlabeled multispectral points in the target domain scene.
4. The cross-scene multispectral point cloud classification method based on pseudo tag learning of claim 1, wherein the method comprises the following steps of: the Step3 specifically comprises the following steps:
respectively marking the source domain scene and the target domain scene map features extracted from Step2 asAnd->The source domain classification loss calculation formula is:
；
wherein,is the label of the i-th point in the source domain scene,/->Is the predictive label of the i-th point in the source domain scene,>is a source domain scene tag set, N s The number of the multi-spectrum points is the number of the labeled multi-spectrum points for the source field scene;
in order to measure the difference between the extracted features, the feature deviation of two scenes is calculated by adopting the maximum mean difference MMD loss, so as to promote the GCN extraction domain invariant feature:
；
wherein,is a mapping function mapping the original variable to a high-dimensional space,/->Is a graph characteristic of the ith source domain multispectral point,is the graph characteristic of the j-th target domain multispectral point, N t The number of unlabeled multispectral points in the target domain scene;
the shannon entropy loss constraint network is adopted to obtain a target domain scene pseudo tag with higher confidence, and a specific shannon entropy loss formula is as follows:
；
wherein H is shannon entropy matrix,is an element in H, in particular +.>The calculation formula is as follows:
；
wherein P is a prediction probability matrix of the network to the target domain multispectral laser radar point cloud,for predicting probability, l is the number of characteristic channels of the multispectral point cloud, +.>Pre-aligned features for the target domain node.
5. The cross-scene multispectral point cloud classification method based on pseudo tag learning of claim 4, wherein the method comprises the following steps: the Step4 updates the source domain-target domain alignment network parameters specifically as follows:
(1) All parameters are optimized using a standard back propagation algorithm;
(2) In training, the overall loss is a combination of source domain classification loss, maximum mean difference MMD loss and target domain shannon entropy loss, and the overall loss of training is as follows:
；
wherein,and->Is the balance coefficient of the balance loss.
6. A cross-scene multispectral point cloud classification method based on pseudo tag learning as claimed in claim 3, wherein: the adjacent matrix and the feature matrix in the splicing target domain in Step6 specifically are:
marking the target domain adjacency matrix asThe target domain feature of M dimension is +.>And->Splicing to obtain updated target domain characteristics +.>。
7. The cross-scene multispectral point cloud classification method based on pseudo tag learning of claim 1, wherein the method comprises the following steps of: the specific formula for calculating the target domain classification loss in Step7 is as follows:
；
wherein,pseudo tag which is the i-th point in the target domain scene,>is the predictive label of the ith point in the target domain scene, N t For the number of unlabeled multispectral points in the target domain scene, < >>For the target domain scene graph feature extracted from Step2,/a>Is a set of target domain pseudo tags.
8. The cross-scene multispectral point cloud classification method based on pseudo tag learning of claim 1, wherein the method comprises the following steps of: the Step8 updates the target domain classification network parameters specifically as follows:
(1) All parameters are optimized using a standard back propagation algorithm;
(2) In training, the target domain classification loss in Step7 is used as a training loss.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410061674.6A CN117572457B (en) | 2024-01-16 | 2024-01-16 | Cross-scene multispectral point cloud classification method based on pseudo tag learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410061674.6A CN117572457B (en) | 2024-01-16 | 2024-01-16 | Cross-scene multispectral point cloud classification method based on pseudo tag learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117572457A true CN117572457A (en) | 2024-02-20 |
| CN117572457B CN117572457B (en) | 2024-04-05 |
Family
ID=89892215
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410061674.6A Active CN117572457B (en) | 2024-01-16 | 2024-01-16 | Cross-scene multispectral point cloud classification method based on pseudo tag learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117572457B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117830752A (en) * | 2024-03-06 | 2024-04-05 | 昆明理工大学 | Self-adaptive space-spectrum mask graph convolution method for multi-spectrum point cloud classification |
| CN117953384A (en) * | 2024-03-27 | 2024-04-30 | 昆明理工大学 | Cross-scene multispectral laser radar point cloud building extraction and vectorization method |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115841574A (en) * | 2022-12-19 | 2023-03-24 | 中国科学技术大学 | Domain-adaptive laser radar point cloud semantic segmentation method, device and storage medium |
| CN116403058A (en) * | 2023-06-09 | 2023-07-07 | 昆明理工大学 | Remote sensing cross-scene multispectral laser radar point cloud classification method |
| CN117015813A (en) * | 2021-03-16 | 2023-11-07 | 华为技术有限公司 | Apparatus, system, method, and medium for adaptively enhancing point cloud data sets for training |
| CN117315612A (en) * | 2023-11-13 | 2023-12-29 | 重庆邮电大学 | 3D point cloud target detection method based on dynamic self-adaptive data enhancement |
-
2024
- 2024-01-16 CN CN202410061674.6A patent/CN117572457B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117015813A (en) * | 2021-03-16 | 2023-11-07 | 华为技术有限公司 | Apparatus, system, method, and medium for adaptively enhancing point cloud data sets for training |
| CN115841574A (en) * | 2022-12-19 | 2023-03-24 | 中国科学技术大学 | Domain-adaptive laser radar point cloud semantic segmentation method, device and storage medium |
| CN116403058A (en) * | 2023-06-09 | 2023-07-07 | 昆明理工大学 | Remote sensing cross-scene multispectral laser radar point cloud classification method |
| CN117315612A (en) * | 2023-11-13 | 2023-12-29 | 重庆邮电大学 | 3D point cloud target detection method based on dynamic self-adaptive data enhancement |
Non-Patent Citations (2)
| Title |
|---|
| 杨德东: "基于置信域伪标签策略的半监督三维目标检测", 《计算机应用研究》, vol. 40, no. 6, 30 June 2023 (2023-06-30), pages 1888 - 1893 * |
| 王青旺: "多/高光谱图像和LiDAR数据联合分类方法研究", 《中国博士学位论文全文数据库 工程科技II辑》, vol. 2021, no. 1, 15 January 2021 (2021-01-15), pages 028 - 25 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117830752A (en) * | 2024-03-06 | 2024-04-05 | 昆明理工大学 | Self-adaptive space-spectrum mask graph convolution method for multi-spectrum point cloud classification |
| CN117830752B (en) * | 2024-03-06 | 2024-05-07 | 昆明理工大学 | Self-adaptive space-spectrum mask graph convolution method for multi-spectrum point cloud classification |
| CN117953384A (en) * | 2024-03-27 | 2024-04-30 | 昆明理工大学 | Cross-scene multispectral laser radar point cloud building extraction and vectorization method |
| CN117953384B (en) * | 2024-03-27 | 2024-06-07 | 昆明理工大学 | Cross-scene multispectral laser radar point cloud building extraction and vectorization method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117572457B (en) | 2024-04-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN117572457B (en) | Cross-scene multispectral point cloud classification method based on pseudo tag learning | |
| CN116403058B (en) | Remote sensing cross-scene multispectral laser radar point cloud classification method | |
| Huang et al. | Building extraction from multi-source remote sensing images via deep deconvolution neural networks | |
| CN110245709B (en) | 3D point cloud data semantic segmentation method based on deep learning and self-attention | |
| CN112884791B (en) | Method for constructing large-scale remote sensing image semantic segmentation model training sample set | |
| CN108764138B (en) | Plateau area cloud and snow classification method based on multidimensional and multi-granularity cascade forest | |
| CN111126287B (en) | Remote sensing image dense target deep learning detection method | |
| CN111079847B (en) | Remote sensing image automatic labeling method based on deep learning | |
| CN109063754B (en) | Remote sensing image multi-feature joint classification method based on OpenStreetMap | |
| Xu et al. | A supervoxel approach to the segmentation of individual trees from LiDAR point clouds | |
| CN113313166B (en) | Ship target automatic labeling method based on feature consistency learning | |
| CN111104850A (en) | Remote sensing image building automatic extraction method and system based on residual error network | |
| CN116910571B (en) | Open-domain adaptation method and system based on prototype comparison learning | |
| Cai et al. | A comparative study of deep learning approaches to rooftop detection in aerial images | |
| CN113837134A (en) | Wetland vegetation identification method based on object-oriented deep learning model and transfer learning | |
| Liu et al. | Density saliency for clustered building detection and population capacity estimation | |
| CN111461067B (en) | Zero sample remote sensing image scene identification method based on priori knowledge mapping and correction | |
| Matsuoka et al. | Automatic detection of stationary fronts around Japan using a deep convolutional neural network | |
| Andreev et al. | Cloud detection from the Himawari-8 satellite data using a convolutional neural network | |
| CN113673534A (en) | RGB-D image fruit detection method based on fast RCNN | |
| CN115841557A (en) | Intelligent crane operation environment construction method based on digital twinning technology | |
| CN115346055A (en) | Multi-kernel width map based neural network feature extraction and classification method | |
| CN114842330A (en) | Multi-scale background perception pooling weak supervised building extraction method | |
| Wolters et al. | Classification of large-scale remote sensing images for automatic identification of health hazards: Smoke detection using an autologistic regression classifier | |
| Kiani et al. | Design and implementation of an expert interpreter system for intelligent acquisition of spatial data from aerial or remotely sensed images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |