CN115731476A - Multi-source hyperspectral point cloud semantic segmentation method based on integrated prediction weak surveillance unmanned aerial vehicle - Google Patents
Multi-source hyperspectral point cloud semantic segmentation method based on integrated prediction weak surveillance unmanned aerial vehicle Download PDFInfo
- Publication number
- CN115731476A CN115731476A CN202211214169.8A CN202211214169A CN115731476A CN 115731476 A CN115731476 A CN 115731476A CN 202211214169 A CN202211214169 A CN 202211214169A CN 115731476 A CN115731476 A CN 115731476A
- Authority
- CN
- China
- Prior art keywords
- prediction
- hyperspectral
- unmanned aerial
- aerial vehicle
- point cloud
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a multisource hyperspectral point cloud semantic segmentation method based on an integrated prediction weak surveillance unmanned aerial vehicle. The method is based on that multi-source hyperspectral point cloud data of an unmanned aerial vehicle are taken as research objects, a weak supervision semantic segmentation framework based on integrated prediction is built, on the basis of incomplete supervision, consistency constraint of integrated prediction, entropy regularization guided by integrated prediction results and a self-adaptive soft pseudo-label method are embedded, coding information of unlabeled data is fully utilized, various space and spectrum constraints are provided for weak supervision training, and the semantic segmentation capability of a network model is enhanced. The method solves the problems that in the prior art, forestry vegetation data sampling cost is high, forest samples are rare and difficult to obtain, and manual high-precision labeling of vegetation data in a large-scene forest area wastes time and labor; the method has the advantages that the forest sample training efficiency is guaranteed, meanwhile, supervision sources are added for the weak supervision forestry information extraction network model, and the forestry vegetation information extraction capability of the learning model is improved.
Description
Technical Field
The invention relates to a weakly supervised unmanned aerial vehicle multi-source hyperspectral point cloud semantic segmentation method based on integrated prediction. More specifically, the method solves the problems of difficulty in vegetation sample sampling and insufficient tag data under a deep learning framework, and helps to promote informatization of hyperspectral point cloud and forestry of the unmanned aerial vehicle.
Background
Forest vegetation is the main body of a land ecosystem, bears the functions of adjusting climate, conserving water sources and other ecological services, plays an irreplaceable role in maintaining regional ecology and environment and global carbon balance and relieving global climate change, and has important influence on a land biosphere and other earth surface processes. Globally, many countries and regional authorities seek more limited spatial coverage, more accurate forest inventory, and more accurate forest vegetation attribute data and spatial distribution information to monitor forest ecosystem changes caused by natural and man-made activities. The forest vegetation resource monitoring provides more information support for policy makers about carbon resources, regional biodiversity and sustainable resource development strategies; along with the continuous enhancement of the supervision of forest vegetation resources, the requirements of users on the accuracy and the resolution of forest vegetation parameter extraction are higher and higher, the forest vegetation monitoring develops continuously towards the directions of real-time performance, multiple dimensions and refinement, and higher requirements are put forward on the accuracy and the breadth of data fusion processing.
The hyperspectral data can obtain various biophysical and chemical characteristics, fine spectral resolution bands and rich spectral information of different time and space scales, and the extracted narrowband vegetation index can reduce the influence of atmospheric and moisture absorption and broadband vegetation index saturation, so the hyperspectral data has obvious advantages in estimating the canopy parameters of trees, such as chlorophyll content, forest type (type group), accumulation amount and the like, and acquiring quantitative and qualitative information of crops and vegetation. In recent years, many researchers have classified or identified tree species using hyperspectral techniques. Researches show that the accuracy of classification of partial tree species only by using airborne hyperspectral data reaches 60% -90%, and the hyperspectral tree species classification capability is proved.
Although the spectral range of the hyperspectral data is narrow, various ground object types with slight spectral differences can be accurately detected, the precision of the tree species vegetation classification method based on the hyperspectral data is improved, the distinguishing of tree species with similar spectral characteristics is still limited, and the optical data can only detect the information of the surface of the canopy, so that the precision of the tree species vegetation identification is limited. The LiDAR can acquire detailed tree vegetation structure three-dimensional information, and has obvious advantages in forest vegetation type identification, forest vegetation structure characteristics and canopy physicochemical characteristics, but is lack of corresponding spectral information, and is still difficult to classify complex forest vegetation types. The hyperspectral and LiDAR multi-source data are combined to realize advantage complementation, and the application of the hyperspectral and LiDAR multi-source data to forestry vegetation research becomes a new research hotspot, and some scholars develop related research in the field. And studies have demonstrated that: the fusion of the LiDAR data and the hyperspectral image data can better improve the forest vegetation parameter estimation and the tree species type identification precision compared with single-source data. The tree species charting research area based on hyperspectral and laser radar data fusion is very wide in distribution range, such as urban areas, subtropical forests, natural temperate forests, natural forests and the like. In addition, the unmanned aerial vehicle remote sensing technology and the communication technology develop rapidly, and the fusion of optical and laser LiDAR data under the same unmanned aerial vehicle remote sensing platform is used for single-tree species identification and tree species classification research, so that the identification precision is more outstanding than the classification performance based on single-data type identification.
In the researches of ground feature classification, forestry vegetation information extraction, tree species identification and classification and the like of multisource hyperspectral point cloud data generated by fusing LiDAR and hyperspectral data at present, the LiDAR data are generally converted into image data, fused with the hyperspectral image data and expanded by adopting deep learning methods such as a convolutional neural network and the like; however, the deep learning model training not only needs a large amount of supervision samples, but also needs accurate labeling, so that the method is very time-consuming and labor-consuming work for labeling multi-source hyperspectral point cloud data with complicated ground features, large data volume, disorder and dispersion, especially in forest regions. The weak supervised semantic segmentation method can complete the point cloud segmentation and classification task only by depending on part of label samples, and is an effective way for solving the problems. The traditional weak supervision point cloud segmentation framework has the problems of high computational cost consistency constraint, minimized entropy value generation over confidence (overfitting), incapability of determining training efficiency and weight in a pseudo tag learning method and the like, so that the weak supervision point cloud segmentation effect is poor.
Therefore, it is necessary to develop multi-source hyperspectral point cloud data of the unmanned aerial vehicle, which can effectively process large data volume and unsupervised information by using a small number of labeled samples.
Disclosure of Invention
The invention aims to provide a weak supervision unmanned aerial vehicle multisource hyperspectral point cloud semantic segmentation method based on integrated prediction, which adopts the unmanned aerial vehicle multisource hyperspectral point cloud weak supervision semantic segmentation method, develops an integrated constraint method by utilizing the potential space and spectrum information of unlabeled data in an incomplete supervision network, focuses on researching consistency constraint based on integrated prediction, and generates more stable forestry vegetation target semantic prediction on the basis of not increasing the calculation cost; an integrated prediction result is introduced to assist unlabeled points to carry out entropy regularization, so that the phenomenon of 'overfitting' in a minimized entropy value is inhibited while the overlapping between classes is reduced and the reliability of network classification is improved; on the basis of integrating the predicted consistency constraint, a self-adaptive pseudo label learning strategy is researched, a supervision source is added for a weak supervision network model while the sample training efficiency is ensured, and the forestry vegetation information extraction capability of a learning model is improved; the method solves the problems that in the prior art, forestry vegetation data sampling cost is high, samples are rare and difficult to obtain, vegetation data in a large-scene forest area are time-consuming and labor-consuming through manual high-precision labels, consistency constraint of high calculation cost exists in a traditional weak supervision point cloud segmentation framework, over confidence (over fitting) is generated due to a minimized entropy value, training efficiency and weight cannot be determined in a pseudo label learning method, and the like, so that the weak supervision point cloud segmentation effect is poor.
In order to realize the purpose, the technical scheme of the invention is as follows: a multisource hyperspectral point cloud semantic segmentation method based on an integrated prediction weak surveillance unmanned aerial vehicle is characterized by comprising the following steps: the unmanned aerial vehicle multi-source hyperspectral point cloud data based on the integration of the unmanned aerial vehicle laser radar and the hyperspectral image are taken as research objects, more useful characteristic information is extracted, a weak supervision semantic segmentation framework based on integrated prediction is constructed, on the basis of incomplete supervision, consistency constraint of integrated prediction, entropy regularization guided by integrated prediction results and a self-adaptive soft pseudo label method are embedded, coding information of unlabeled data space and spectrum is fully utilized, various constraints are provided for weak supervision training, and the semantic segmentation capability of the weak supervision semantic segmentation network model framework based on the integrated prediction is enhanced.
In the technical scheme, the weak supervision unmanned aerial vehicle multi-source hyperspectral point cloud semantic segmentation method based on integrated prediction specifically comprises the following steps:
step 1, incomplete supervised learning;
step 2, consistency constraint based on integrated prediction;
step 3, entropy regularization guided by a prediction result;
and 4, self-adaptive pseudo label learning.
In the above technical solution, in step 1, the detailed method of incomplete supervised learning is as follows:
when a certain amount of sampling data is provided and the sampling label data is close to the assumption of independent uniform distribution (independent uniform distribution), the model can obtain a semantic segmentation result similar to the fully supervised learning in a weak supervised learning mode. Firstly, selecting label sample data by adopting a random sampling strategy to perform multi-source hyperspectral point cloud incomplete supervised learning of the unmanned aerial vehicle; and then calculating square root weighted cross entropy loss according to the number of the categories of the selected label sample points, and constructing an incomplete supervised learning network framework.
In the above technical solution, in step 2, a specific method of consistency constraint based on integrated prediction is as follows:
aiming at the problems that the input of the backbone network has randomness and the input samples overlap, the integrated prediction iteration updating is adopted in the training stage, so that each step of training only needs one forward propagation, and the sample training efficiency can be kept. In addition, different global information is contained in the overlapping training areas of different sample inputs, and the method can be regarded as a point-level data enhancement. For each point level data, an exponential moving average is used to calculate an integrated value
Performing association updating; i.e. the integrated predictor for the t-th update
Is shown in formula (1)
Wherein: alpha is the weight of the update,
integrated prediction value, p, for the t-1 th update of the model t The current predicted value is used;
in each training, the updated integrated prediction distribution is distributed
With the current prediction distribution p i By KL divergence(Kullback-Leibler divergence, KLD) to describe the consistency cost, and carry out consistency constraint; cost of consistency V (P) i ) And a loss of consistency L epc See equations (2) and (3);
wherein:
is an integrated prediction P i Posterior probability with point as class c, p ic Is the current prediction P i The point is the posterior probability of the category c, K is the number of categories in the hyperspectral point cloud data set of the unmanned aerial vehicle, and N is the number of points in the training data set.
In the above technical solution, in step 3, entropy Regularization guided based on prediction results adopts an Entropy Regularization (ER) method to improve semantic segmentation performance of the model by using posterior probability of multi-source hyperspectral points of the unlabeled unmanned aerial vehicle; guiding the entropy regularization processing of the unlabeled points by comparing the current prediction with the integrated prediction;
the specific method of entropy regularization based on prediction result guidance comprises the following steps:
comparing current prediction with integrated prediction to obtain consistent unlabeled point P ic Adopting a method of minimizing prediction entropy to reduce category overlapping and obtain obvious semantic features;
predicting non-labeled point P with inconsistent alignment iu Namely the prediction result of the point is unstable, a method of maximizing the prediction entropy is adopted, high uncertainty of network prediction is encouraged, and overfitting is inhibited; maximizing the prediction entropy value is equivalent to minimizing the prediction entropy negative value; entropy value H (P) i ) And entropy regularization loss L er See formulas (4) and (5):
wherein p is ic Is P i The point prediction is the posterior probability of the category c, K is the number of the categories appearing in the multi-source hyperspectral point cloud data set of the unmanned aerial vehicle, and | is the number of the point set points.
In the above technical solution, in step 4, a specific method for adaptive pseudo tag learning is as follows:
directly taking the integrated prediction result of the multi-source high spectrum points of the unlabeled unmanned aerial vehicle as pseudo label data, and keeping the training speed of the original model to the maximum extent; at a consistency cost V (P) i ) As a measurement standard of the weight of the point pseudo label, the weight of the pseudo label is calculated in a self-adaptive manner; weight of each pseudo label point
And a loss of learning of the pseudo tag L ps See equations (6) and (7);
wherein, y ic Is a pseudo tag, p, obtained by integrating the predicted values through the argmax function ic Is P i Predicting points as posterior probability of category c, K is the number of categories appearing in the multi-source hyperspectral point cloud data set of the unmanned aerial vehicle, and | is the number of points in the point set;
synthesizing all the above constraint losses, the loss function L of the final network all Calculated by equation (8):
L all ＝L se +L epc +L er +λL ps (8)
wherein λ is a weighting factor; l is se The loss function is represented.
The multi-source method is a result obtained by fusing high-spectrum point cloud, laser and high-spectrum data of the unmanned aerial vehicle.
The invention has the following advantages:
(1) The weak supervised unmanned aerial vehicle hyperspectral point cloud semantic segmentation method framework based on the integrated prediction fully utilizes coding information of unlabeled points, provides various space and spectrum constraints for the weak supervised network training by adopting an incomplete supervised learning method, a consistency constraint based on the integrated prediction, an entropy regularization based on the prediction comparison guidance and an adaptive pseudo label learning method, and improves the multisource hyperspectral point cloud semantic segmentation capability of an unmanned aerial vehicle.
(2) The method has the characteristics that the training efficiency of the forest sample is guaranteed, meanwhile, a supervision source is added for the weakly supervised forestry information extraction network model, and the forestry vegetation information extraction capability of the learning model is improved; the problem of forestry vegetation data sampling cost height, forest sample rare and difficult acquisition under the prior art, artifical high accuracy label large-scale scene forest zone vegetation data waste time and energy is solved.
(3) According to the method, the characteristics of unmarked points can be effectively utilized by adding integrated prediction and consistency constraint, and positive self-supervision signals are provided for model training; the problem of overfitting of the model caused by entropy minimization is effectively relieved through an entropy regularization method, the self characteristics of the point cloud are more fully mined, and the classification performance of the model is further improved, so that the method can effectively process large data volume (namely data result of fusion of large-scale laser point cloud and hyperspectral image) and unsupervised information by only using a small amount of labeled samples (about 0.1% of the number of training point cloud samples in a supervised classification method).
Drawings
FIG. 1 is a frame diagram of a semantic segmentation method for hyperspectral point clouds of a weakly supervised unmanned aerial vehicle based on integrated prediction.
Detailed Description
The embodiments of the present invention will be described in detail with reference to the accompanying drawings, which are not intended to limit the present invention, but are merely exemplary. While the advantages of the invention will be apparent and understood from the description.
The invention provides an unmanned aerial vehicle multi-source hyperspectral point cloud weak supervised semantic segmentation method framework, introduces an ensemble learning idea to improve training efficiency, creates a more representative contrast sample and enhances the stability of a semantic segmentation model; the entropy regularization method assisted by the integrated prediction result is utilized to reduce class overlapping and inhibit the phenomenon of 'overfitting' caused by minimized entropy; and adjusting the weight by using an adaptive pseudo label learning method to increase the supervision source of model training. The model is expected to solve the problems of weak intelligent interpretation and insufficient sample sampling of multi-source hyperspectral point cloud data of the unmanned aerial vehicle, and provides a universal framework for extracting vegetation information of the hyperspectral point cloud forestry of the unmanned aerial vehicle.
With reference to the accompanying drawings: a weakly supervised unmanned aerial vehicle multi-source hyperspectral point cloud semantic segmentation method based on integrated prediction is characterized in that unmanned aerial vehicle multi-source hyperspectral point cloud data based on integration of an unmanned aerial vehicle laser radar and a hyperspectral image are taken as research objects, a weakly supervised semantic segmentation frame based on integrated prediction is constructed, on the basis of incomplete supervision, consistency constraint of integrated prediction, entropy regularization guided by an integrated prediction result and a self-adaptive soft pseudo label method are embedded, coding information of unlabeled data space and spectrum is fully utilized, various space and spectrum constraints are provided for weakly supervised training, and the semantic segmentation capability of a weakly supervised semantic segmentation network model frame based on integrated prediction is enhanced.
Further, the weakly supervised unmanned aerial vehicle multi-source hyperspectral point cloud semantic segmentation method based on integrated prediction specifically comprises the following steps:
step 1, incomplete supervised learning;
step 2, consistency constraint based on integrated prediction;
step 3, entropy regularization guided by a prediction result;
and 4, self-adaptive pseudo tag learning (as shown in figure 1, an encoder and a decoder are encoding and decoding structures, namely algorithm network structures, input data are fused unmanned aerial vehicle hyperspectral point cloud data, and tag data are also unmanned aerial vehicle hyperspectral point cloud data).
Further, in step 1, the specific method of incomplete supervised learning is as follows:
when a certain amount of sampling data is provided and the sampling label data is close to the assumption of independent uniform distribution (independent uniform distribution), the model can obtain a semantic segmentation result similar to the fully supervised learning in a weak supervised learning mode. Firstly, selecting label sample data by adopting a random sampling strategy to perform multi-source hyperspectral point cloud incomplete supervised learning of the unmanned aerial vehicle; and then, calculating square root weighted cross entropy loss according to the category number of the selected label sample points, and constructing an unmanned aerial vehicle multi-source hyperspectral point cloud incomplete supervised learning network framework.
Further, in step 2, a specific method of consistency constraint based on integrated prediction is as follows:
aiming at the problems that the input of the backbone network has randomness and the input samples overlap, the integrated prediction iteration updating is adopted in the training stage, so that each step of training only needs one forward propagation, and the sample training efficiency can be kept. In addition, different global information is contained in the overlapped training areas of different sample inputs, and the method can be regarded as point-level data enhancement. For each point level data, an exponential moving average is used to calculate an integrated value
Performing association updating; i.e. the integrated predictor of the t-th update
Is shown in formula (1)
Wherein: alpha is the weight of the update,
integrated prediction value, p, for the t-1 th update of the model t The current predicted value is used;
in each training, the updated integrated prediction is distributed
With the current prediction distribution p i Describing consistency cost through KL divergence (KLD) and carrying out consistency constraint; cost of consistency V (P) i ) And loss of consistency L epc See equations (2) and (3);
wherein:
is an integrated prediction P i The point is the posterior probability of class c, p ic Is the current prediction P i The point is the posterior probability of the category c, K is the number of categories in the data set, and N is the number of points in the training data set.
Further, in step 3, entropy Regularization guided based on the prediction result adopts an Entropy Regularization (ER) method to improve the semantic segmentation performance of the model by utilizing the posterior probability of the multi-source hyperspectral point of the unlabeled unmanned aerial vehicle; guiding the entropy regularization processing of the unlabeled points by comparing the current prediction with the integrated prediction;
the specific method of entropy regularization based on prediction result guidance comprises the following steps:
comparing current prediction with integrated prediction to obtain consistent unlabeled point P ic The method of minimizing prediction entropy is adopted, category overlapping is reduced, and remarkable multisource hyperspectral point cloud semantic features of the unmanned aerial vehicle are obtained;
predicting non-labeled point P with inconsistent alignment iu That is, the prediction result of the point is unstable, and a method of maximizing the prediction entropy is adopted to encourage high uncertainty of network prediction and inhibitMaking an overfitting; maximizing the prediction entropy value is equivalent to minimizing the prediction entropy negative value; entropy value H (P) i ) And entropy regularization loss L er See formulas (4) and (5):
wherein p is ic Is P i The point prediction is the posterior probability of the category c, K is the number of the categories appearing in the multi-source hyperspectral point cloud data set of the unmanned aerial vehicle, and | is the number of the point set points.
Furthermore, in step 4, the specific method for adaptive pseudo tag learning is as follows:
directly taking the integrated prediction result of the multi-source high spectrum points of the unlabeled unmanned aerial vehicle as pseudo label data, and keeping the training speed of the original network model to the maximum extent; at a consistency cost V (P) i ) The weight of the pseudo label is calculated in a self-adaptive manner as a weighing standard of the weight of the point pseudo label; weight of each pseudo label point
And a loss of pseudo label learning L ps See equations (6) and (7);
wherein, y ic Is a pseudo tag, p, obtained by integrating the predicted values through the argmax function ic Is P i Predicting points as posterior probability of category c, K is the number of categories appearing in the multi-source hyperspectral point cloud data set of the unmanned aerial vehicle, and | is the number of points in the point set;
synthesizing all the above constraint losses, the loss function L of the final network all Calculated by equation (8):
L all ＝L se +L epc +L er +λL ps (8)
where λ is a weighting factor.
In order to verify the accuracy of the application, the invention carries out the following experimental verification:
a weak surveillance unmanned aerial vehicle multi-source high spectral point cloud semantic segmentation method based on integrated prediction is evaluated on an onboard multi-spectral LiDAR point cloud data set, and each point of a laser point of the data set comprises geometric coordinate information and three wave band information of 532nm,1062nm and 1550nm. Through a manual interpretation method, the ground features of the whole scene are divided into six types, namely roads, grasslands, trees, buildings, bare land and power lines.
The airborne multi-spectral LiDAR point cloud dataset consists of 12137, 1870, and 2874 point cloud samples in the training, validation, and test datasets, respectively. N =4096 points per sample.
An Adam optimizer is selected for the experiment, the initial learning rate, the number K of neighboring points and the downsampling size of the grid are set to 0.01, 16 and 0.05m respectively, and the model is trained for 200 rounds by using 0.98 as the decay index of the learning rate. According to the video memory size, the number of input points is selected to be 65536, and the batch size is 3. Randomly selecting one thousandth of points of the original data set from the training data as marking points to perform model training.
In order to reduce the influence of the randomness of the model on the result, fixed sparse mark points and test data are selected for model training and testing. In addition, for more accurate evaluation of model test Accuracy, the results of the experiment are analyzed using Overall Accuracy (OA), mean Intersection over unit (mlou), accuracy (Precision), recall (Recall), and F1 score (F1-score, F1) as evaluation indices. And the coordinate information and the spectrum information are used as input characteristics to carry out network training and testing, and the testing results are shown in the following table 1.
TABLE 1 test results
As can be seen from Table 1, when compared with the fully supervised methods using coordinate information (SE-PointNet + + and FR-GCNet), the method of the present invention uses only one in a thousand of marked points, OA, mIoU and IoU of roads, buildings, vegetation, bare land and grassland are all equivalent to the fully supervised method, wherein the supervised coverage rates of OA and mIoU reach 90.19% and 64.12%, respectively.
The analysis result shows that compared with a full-supervision method, the weak supervision unmanned aerial vehicle multi-source hyperspectral point cloud semantic segmentation method based on integrated prediction can obtain competitive performance expression under the condition of sparse mark points, namely the weak supervision unmanned aerial vehicle multi-source hyperspectral point cloud semantic segmentation method can effectively process large data volume and unsupervised information by only using a small number of mark samples.
In summary, the point cloud weak supervision semantic segmentation method based on integrated prediction provided by the invention has the following technical effects: the integration prediction and consistency constraint are added, so that the unmarked point characteristics can be effectively utilized, and positive self-supervision signals are provided for model training; the entropy regularization method can effectively relieve the problem of overfitting of the model caused by entropy minimization, more fully excavate the self characteristics of the point cloud, and further improve the classification performance of the model, so that the method can effectively process large data volume and unsupervised information by only using a small amount of labeled samples (about 0.1 percent of the number of the training point cloud samples in the supervised classification method).
Other parts not described belong to the prior art.
Claims (6)
1. A multisource hyperspectral point cloud semantic segmentation method based on an integrated prediction weak surveillance unmanned aerial vehicle is characterized by comprising the following steps: the unmanned aerial vehicle multi-source hyperspectral point cloud data based on the integration of the unmanned aerial vehicle laser radar and the hyperspectral image are taken as research objects, a weak supervision semantic segmentation framework based on the integration prediction is constructed, on the basis of incomplete supervision, consistency constraint of the integration prediction, entropy regularization guided by an integration prediction result and a self-adaptive soft pseudo-label method are embedded, coding information of unlabeled data space and spectrum is fully utilized, constraints of geometry and spectrum are provided for weak supervision training, and the semantic segmentation capability of the weak supervision semantic segmentation network model framework based on the integration prediction is enhanced.
2. The weakly supervised unmanned aerial vehicle multisource hyperspectral point cloud semantic segmentation method based on integrated prediction of claim 1, which is characterized in that: the weakly supervised unmanned aerial vehicle multi-source hyperspectral point cloud semantic segmentation method based on integrated prediction specifically comprises the following steps:
step 1, incomplete supervised learning;
step 2, consistency constraint based on integrated prediction;
step 3, entropy regularization guided by a prediction result;
and 4, self-adaptive pseudo label learning.
3. The weakly supervised unmanned aerial vehicle multisource hyperspectral point cloud semantic segmentation method based on integrated prediction of claim 2, wherein: in step 1, the detailed method of incomplete supervised learning is as follows:
firstly, selecting label sample data by adopting a random sampling strategy to perform multi-source hyperspectral point cloud incomplete supervised learning of the unmanned aerial vehicle; and then, calculating square root weighted cross entropy loss according to the category number of the selected label sample points, and constructing an unmanned aerial vehicle hyperspectral point cloud incomplete supervised learning network framework.
4. The weakly supervised unmanned aerial vehicle multisource hyperspectral point cloud semantic segmentation method based on integrated prediction of claim 3, wherein: in step 2, a specific method of consistency constraint based on integrated prediction is as follows:
in the training stage, integrated prediction iterative updating is adopted, and different global information is contained in overlapped training areas input by hyperspectral point cloud samples of different unmanned aerial vehicles, so that point-level data enhancement is realized; for each point level data, an exponential moving average is used to calculate an integrated value
Performing association updating; i.e. the integrated predictor for the t-th update
See formula (1):
wherein: alpha is the weight of the update,
integrated prediction value, p, for the t-1 th update of the model t The current predicted value is used;
in each training, the updated integrated prediction distribution is distributed
With the current prediction distribution p i Describing consistency cost through KL divergence, and carrying out consistency constraint; consistency cost V (P) i ) And loss of consistency L epc See formulas (2) and (3):
wherein:
is an integrated prediction P i Posterior probability with point as class c, p ic Is the current prediction P i The point is the posterior probability of the category c, K is the number of categories in the hyperspectral point cloud data set of the unmanned aerial vehicle, and N is the number of points in the hyperspectral point cloud training data set of the unmanned aerial vehicle.
5. The weakly supervised unmanned aerial vehicle multisource hyperspectral point cloud semantic segmentation method based on integrated prediction of claim 4, wherein: in step 3, entropy regularization guided by a prediction result adopts an entropy regularization method to improve the semantic segmentation performance of the learning model by utilizing the posterior probability of the unlabeled unmanned aerial vehicle hyperspectral point cloud; guiding the entropy regularization processing of the unlabeled points by comparing the current prediction with the integrated prediction;
the specific method for entropy regularization based on prediction result guidance comprises the following steps:
comparing current prediction with integrated prediction to obtain consistent unlabeled point P ic Adopting a method of minimizing prediction entropy to reduce category overlapping and obtain obvious semantic features;
predicting non-labeled point P with inconsistent alignment iu Namely, the prediction result of the point is unstable, and a method for maximizing the prediction entropy is adopted, so that high uncertainty of network prediction is encouraged, and overfitting is inhibited; maximizing the prediction entropy value is equivalent to minimizing the prediction entropy negative value; entropy value H (P) i ) And entropy regularization loss L er See formulas (4) and (5):
wherein p is ic Is P i The point prediction is the posterior probability of the category c, K is the number of the categories appearing in the hyperspectral point cloud data set of the unmanned aerial vehicle, and | is the number of the point set points.
6. The integrated prediction weak surveillance unmanned aerial vehicle-based multi-source hyperspectral point cloud semantic segmentation method according to claim 5, wherein the method comprises the following steps: in step 4, the specific method for self-adaptive pseudo label learning is as follows:
directly will not labelAn integrated prediction result of multisource hyperspectral point clouds of the unmanned aerial vehicle is used as pseudo label data, and the training speed of an original model is kept to the maximum extent; at a consistency cost V (P) i ) As a measurement standard of the weight of the point pseudo label, the weight of the pseudo label is calculated in a self-adaptive manner; weight of each pseudo label point
And a loss of learning of the pseudo tag L ps See equations (6) and (7);
wherein, y ic Is a pseudo tag, p, obtained by integrating the predicted values through the argmax function ic Is P i The point prediction is posterior probability of the category c, K is the number of the categories appearing in the high-spectrum point cloud data set of the unmanned aerial vehicle, and | is the number of points in the point set;
synthesizing all constraint losses, the loss function L of the final network all Calculated by equation (8):
L all ＝L se +L epc +L er +λL ps (8)
where λ is a weighting factor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211214169.8A CN115731476A (en) | 2022-09-30 | 2022-09-30 | Multi-source hyperspectral point cloud semantic segmentation method based on integrated prediction weak surveillance unmanned aerial vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211214169.8A CN115731476A (en) | 2022-09-30 | 2022-09-30 | Multi-source hyperspectral point cloud semantic segmentation method based on integrated prediction weak surveillance unmanned aerial vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115731476A true CN115731476A (en) | 2023-03-03 |
Family
ID=85293587
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211214169.8A Pending CN115731476A (en) | 2022-09-30 | 2022-09-30 | Multi-source hyperspectral point cloud semantic segmentation method based on integrated prediction weak surveillance unmanned aerial vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115731476A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117422908A (en) * | 2023-10-17 | 2024-01-19 | 煜邦数字科技(广东)有限公司 | Point cloud identification method based on SE-PointNet++ neural network model |
-
2022
- 2022-09-30 CN CN202211214169.8A patent/CN115731476A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117422908A (en) * | 2023-10-17 | 2024-01-19 | 煜邦数字科技(广东)有限公司 | Point cloud identification method based on SE-PointNet++ neural network model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ran et al. | Large-scale land cover mapping with the integration of multi-source information based on the Dempster–Shafer theory | |
| CN113033520A (en) | Tree nematode disease wood identification method and system based on deep learning | |
| Fan et al. | Understanding spatial-temporal urban expansion pattern (1990–2009) using impervious surface data and landscape indexes: a case study in Guangzhou (China) | |
| Zhang et al. | Spatial domain bridge transfer: An automated paddy rice mapping method with no training data required and decreased image inputs for the large cloudy area | |
| Liu et al. | Deep learning in forest structural parameter estimation using airborne lidar data | |
| Zhong et al. | Identification of tree species based on the fusion of UAV hyperspectral image and LiDAR data in a coniferous and broad-leaved mixed forest in Northeast China | |
| CN115731476A (en) | Multi-source hyperspectral point cloud semantic segmentation method based on integrated prediction weak surveillance unmanned aerial vehicle | |
| Dutra et al. | Comparative analysis of methods applied in vegetation cover delimitation using Landsat 8 images | |
| Wang et al. | Hybrid model for estimating forest canopy heights using fused multimodal spaceborne LiDAR data and optical imagery | |
| Lin et al. | An Unsupervised Transformer-based Multivariate Alteration Detection Approach for Change Detection in VHR Remote Sensing Images | |
| MohanRajan et al. | Fuzzy Swin transformer for Land Use/Land Cover change detection using LISS-III Satellite data | |
| Ma et al. | Total and component forest aboveground biomass inversion via LiDAR-derived features and machine learning algorithms | |
| CN117349784A (en) | Remote sensing data processing method, device and equipment | |
| Wang et al. | Deep learning in extracting tropical cyclone intensity and wind radius information from satellite infrared images—A review | |
| Maqsoom et al. | Extracting built-up areas from spectro-textural information using machine learning | |
| Yu et al. | A vegetation classification method based on improved dual-way branch feature fusion U-net | |
| Wang et al. | Information Extraction of the Vehicle from High-Resolution Remote Sensing Image Based on Convolution Neural Network | |
| Zhao et al. | Identification of remote sensing-based land cover types combining nearest-neighbor classification and SEaTH algorithm | |
| Singh et al. | 3D-MFDNN: Three-dimensional multi-feature descriptors combined deep neural network for vegetation segmentation from airborne laser scanning data | |
| Wu et al. | DF4LCZ: A SAM-Empowered Data Fusion Framework for Scene-Level Local Climate Zone Classification | |
| Yang et al. | A Knowledge Distillation-based Ground Feature Classification Network with Multiscale Feature Fusion in Remote Sensing Images | |
| Awuah | Effects of spatial resolution, land-cover heterogeneity and different classification methods on accuracy of land-cover mapping | |
| Zang et al. | CBF-Net: An Adaptive Context Balancing and Feature Filtering Network for Point Cloud Classification | |
| Huang et al. | Extraction of pine wilt disease based on a two-stage unmanned aerial vehicle deep learning method | |
| CN117670527B (en) | Method and system for determining peasant household loan credit limit based on land parcel data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |