CN117541799B - Large-scale point cloud semantic segmentation method based on online random forest model multiplexing - Google Patents

Abstract

This application discloses a large-scale point cloud semantic segmentation method based on online random forest model reuse, and relates to the field of point cloud data processing. This application includes: preprocessing point cloud data; feature extraction of point cloud block data, and obtaining based on A mature online random forest model for point cloud block data; for the point cloud data to be predicted, its feature distribution is matched in the point cloud feature space distribution database, and the mature online random forest corresponding to the matching result is used to predict the point cloud data to be predicted. The method of this application improves generalization ability, real-time performance and reduces computing resource requirements. This application provides a framework design with better compatibility, which can be compatible and adaptable to different types of point cloud data processing methods, and efficiently integrate the results of a single machine.

Description

Large-scale point cloud semantic segmentation method based on online random forest model reuse

Technical field

This application relates to the field of point cloud data processing, specifically to a large-scale point cloud semantic segmentation method based on online random forest model reuse.

Background technique

With the development of remote sensing technology, it has become easier to obtain high-precision large-scale remote sensing urban point clouds. However, for remote sensing urban scene point clouds covering hundreds of meters and containing tens of millions of points, quickly obtaining accurate semantic segmentation results in an engineering context still requires a lot of manual annotation work. To reduce this tedious task, researchers need to explore the potential of machine learning techniques in processing point cloud semantic segmentation.

Classic machine learning models such as support vector machine SVM, random forest RF, AdaBoost, etc. can achieve fast and good training results on point cloud scenes, but they require manually designed features to ensure the classification ability of the model.

In recent years, deep learning has made significant progress in the field of point cloud semantic segmentation. For example, PointNet, PointNet++, and PointConv can process the entire point cloud, but the segmentation accuracy on large-scale point clouds is limited. RandLaNet can perform large-scale semantic segmentation quickly and accurately, but it is easily affected by data collection noise. SQN reduces labeling costs in a weakly supervised manner, but the model is sensitive to the area and size of the training data. Although deep learning technology has demonstrated powerful capabilities in complex feature extraction, a large amount of training data is still required to avoid overfitting problems in training. Moreover, because urban scenes contain complex regional characteristics and terrain information, current methods cannot solve the generalization problem well, resulting in a large number of segmentation errors in the pre-trained models of these methods when facing new point cloud data. Therefore, in order to assist point cloud semantic segmentation, it still takes a lot of time to annotate new data sets to train a large-scale deep learning model.

Therefore, there is an urgent need for an efficient and powerful large-scale point cloud data semantic segmentation method. This approach should be able to quickly process differently distributed datasets and achieve efficient and accurate semantic segmentation without the need for extensive manual annotation. Such a method will significantly improve the efficiency and reliability of point cloud semantic segmentation and promote the development of smart city modeling and environmental intelligent perception.

Contents of the invention

This application is based on a large-scale point cloud semantic segmentation method based on online random forest model reuse, aiming to solve the problems in the existing technology.

In the first aspect, this application provides a large-scale point cloud semantic segmentation method based on online random forest model reuse, including: training multiple mature online random forest models in blocks;

(1) Use a pre-trained deep learning model for initial semantic segmentation. Although the current deep learning method does not solve the generalization problem well, it can quickly provide complete annotation information, reduce labor costs, and lay the foundation for the online random forest model. (2) Fit multiple online random forest models in blocks. Divide the results into regular blocks and select multiple point clouds to extract features. Features designed for geometry, color, elevation, etc. can enhance the model's segmentation capabilities and feature statistical accuracy. Then, an online random forest model is used to fit the initial segmentation results. Through block training, the training time of the online random forest model can be reduced and the requirements of real-time interaction can be achieved. In addition, since it is difficult to extract feature distribution from large-scale point clouds, the blocking method can more accurately extract local feature values and feature distribution characteristics. (3) Manually correct and train a mature online random forest model. By manually correcting and quickly optimizing the erroneous parameters in the model, a mature online random forest model and correct semantic segmentation results are obtained. Due to the characteristics of online learning, the model can learn the logic of manual interaction to improve error parameters and modify other similar error results, which greatly reduces the number of manual interactions and accelerates the training and maturation process of the model. (4) Initialize the feature distribution database and model reuse library. After manually correcting the segmentation results, record the histogram distribution of the multi-dimensional feature space, store the distribution results in the feature distribution database, and store the corresponding mature online random forest model in the model library. Establishing a sizable feature distribution database and online random forest model library can provide appropriate models for subsequent prediction of point cloud blocks and achieve reuse. This avoids repeated training and reduces computational costs.

Incrementally reuse mature models for semantic segmentation on uniformly distributed point cloud patches;

(1) Reuse mature models for initial semantic segmentation on point cloud blocks with consistent distribution: First, extract the multi-dimensional feature space histogram distribution of the predicted point cloud blocks, and use the model reuse strategy to calculate the relative feature distribution of each distribution in the database. Babbitt coefficient similarity. Then, the corresponding model with the highest similarity distribution is selected for initial semantic segmentation. Since machine learning models follow the assumption of independent and identical distribution, the model performs better on prediction sets whose distribution is close to that of the training set. Ensuring distribution consistency between training and prediction sets through our reuse strategy can improve the model's accurate prediction capabilities. The prediction results are better than those of pre-trained deep learning models with insufficient generalization capabilities. This can reduce errors that require manual correction and speed up the training and maturation process of the model.

(2) Update the feature distribution database and model library: Based on the accurate initial segmentation results of the mature model, use manual correction to train a new and mature online random forest model, and update the feature distribution database and model library. This can expand the scope of model reuse strategies, improve prediction effects, and achieve incremental learning effects of prediction while training. As the size of the database increases, the best matching model can be found for arbitrarily distributed point cloud patches, thereby achieving our goal of improving generality. In addition, the initial segmentation results of the model reuse strategy will continue to improve with the size of the model library, approaching the effects of the best deep learning models, and requiring less manual annotation and training waiting time.

Further, the preprocessing of the point cloud data includes: performing regular segmentation of the point cloud data, obtaining the point cloud segmentation data, and using a pre-trained model of the mainstream point cloud segmentation method to initially segment the point cloud segmentation data. , the initial segmentation results are obtained, which are used to provide annotations for the training of strongly supervised online random forests.

Further, the feature extraction of point cloud block data and obtaining a mature online random forest model based on point cloud block data include:

Use multi-dimensional features in geometry, elevation and color to describe semantic differences, extract the multi-dimensional features of the point cloud block data and record them into the point cloud feature spatial distribution database;

The manually corrected data based on the semantic segmentation results of the online random forest model is used as a training set, and the online random forest model is iteratively trained to obtain a mature online random forest model that meets the point cloud block data prediction accuracy threshold, and is recorded in the mature online random forest model. library.

Further, for the point cloud data to be predicted, the feature distribution is matched in the point cloud feature space distribution database, and the mature online random forest corresponding to the matching result is used to predict the point cloud data to be predicted, including:

Use the feature distribution similarity evaluation algorithm to match a feature distribution that is most similar to the feature distribution of the point cloud data to be predicted in the point cloud feature space distribution database;

Corresponding to the multi-dimensional feature spatial distribution of the point cloud to be predicted and the mature online random forest model;

Determine whether the prediction accuracy of the current point cloud block data obtained by the corresponding mature online random forest model meets the threshold;

If the usage requirements are met, the prediction results will be output directly. If the prediction results are not met, a new online random forest model will be trained on the prediction results. The correction of the error area will be completed through manual interaction and the training and maturity of the online random forest will be accelerated to obtain semantic segmentation results that meet the threshold and mature online Random forest model. Add the distribution and mature models of predicted point cloud blocks to the distribution database and mature model library.

Further, the point feature distribution similarity evaluation algorithm is used to match a feature distribution that is most similar to the feature distribution of the point cloud data to be predicted in the point cloud feature space distribution database, including calculation based on the multi-dimensional feature space distribution of the point cloud. , specifically including:

Analyze the proportion of the semantics of the current point cloud block data to be predicted in the total point cloud block data to be processed, and perform feature extraction based on the different semantics of the point cloud block data currently to be predicted, and obtain the to-be-predicted Multi-dimensional characteristics of point cloud block data, and analyze the distribution of feature values of the point cloud block data currently to be predicted.

Further, the artificially corrected data based on the semantic segmentation results of the online random forest model is used as a training set, and the online random forest model is iteratively trained to obtain a mature online random forest model that meets the point cloud block data prediction accuracy threshold, and recorded Mature online random forest model library, including:

The segmentation prediction results of the point cloud semantic segmentation model used for feature extraction are trained using an online random forest model to obtain the semantic segmentation results of the random forest model in training. The error areas of the segmentation results are manually marked to generate the manual correction data. Manual correction data is used as the training set input for updating the branch nodes of different decision trees in the online random forest, and the iterative training output is a mature online random forest model that meets the prediction accuracy of point cloud block data.

Further, the multi-dimensional feature space distribution similarity includes Bhattacharyya distance to measure histogram distribution similarity.

Further, it specifically includes: using a pre-trained deep learning model to obtain prediction results for feature extraction corresponding to the point cloud block data.

Further, it specifically includes: using a pre-trained point cloud semantic segmentation model to obtain prediction results for feature extraction of corresponding point cloud block data.

In a second aspect, the present invention also provides a readable storage medium. A computer program is stored on the readable storage medium. When the computer program is executed by a processor, the computer program implements any of the above online-based methods described in the first aspect. A large-scale point cloud semantic segmentation method using random forest model reuse.

The beneficial effects of this application include:

This application improves real-time performance and reduces computing resource requirements: By using online learning ideas and random forest machine learning models, the demand for computing resources on a single computer can be significantly reduced through block parallel processing, and feedback can be provided in real-time. , allowing multiple people to cooperate;

This application improves generalization capabilities: The method provided by this application combines machine learning models and manual interaction to better process point cloud data with different collection methods and different ground features, achieving excellent segmentation effects;

This application provides a framework design with better compatibility, which can be compatible and adaptable to different types of point cloud data processing methods, and efficiently integrate the results of a single machine;

The model reuse framework provided by this application can learn the logic contained in manual interaction, learn and imitate manual interaction through machine learning models, and achieve the effect of manual interaction after sufficient training, greatly reducing the workload of manual correction and reducing labor costs. .

Description of drawings

The drawings described here are used to provide a further understanding of the embodiments of the present application, constitute a part of the present application, and do not constitute a limitation to the embodiments of the present application. In the attached picture:

Figure 1 is a schematic diagram of the training phase of obtaining multiple mature models and point cloud distribution characteristics online based on block partitioning in an exemplary embodiment of the present application.

Figure 2 is a schematic diagram of the prediction stage of semantic segmentation on distributed consistent point cloud blocks based on reusing mature models in an exemplary embodiment of the present application.

Figure 3 is a segmented flow chart of the large-scale point cloud semantic segmentation method based on online random forest model reuse in this application.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatus and methods consistent with aspects of the application as detailed in the appended claims.

Large-scale point cloud semantic segmentation methods based on online random forest model reuse require a large amount of fully labeled data. Although collecting large-scale point cloud data is no longer a tedious process, manually generating point-level labels on large-scale data sets requires High costs. In addition, the current large-scale point cloud segmentation methods are only trained for specific scenarios, and the model generalization is not ideal. The segmentation accuracy depends on the training data set, and the prediction results do not meet the requirements for urban modeling. They also rely on manual labor. Proofreading. To sum up, research on point cloud segmentation has made good progress, but there are still some problems that need to be solved:

(1) Time overhead for processing large-scale data: The point cloud model of smart city scenes is huge, reaching billions of point clouds. Processing these large-scale data requires a large amount of computing resources and time, which largely limits the application of existing methods. For the most advanced deep learning models, the demand for training set size is also larger than that of machine learning models, and the time and cost are further increased.

(2) Prediction accuracy for processing large-scale data: Even if the most advanced deep learning method can obtain a good prediction result in other areas through high-cost manual production of a large-scale training set, it still cannot guarantee the completeness of the result. Correct, the rest of the erroneous data needs to be manually reviewed and processed, further increasing labor costs.

(3) Generalization ability: Although a deep learning model may perform well on specific training data, it may perform poorly on new, unseen data. This is particularly important in the context of smart cities. Because of the complexity and diversity of the urban environment, the prediction effect of the model that has been trained with a lot of manpower cannot achieve satisfactory results in other cities, and it has to be retrained again. Huge model. In addition, there are also significant differences in the geographical characteristics of different districts in a large city. For example, there are great differences in the width of roads, the density of trees, the height and density of buildings between densely populated areas in the city center and parks in the suburbs, resulting in If the model wants to take into account these two scenarios, it needs to create training sets corresponding to the scenarios to ensure the prediction effect, which further increases the labor cost required for model training.

Technical concept of this application:

A large-scale point cloud semantic segmentation framework is designed that enables efficient multi-person collaborative correction, adaptive learning of manual correction logic, and reduced labor costs. Essentially, it maintains a trained dynamic growth model library and a corresponding dynamic growth distribution database that records the distribution of point cloud blocks in multi-dimensional feature space. And there is a one-to-one correspondence between the model library and the records in the database.

The specific application scenarios of this application are extraction scenarios for complex urban areas and surrounding hilly farmland areas.

The large-scale point cloud semantic segmentation method based on online random forest model reuse provided by this application aims to solve the above technical problems of the existing technology.

The technical solution of the present application and how the technical solution of the present application solves the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

This application provides a large-scale point cloud semantic segmentation method based on online random forest model reuse, including:

Preprocess point cloud data; use the pre-trained deep learning model to perform initial semantic segmentation on the new data set to provide annotation information for subsequent online random forests.

Extract features from point cloud block data and obtain a mature online random forest model based on point cloud block data; divide the new data set into blocks and train an online random forest model on each block to fit the pre-trained deep learning model the result of. New features are designed to describe the geometric, color, and elevation characteristics of different semantic categories to enhance the learning ability of the online random forest model. An online random forest model is used to fit the initial semantic segmentation results of the corresponding point cloud blocks, supplemented by multiple manual interactions to complete error correction. Due to the characteristics of online learning, every time there is an interaction, the online learning model will learn the characteristics of the interaction, thereby optimizing the random forest model and applying other non-interaction error areas to reduce the total number of interactions. Each point cloud block is manually corrected. You will get complete semantic segmentation results and a mature online random forest model that learns the distribution characteristics of the current point cloud blocks.

For the point cloud data to be predicted, match its feature distribution in the point cloud feature spatial distribution database, and use the mature online random forest corresponding to the matching result to predict the point cloud data to be predicted: As the number of manually corrected point cloud blocks increases, the histogram is designed Graph distribution statistics points the distribution of point cloud blocks in multi-dimensional feature space. The new features of the design used as the feature description of the histogram can reflect the distribution differences of different blocks. A distribution similarity calculation formula is proposed to establish the distribution relationship between manually corrected point cloud blocks and predicted point cloud blocks, and reuse a mature online random forest model for initial semantic segmentation under the condition that the training set and prediction set are consistent. This can achieve much higher results. Regarding the semantic segmentation accuracy of the pre-trained deep learning model, due to the independent and identical distribution assumption of the model, the prediction effect is better on data sets with high similarity. By reusing the mature model to predict on point cloud blocks with high similarity, it can It obtains better semantic segmentation results than reusing other models; and continues to manually correct the results predicted using the online random forest model, and trains a new online random forest model, which is added to the reused model to expand model complexity. Use scope.

Further, the preprocessing of the point cloud data includes: performing regular segmentation of the point cloud data, obtaining the point cloud segmentation data, and using a pre-trained model of the mainstream point cloud segmentation method to initially segment the point cloud segmentation data. , the initial segmentation results are obtained, which are used to provide annotations for the training of strongly supervised online random forests.

Further, the feature extraction of point cloud block data and obtaining a mature online random forest model based on point cloud block data include:

Extract multi-dimensional features of the point cloud block data and record them into a point cloud feature spatial distribution database;

The manually corrected data based on the semantic segmentation results of the online random forest model is used as a training set, and the online random forest model is iteratively trained to obtain a mature online random forest model that meets the point cloud block data prediction accuracy threshold, and is recorded in the mature online random forest model. library.

Further, for the point cloud data to be predicted, the feature distribution is matched in the point cloud feature space distribution database, and the mature online random forest corresponding to the matching result is used to predict the point cloud data to be predicted, including:

Use the point feature distribution similarity evaluation algorithm to match a feature distribution that is most similar to the feature distribution of the point cloud data to be predicted in the point cloud feature spatial distribution database;

Corresponding to the multi-dimensional feature spatial distribution of the point cloud to be predicted and the mature online random forest model;

Determine whether the prediction accuracy of the current point cloud block data obtained by the corresponding mature online random forest model meets the threshold;

Output the prediction results that meet the usage requirements, or use the point cloud block data corresponding to the multi-dimensional feature space distribution of the point cloud to be predicted as the verification set input, continue to train the mature online random forest model, and obtain the verification set input. Prediction accuracy of the fitted mature online random forest model, binding the fitted mature online random forest model with the current multi-dimensional feature spatial distribution information of the point cloud to be predicted plus the multi-dimensional point cloud of the original mature online random forest model Feature distribution information.

Further, the point feature distribution similarity evaluation algorithm is used to match a feature distribution that is most similar to the feature distribution of the point cloud data to be predicted in the point cloud feature space distribution database, including calculation based on the multi-dimensional feature space distribution of the point cloud. , specifically including:

Analyze the proportion of the semantics of the current point cloud block data to be predicted in the total point cloud block data to be processed, and perform feature extraction based on the different semantics of the point cloud block data currently to be predicted, and obtain the to-be-predicted Multi-dimensional characteristics of point cloud block data, and analyze the distribution of feature values of the point cloud block data currently to be predicted.

Further, the artificially corrected data based on the semantic segmentation results of the online random forest model is used as a training set, and the online random forest model is iteratively trained to obtain a mature online random forest model that meets the point cloud block data prediction accuracy threshold, and recorded Mature online random forest model library, including:

The segmentation prediction results of the point cloud semantic segmentation model used for feature extraction are trained using an online random forest model to obtain the semantic segmentation results of the random forest model in training. The error areas of the segmentation results are manually marked to generate the manual correction data. The artificial correction data is used as the training set input for updating the branch nodes of different decision trees in the online random forest, and the iterative training output is a mature online random forest model that meets the prediction accuracy of point cloud block data.

Further, the multi-dimensional feature space distribution similarity includes histogram similarity and SSIM structural similarity coefficient.

Further, it specifically includes: using a weakly supervised deep learning model to obtain prediction results for feature extraction corresponding to the point cloud block data.

Further, it specifically includes: using a pre-trained point cloud semantic segmentation model to obtain prediction results for feature extraction of corresponding point cloud block data.

Figure 3 is a segmented flow chart of the large-scale point cloud semantic segmentation method based on online random forest model reuse in this application. As shown in Figure 3, the entire framework is divided into two stages according to the number of models included in the model library: The training phase and the maturity phase, the maturity phase, are used for the prediction phase. Five modules are used in different stages. The process relationship between the modules is as follows:

Module 1: The input is the original large-scale remote sensing urban point cloud. The output is regular point cloud patches with initial segmentation semantic labels. The specific process is to use a pre-trained deep learning model to perform initial semantic segmentation on the original data set. Since the distribution of the original data set is significantly different from that of the pre-trained model, the generalization ability of the pre-trained model is insufficient to cope with data sets with significantly different distributions, so there is much room for improvement in the initial semantic segmentation results. Then, the initial segmentation result rule is cut into blocks and output to module two.

Module 2: The input is multiple point cloud blocks with semantic label information. The output is multiple point cloud blocks with multi-dimensional feature values and semantic label information extracted. The specific process is to execute the same feature extraction algorithm on each point cloud block. Because the machine learning method requires manual design of feature training models, and the more effective the features are, the more accurately they can describe the distribution information of point cloud blocks, so a total of 16 features were designed in three aspects: geometry, elevation, and color. The geometric aspect uses difference of normals (DoN) and covariance distribution features. DoN can measure the roughness or smoothness of a surface by comparing surface normals calculated over different scale ranges. DoN has a better effect on identifying the edges of different objects. For example, at the edges of the vehicle and the ground, the difference in normals at different scales is usually larger, resulting in larger DoN values. The covariance feature can explain the local geometry and structure of the point cloud by combining the three eigenvalues of the covariance matrix. For example, Curvature, Linearity, Flatness, Scattering, and Anisotropy. Since the semantics of buildings and roads are relatively regular and flat, while the semantic distribution of trees and cars is irregular, covariance features can play a good role in distinguishing them. The elevation features use projected elevation, projected elevation difference, projected elevation variance, flatness based on projected elevation weight, and projected density. The projected elevation will first project the point cloud on the XOY plane, and cluster the overlapping points in the Z-axis direction. Each cluster will uniformly use the maximum elevation value among all points as the projected elevation feature result. Since the road surface is flat but lower than the building, the projected elevation can express the differences between the ground and buildings, the ground and trees, and the ground and cars. The projected elevation difference counts the difference between the maximum elevation and the minimum elevation within the cluster of projected overlapping points in the Z-axis direction as a feature. The projected elevation variance calculates the offset distance of each point relative to the average projected elevation difference of the neighborhood based on the projected elevation difference. Flatness based on projected elevation weight uses the projected elevation difference as the weight coefficient of flatness to distinguish flatness features at different heights. It can effectively distinguish between roads and building roofs that are also very flat. Because the building is much higher than the road, its weight coefficient is larger, and thus the weight flatness is greater. The projection density calculates the number of points included in the cluster of overlapping points projected in the Z-axis direction. Since buildings are generally higher than trees, cars and roads, there will be a lot of overlapping points in the cluster of the building's facade after projection in the Z-axis direction, thus distinguished from other semantics. Color features use RGB, RGB entropy, and LAB. RGB information comes with the data set and helps distinguish trees from other semantics. RGB entropy counts areas where RGB changes drastically, which can enhance the extraction of different semantic edges. LAB color space is a way of describing color that can accurately represent the human visual system's perception of color and reduce the impact of shadow problems in RGB information. After module two extracts multi-dimensional features, it will output the combination of feature results and semantic label information to module three as the fitting input of the online random forest, and output the feature results to module five as the input of feature distribution statistics.

Module 3: The input is the initial segmented point cloud patch with feature results and semantic labels. The output is an accurate segmentation point cloud patch that meets the requirements and a corresponding mature online random forest model. The specific process is to manually select unsatisfactory areas in the segmentation results and generate an online random forest training set. The characteristic of online learning is that some parameters of the model will be re-updated based on the training set, and the model will be optimized to be closer to the latest training set. Therefore, the trained model will not only correct the manually corrected error areas, but also modify other approximate error areas together, thus significantly reducing the number of manual interactions. Repeat the correction-training-prediction process until the prediction result meets the threshold and the current module task is completed. In Figure 1, the threshold is set to 95%. Output the fully trained online random forest model to the model library of module four.

Module 5: The input is a point cloud block with feature results. The output is the histogram distribution of point cloud blocks in multi-dimensional feature space. The specific process is that one feature corresponds to a histogram, each histogram is divided into multiple intervals, the number of points containing the feature value in each interval is counted, and the distribution of the points contained in the interval is used as the statistical result of the current feature dimension. In addition, the module can also calculate the similarity between distributions. The specific process is to input two multi-dimensional feature space histogram distributions, use the Bhabha's coefficient to measure the similarity of the histogram distributions in each dimension, and calculate the average similarity of all dimensions.

Module 4: The input is the multi-dimensional feature space histogram distribution of the mature online random forest model and the model training point cloud block. The specific process is to record the correspondence between the mature online random forest model and the multidimensional feature space histogram distribution, and reuse the semantic segmentation of the mature model with the closest distribution based on the prediction sets of different distributions. Because the machine learning model is based on the assumption that the training set and the prediction set are independent and identically distributed, the model needs to predict on the prediction set with the same distribution to ensure better prediction results. As the number of model libraries increases, consistent model reuse can be found for prediction sets with different distributions. The segmentation accuracy of reuse is relatively better, which can solve the problem of insufficient generalization performance of traditional models on other distributed data sets.

Among them, Figure 1 is a schematic diagram of the training phase for obtaining multiple mature models and point cloud distribution characteristics online based on block-based in an exemplary embodiment of the present application. As shown in Figure 1, the purpose of the training phase is to increase the number of models in the model library. And increase the number of feature space distribution records in the database. The steps are: preprocess the point cloud data, extract features of the point cloud block data, and obtain a mature online random forest model based on the point cloud block data; the details are: extract the multi-dimensional features of the point cloud block data and Record to the point cloud feature spatial distribution database;

The manually corrected data based on the semantic segmentation results of the online random forest model is used as a training set, and the online random forest model is iteratively trained to obtain a mature online random forest model that meets the point cloud block data prediction accuracy threshold, and is recorded in the mature online random forest model. Library

Specifically: use the pre-trained deep learning model in module 1 to perform initial segmentation on the original data set and divide it into regular blocks. Then, use module 2 to extract multi-dimensional feature results. And use the manual interaction in module three to update the online random forest model, and train a mature online random forest model with a prediction accuracy above the threshold and complete semantic segmentation results. The threshold value here in Figure 1 is set to 95%. Module 5 extracts the histogram distribution of training point cloud blocks in multi-dimensional feature space. Finally, add the trained online random forest model to the model library in module four, and add the distribution of training point cloud blocks to the distribution database in module four.

Among them, Figure 2 is a schematic diagram of the prediction stage of semantic segmentation on uniformly distributed point cloud blocks based on reused mature models in an exemplary embodiment of the present application. As shown in Figure 2, the purpose of the mature stage is to use mature random data in the model library. The forest model assists in segmenting the subsequent predicted point cloud blocks, selecting the most appropriate model as much as possible, improving the prediction accuracy, thereby reducing the number of manual corrections. The steps are: for the point cloud data to be predicted, match the training point cloud block closest to its characteristic distribution in the multi-dimensional feature space distribution database, reuse the mature online random forest corresponding to the matching result to predict the point cloud data to be predicted, and based on the prediction result Train new mature models and statistical feature distributions to update the model library and distribution library. The details are: Use the point feature distribution similarity evaluation algorithm to match a feature distribution that is most similar to the feature distribution of the point cloud data to be predicted in the point cloud feature spatial distribution database; reuse the point cloud blocks with the closest distribution for training The mature random forest model is used for semantic segmentation; train a new online random forest model based on the segmentation results and statistically predict the feature distribution of point cloud blocks; judge whether the accuracy meets the threshold; if not, continue to manually correct the error area and optimize it; the output meets the usage requirements prediction results. Specifically: Figure 2 is a schematic diagram of the prediction stage of semantic segmentation on distributed consistent point cloud blocks based on multiplexed mature models in an exemplary embodiment of the present application. As shown in Figure 2, the schematic diagram of the prediction stage shows that the point cloud to be predicted is cut into blocks and multi-dimensional feature distribution is extracted, and the random forest model reuse algorithm is used to select the most appropriate mature random forest model to predict point cloud blocks to ensure that the model has consistent distribution. Semantic segmentation on prediction sets. In addition, the effect of reusing a mature random forest model will be better than that of a pre-trained deep learning model, so the areas that require manual correction are significantly reduced, or results that meet the prediction accuracy threshold can be directly output, that is, to meet user requirements. As random forest models continue to be added to the database, the number of models available for selection increases, and models with higher prediction accuracy increase accordingly. As long as the number of models reaches a certain scale, the prediction accuracy of the selected random forest model can ultimately be guaranteed to be directly Reaching above the threshold, in Figure 2, the threshold is taken as 95% to avoid the workload of subsequent manual correction. If the prediction accuracy is not satisfactory, it can also be corrected through the online random forest training process. The correction process is: taking the semantic segmentation results of the reused model as input, training a new online random forest model, and training through manual interaction. Mature, obtain a fitted mature online random forest model that meets the input prediction accuracy of the verification set, bind the mature online random forest model to the current point cloud multi-dimensional feature space distribution information to be predicted, and add the model to the model library , expand the scope of model reuse. By maintaining a point cloud semantic segmentation model library, it can include online learning models that support real-time interaction, that is, online random forest models and deep learning models that are expensive to train but have excellent results.

In actual operation, the following method is adopted in this embodiment. In order to solve the compatibility problem, the model library supports different types of semantic segmentation models. Corresponding methods can be selected according to the needs of different problems. For example, for those with high timeliness requirements, online random forest can be selected. For models with high prediction accuracy, you can choose the OMCBoost model. For data sets with complex scene details, you can choose the PointNet model. For data sets with consistent scene characteristics but extremely large scale, you can choose RandLa-Net. For data sets with large annotation workload, For data sets, you can choose weakly supervised models, etc.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components may be combined or can be integrated into another device, or some features can be ignored, or not implemented.

The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in each embodiment of the present application can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of hardware plus software function modules, such as modules one to five.

It is to be understood that the present application is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods or devices. Thus, the invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

It should also be noted that the terms "comprises," "comprises" or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements not only includes those elements, but also includes Other elements are not expressly listed or are inherent to the process, method, article or equipment. Without further limitation, an element qualified by the statement "comprises a..." does not exclude the presence of additional identical elements in the process, method, good, or device that includes the element.

The above are only examples of the present application and are not used to limit the present application. To those skilled in the art, various modifications and variations may be made to this application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application shall be included in the scope of the claims of this application.

Other embodiments of the present application will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary technical means in the technical field that are not disclosed in this application. . It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

It is to be understood that the present application is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (9)

1. A large-scale point cloud semantic segmentation method based on online random forest model reuse, which is characterized by: Preprocess point cloud data: Use pre-trained deep learning models to perform initial semantic segmentation on new datasets; Extract features from point cloud segmented data and obtain a mature online random forest model based on point cloud segmented data: Divide the new data set into chunks and train an online random forest model on each chunk, fit the results of the pre-trained deep learning model, design new features to describe the characteristics of different semantic categories, and use the online random forest model to fit the corresponding point cloud chunks The initial semantic segmentation results are supplemented by multiple manual interactions to complete error correction. For each interaction, the online learning model learns the characteristics of the interaction, optimizes the random forest model, and applies other uninteracted error areas. Each manual correction is a point cloud. blocks, obtain complete semantic segmentation results, and learn the mature online random forest model of the current point cloud block distribution characteristics; For the point cloud data to be predicted, match its feature distribution in the point cloud feature spatial distribution database, and use the mature online random forest corresponding to the matching result to predict the point cloud data to be predicted: As the number of manually corrected point cloud blocks increases, the histogram distribution is designed to count the distribution of point cloud blocks in the multi-dimensional feature space. The designed new features are used as the feature description of the histogram, and the distribution similarity calculation formula is used to establish the existing Manually correct the distribution relationship between point cloud blocks and predicted point cloud blocks, reuse the mature online random forest model under consistent conditions for the training set and prediction set to perform initial semantic segmentation, and continue to manually improve the results predicted using the online random forest model. Calibrate and train a new online random forest model, add it to the reuse model, and expand the scope of model reuse. 2. The large-scale point cloud semantic segmentation method based on online random forest model reuse according to claim 1, characterized in that the preprocessing point cloud data includes: The point cloud data is divided into regular blocks to obtain the point cloud block data, and the pre-trained model of the mainstream point cloud segmentation method is used to initially segment the point cloud block data to obtain the initial segmentation result, which is used for strong supervision online randomization. Forest training provides annotations. 3. The large-scale point cloud semantic segmentation method based on online random forest model multiplexing according to claim 2, characterized in that the feature extraction of point cloud segmented data and acquisition of features based on point cloud segmented data. Mature online random forest models, including: Among them, the designed features newly describe different semantic differences, including geometric, color, and elevation characteristics of different semantic categories used to enhance the prediction ability of the online random forest model; Extract multi-dimensional features of the point cloud block data and record them into a point cloud feature spatial distribution database; The manually corrected data based on the semantic segmentation results of the online random forest model is used as a training set, and the online random forest model is iteratively trained to obtain a mature online random forest model that meets the point cloud block data prediction accuracy threshold, and is recorded in the mature online random forest model. library. 4. The large-scale point cloud semantic segmentation method based on online random forest model reuse according to claim 3, characterized in that, for the point cloud data to be predicted, the feature distribution is matched in the point cloud feature space distribution database. , use the mature online random forest corresponding to the matching results to predict the point cloud data to be predicted, including: Use the point feature distribution similarity evaluation algorithm to match a feature distribution that is most similar to the feature distribution of the point cloud data to be predicted in the point cloud feature spatial distribution database; Corresponding to the multi-dimensional feature spatial distribution of the point cloud to be predicted and the mature online random forest model; Determine whether the prediction accuracy of the current point cloud block data obtained by the corresponding mature online random forest model meets the threshold; Output the prediction results that meet the usage requirements, or use the point cloud block data corresponding to the multi-dimensional feature space distribution of the point cloud to be predicted as the verification set input, continue to train the mature online random forest model, and obtain the verification set input. Prediction accuracy of the fitted mature online random forest model, binding the fitted mature online random forest model with the current multi-dimensional feature spatial distribution information of the point cloud to be predicted plus the multi-dimensional point cloud of the original mature online random forest model Feature distribution information. 5. The large-scale point cloud semantic segmentation method based on online random forest model reuse according to claim 4, characterized in that the point feature distribution similarity assessment algorithm is used to match a feature distribution, which is distributed in the point cloud. The feature distribution in the feature space distribution database is most similar to the feature distribution of the point cloud data to be predicted, including calculation based on multi-dimensional feature space distribution of point clouds, including: Analyze the proportion of the semantics of the current point cloud block data to be predicted in the total point cloud block data to be processed, and perform feature extraction based on the different semantics of the current point cloud block data to be predicted, and obtain the point cloud to be predicted. Multi-dimensional characteristics of block data, analyze the distribution of feature values of the point cloud block data currently to be predicted. 6. The large-scale point cloud semantic segmentation method based on online random forest model reuse according to claim 3, characterized in that the artificial correction data based on the online random forest model semantic segmentation results is used as a training set, and iterative online random The forest model is trained to obtain a mature online random forest model that meets the point cloud block data prediction accuracy threshold, and is recorded in the mature online random forest model library, including: The segmentation prediction results of the point cloud semantic segmentation model used for feature extraction are trained using an online random forest model to obtain the semantic segmentation results of the random forest model in training. The error areas of the segmentation results are manually marked to generate the manual correction data. The artificial correction data is used as the training set input for updating the branch nodes of different decision trees in the online random forest, and the iterative training output is a mature online random forest model that meets the prediction accuracy of point cloud block data. 7. The large-scale point cloud semantic segmentation method based on online random forest model reuse according to claim 1, wherein the multi-dimensional feature space distribution similarity includes histogram similarity and SSIM structural similarity coefficient. 8. The large-scale point cloud semantic segmentation method based on online random forest model reuse according to claim 2, which specifically includes: using a weakly supervised deep learning model to obtain features for corresponding point cloud block data. Extracted prediction results. 9. The large-scale point cloud semantic segmentation method based on online random forest model reuse according to claim 2, characterized in that it specifically includes: using a pre-trained point cloud semantic segmentation model to obtain corresponding point cloud block data. Prediction results of feature extraction.