CN114694022A - Spherical neighborhood based multi-scale multi-feature algorithm semantic segmentation method - Google Patents

Spherical neighborhood based multi-scale multi-feature algorithm semantic segmentation method Download PDF

Info

Publication number
CN114694022A
CN114694022A CN202210244408.8A CN202210244408A CN114694022A CN 114694022 A CN114694022 A CN 114694022A CN 202210244408 A CN202210244408 A CN 202210244408A CN 114694022 A CN114694022 A CN 114694022A
Authority
CN
China
Prior art keywords
point cloud
neighborhood
pointnet
scale
features
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210244408.8A
Other languages
Chinese (zh)
Inventor
何培培
费美琪
王靖伟
程星星
胡青峰
高科甲
廖磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Rizhao Land Spatial Data Center
North China University of Water Resources and Electric Power
Original Assignee
Rizhao Land Spatial Data Center
North China University of Water Resources and Electric Power
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Rizhao Land Spatial Data Center, North China University of Water Resources and Electric Power filed Critical Rizhao Land Spatial Data Center
Priority to CN202210244408.8A priority Critical patent/CN114694022A/en
Publication of CN114694022A publication Critical patent/CN114694022A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features

Abstract

A semantic segmentation method of a multi-scale and multi-feature algorithm based on a spherical neighborhood, the method comprising: registering the acquired point cloud data with the remote sensing image to generate point cloud data fusing RGB information; selecting a spherical neighborhood to obtain local neighborhood characteristics of point cloud data fused with RGB information, and extracting multi-scale point cloud characteristics by changing the radius of the spherical neighborhood; and combining the extracted basic features, the 5-dimensional neighborhood features of at least two scales and the xyz coordinate information of the point cloud, inputting the combination into an improved model MSMF-PointNet based on PointNet for semantic segmentation, and outputting a classification result. The method can obtain the classification precision far better than PointNet in the outdoor scene point cloud data obtained by airborne LiDAR scanning, and has the advantages of better classification of building facades, fences and the like due to the addition of characteristics such as linearity and verticality, better classification results of trees and shrubs due to the addition of roughness and total variance, better classification results of roofs and impervious grounds due to the addition of flatness.

Description

Spherical neighborhood based multi-scale multi-feature algorithm semantic segmentation method
Technical Field
The invention belongs to the technical field of remote sensing and photogrammetry, and particularly relates to a semantic segmentation method of a multi-scale and multi-feature algorithm based on a spherical neighborhood.
Background
Deep learning is a new technology capable of automatically learning and extracting advanced features of input data through a deep network structure, and is a leading-edge technology which is most influential and fastest developed in current pattern recognition, computer vision and data analysis. Before being applied to 3D data, deep learning has become an effective power for various tasks in 2D computer vision and image processing, and particularly, in 2012 AlexNe has captured the second scores of ten and a few percent higher by applying Convolutional Neural Networks (CNNs) to image recognition competitions of ImageNet, and then the deep Neural network structure mainly based on CNNs has made a major breakthrough in the fields of image classification, segmentation, recognition, and the like. However, due to the characteristics of high density, mass and no structure of the three-dimensional laser point cloud, the traditional deep learning method cannot be directly applied to the segmentation of the three-dimensional point cloud.
A large number of scholars do a large amount of work on deep learning of three-dimensional point cloud data, and in order to apply the existing neural network structure to the three-dimensional laser point cloud data, data preprocessing is required to be performed on the point cloud data before the point cloud data is input into a neural network, three methods are commonly used at present, (1) the 3D point cloud data is projected into a multi-view 2D image and then a traditional convolutional neural network is used; (2) converting the point cloud into grid voxels and then using a 3D convolutional neural network; (3) the point cloud is converted into a Graph (Graph) structure and then a neural network is convolved with a 3D Graph.
However, the data information is lost in the preprocessing process, and aiming at the defect that the data information is lost in 2017, a pioneer study work is published by a student Qi of Stanford university and the like, a deep learning model PointNet is provided and can be directly applied to three-dimensional point cloud data, so that the semantic segmentation precision is further improved. However, the PointNet network structure mainly extracts global features among point clouds, omits the extraction of associated local features between points in the point clouds, and often causes the problems of insufficient segmentation precision, poor object detail segmentation effect and the like due to insufficient extraction capability of the local features.
On one hand, due to the high density, the mass and the non-structural characteristics of the three-dimensional laser point cloud, the method has important theoretical value in researching the rapid and effective three-dimensional scene semantic segmentation algorithm. On the other hand, due to the complexity of a real natural scene and the phenomena of overlapping, shielding and the like among three-dimensional targets, research is combined with technologies in other fields, and the high-robustness, automatic and intelligent method for segmenting the target semantics of the complex three-dimensional scene is provided, so that the method has important practical significance for further researching point cloud semantics segmentation and application of the point cloud semantics in various fields.
Disclosure of Invention
In order to solve the problems, a semantic segmentation method of a multi-scale and multi-feature algorithm based on a spherical neighborhood is provided.
The object of the invention is achieved in the following way:
a semantic segmentation method of a multi-scale and multi-feature algorithm based on a spherical neighborhood, the method comprising:
s1: registering the acquired point cloud data with the remote sensing image to generate point cloud data fusing RGB information;
s2, carrying out multi-scale neighborhood design and feature extraction on the point cloud data fused with the RGB information: the method comprises the steps of obtaining local neighborhood characteristics of point cloud data fusing RGB information by researching a point cloud space index structure and selecting a spherical neighborhood, and extracting multi-scale point cloud characteristics by changing the radius of the spherical neighborhood; the point cloud features comprise basic features and covariance-based multi-features; the basic features comprise xyz coordinate information and RGB information of the point cloud; the covariance-based multi-feature comprises a 5-dimensional neighborhood feature, namely covariance-based full-variance, roughness, flatness, linearity and verticality information;
s3: and combining the extracted basic features, the 5-dimensional neighborhood features of at least two scales and the xyz coordinate information of the point cloud, inputting the combination into an improved model MSMF-PointNet based on PointNet for semantic segmentation, and outputting a classification result.
The improved model MSMF-PointNet based on PointNet comprises an improved PointNet network and at least two Mini-PointNet networks; inputting xyz coordinate information and RGB information of the point cloud into an improved PointNet network, wherein 64-dimensional point features and 1024-dimensional global features are output by the improved PointNet network; combining the 5-dimensional neighborhood characteristics of at least two scales with the xyz coordinate information of the point cloud and inputting the combination into a Mini-pointet network; the Mini-PointNet outputs two 256-dimensional characteristic vectors, the two parts of output data are fully connected, and the data are input into a softmax classifier for classification.
The improved PointNet network comprises six layers, namely a first T-Net point cloud rotation transformation, a first sensor mlp, a second T-Net point cloud rotation transformation, a second sensor mlp, a third sensor mlp and a Max pooling network from input to output.
The Mini-pointet network comprises 4 layers, namely T-Net point cloud rotation transformation, two layers of perceptrons mlp and a Max poiling network from input to output.
The xyz coordinate information of the point cloud is calculated according to the data recorded by the GPS, the INS and the laser range finder of the system; the RGB information is acquired by the imaging device.
In the step S1, the registration is performed by assigning spectral information of different pixel points of each band of the image to the point cloud data mainly through the function of "value to point" in arcs.
An electronic device, comprising:
at least one processor;
and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method described above.
A non-transitory computer readable storage medium storing computer instructions, wherein,
the computer instructions are for causing the computer to perform the method described above.
A computer program product comprising a computer program which, when executed by a processor, implements the method described above.
The invention has the beneficial effects that: (1) the method can obtain the classification precision far better than PointNet in the outdoor scene point cloud data obtained by airborne LiDAR scanning, and has the advantages of better classification of building facades, fences and the like due to the addition of characteristics such as linearity and verticality, better classification results of trees and shrubs due to the addition of roughness and total variance, better classification results of roofs and impervious grounds due to the addition of flatness.
(2) The method of the invention adds spectral information and other geometric characteristics, and trains based on deep learning, thereby effectively making up the defects of the spatial geometric characteristics of the point cloud, improving the point cloud classification precision, and having higher classification precision for roofs, impervious grounds and trees.
Drawings
Fig. 1 is a schematic diagram of index creation based on a virtual regular grid.
Fig. 2 is a comparison graph of classification effect of different radii.
Figure 3 is a MSMF-PointNet network model architecture.
FIG. 4 shows the classification accuracy of various types of ground features and the overall classification accuracy at different scales.
FIG. 5 is a flow chart of a semantic segmentation model of a multi-scale multi-feature algorithm based on spherical neighborhood.
Detailed Description
The present invention will be described in further detail with reference to fig. 1 to 5 and the following detailed description.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure as claimed. Unless otherwise defined, all technical and scientific terms used herein have the same technical meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
A semantic segmentation method of a multi-scale and multi-feature algorithm based on a spherical neighborhood, the method comprising:
s1: registering the acquired point cloud data with the remote sensing image to generate point cloud data fusing RGB information;
s2, performing multi-scale neighborhood design and feature extraction on the point cloud data fused with the RGB information: the method comprises the steps of obtaining local neighborhood characteristics of point cloud data fusing RGB information by researching a point cloud space index structure and selecting a spherical neighborhood, and extracting multi-scale point cloud characteristics by changing the radius of the spherical neighborhood; the point cloud features comprise basic features and multi-features based on covariance; the basic features comprise xyz coordinate information and RGB information of the point cloud; the covariance-based multi-feature comprises a 5-dimensional neighborhood feature, namely covariance-based full variance, roughness, flatness, linearity and verticality information;
s3: and combining the extracted basic features, the 5-dimensional neighborhood features of at least two scales and the xyz coordinate information of the point cloud, inputting the combination into an improved model MSMF-PointNet based on PointNet for semantic segmentation, and outputting a classification result.
As shown in fig. 3, the improved model MSMF-PointNet based on PointNet comprises an improved PointNet network and at least two Mini-PointNet networks; inputting xyz coordinate information and RGB information of the point cloud into an improved PointNet network, wherein 64-dimensional point features and 1024-dimensional global features are output by the improved PointNet network; combining the 5-dimensional neighborhood characteristics of at least two scales with the xyz coordinate information of the point cloud and inputting the combination into a Mini-pointet network; the Mini-PointNet outputs two 256-dimensional characteristic vectors, the two parts of output data are fully connected, and the data are input into a softmax classifier for classification.
The improved PointNet network comprises six layers, namely a T-Net point cloud rotation transformation, a first sensor mlp, a T-Net, a second sensor mlp, a third sensor mlp and a Max Pooling network from input to output in sequence.
The Mini-pointet network comprises 4 layers, namely T-Net point cloud rotation transformation, two layers of perceptrons mlp and a Max poiling network from input to output.
Further, a point cloud feature extraction method based on spherical neighborhood is researched:
in order to improve the efficiency of query and retrieval of point cloud data, a spatial index of the point cloud data needs to be constructed, and common indexing methods include a quadtree, an octree, a k-d tree and the like. Because the k-d tree indexing efficiency is high, the k-d tree indexing method is widely applied to point cloud data, different space indexes are adopted according to specific data processing requirements, and for the subsequent filtering algorithm design, the k-d tree is selected to construct the space index of the point cloud, and space division and neighbor search are carried out.
After the K-d tree is established, a neighborhood point query mode needs to be further selected, the scale difference between different objects in an outdoor environment is large, and the description capacity of the features on the different objects is determined by the neighborhood selection mode in the feature extraction process. The single-point classification of the point cloud depends on the extraction of local features, and the local features are extracted from a neighborhood point set of the selected points, so the single-point classification is closely related to the selected local neighborhood region.
Radius query is to give a target point and a threshold of a query distance (with the target point as a circle center and the query distance as a radius R), find out all data (data within the radius) whose distance from the query point is smaller than the threshold from the data set, and there are three common local neighborhood choices: the schematic diagram is shown in fig. 1, wherein (a) is a spherical radius neighborhood, (b) is a columnar neighborhood, and (c) is a K neighbor neighborhood.
Due to the spatial anisotropy of the three-dimensional urban scene, the classification effect under the spherical neighborhood is good. Compared with the K nearest neighbor method, the spherical neighborhood corresponds to a fixed part in the space and is relatively low influenced by the density of the point cloud, so the method selects the spherical neighborhood method.
And obtaining a multi-scale point cloud characteristic by changing the size of the spherical neighborhood radius R.
The selection of the scale directly affects the classification accuracy of the point cloud, so that a proper scale, namely the radius value R in the spherical neighborhood method, needs to be selected according to the ground features in the scene. Too large radius, too many points contained in the spherical neighborhood can greatly increase the calculation time and reduce the efficiency. Generally, the minimum value of the radius is greater than the average density of the point cloud and increases upwards regularly, in order to select the most suitable radius parameter and verify that the segmentation effect of the multi-scale fused point cloud features is better, a part of the Vaihingen dataset is selected, point cloud features calculated by the radius parameter R being 0.4m,0.8m,1.2m,2.0m and four combinations of the point cloud features 0.4+0.8, 0.8+1.2, 0.8+2.0 and 1.2+2.0 are selected and respectively input into the PointNet network, the rest are the same except that the input is different, and a histogram of the classification effect is shown in fig. 2: as shown by comparing and analyzing the histogram in fig. 2, the combined classification accuracy of the multi-scale R-0.8 +1.2 is the highest, and the accuracy of the single-scale R-0.8 is the highest, and the network has not been upgraded only to determine the most suitable radius parameter, so that there is no specific precision, but only the precision is relatively high or low. Therefore, the invention considers comprehensively, and the radius R of the final spherical neighborhood is 0.8m and 1.2 m.
Selection of basic and common features of point clouds and covariance-based multi-features
The basic characteristics of the point cloud are xyz coordinate information and RGB information of the point cloud, xyz can be calculated according to data recorded by a GPS, an INS and a laser range finder of the system, the RGB information is acquired by an imaging device, and the spectral information is beneficial to distinguishing vegetation and other ground features, so that in order to improve the classification accuracy, airborne LiDAR point cloud and multispectral aerial images are generally fused to generate point cloud data with spectral information so as to realize spectral information supplement of the point cloud data.
Under the condition that the inside and outside orientation elements of the aerial photography film are known, the three-dimensional coordinates of the LiDAR point cloud are substituted into a collinear condition equation, the pixel position of the corresponding three-dimensional point on the image can be calculated, and then the gray values of Near Infrared (NIR), red (R) and green (G) channels are obtained through resampling. The collinearity condition equation can be expressed as:
Figure BDA0003543591540000061
wherein f is an image focal length, XYZ is a three-dimensional coordinate of a ground point, and (X)S,YS,ZS) Three line elements which are external orientation elements (a)1,b1,c1,a2,b2,c2,a3,b3,c3) Is a rotation matrix parameter calculated from three angular elements in the exterior orientation elements. And after calculating spectral information (NIR, R and G three-channel gray values) corresponding to each laser foot point, combining the three-dimensional coordinates and the spectral information of the point cloud to obtain enhanced point cloud data which is used as input data of subsequent point cloud classification, and fusing the point cloud and the image.
The covariance characteristic is a common characteristic in the point cloud, can represent the shape of an object or the distribution state of local point cloud, and has important application in point cloud data processing. For three-dimensional point cloud data, D ═ P can be usediN represents the point cloud, where N is the number of points in the point cloud D. To this end, first the mean and covariance matrix of the current point cloud are calculated:
Figure BDA0003543591540000062
Figure BDA0003543591540000063
wherein, the formula of the unbiased estimation of C is:
Figure BDA0003543591540000064
secondly, since the covariance matrix C is a positive definite matrix, the characteristic value of the matrix C is known from the characteristics of the symmetric positive definite matrix: lambda [ alpha ]1≥λ2≥λ3Not less than 0, and the corresponding feature vector e1,e2,e3Perpendicular to each other, an orthogonal coordinate system can be formed, i.e. C can be expressed as:
Figure BDA0003543591540000065
and according to the size condition of the characteristic value, the morphological characteristics of the local neighborhood point cloud can be obtained. At λ1>>λ2≈λ3And then, the local neighborhood point clouds are distributed in a linear way. At λ1≈λ2>>λ3When the point cloud is distributed in a surface shape. Lambda [ alpha ]1>>λ2≈λ3And the point clouds are distributed in a three-dimensional off-line manner. Using a combined linearity LλFlatness PλAnd dispersion SλTo represent one-, two-, and three-dimensional features:
Lλ＝(λ1-λ2)/λ1 (6)
Pλ＝(λ2-λ3)/λ1 (7)
Sλ＝λ3/λ1 (8)
the sum of the three is 1. And (4) obtaining a covariance matrix constructed by the neighborhood of the selected point according to the formula (3), and further obtaining an eigenvalue of the covariance matrix and a corresponding eigenvector.
The invention selects the whole range difference besides the linearity and the flatness:
Figure BDA0003543591540000071
perpendicularity:
Vλ＝1-|Z·N| (10)
wherein Z is a unit vector in the vertical direction, and N is a normal vector of the point.
And intercepting a Vaihingen experimental data set, displaying each neighborhood characteristic when R is 0.8, visualizing the estimation result of each characteristic value, clearly seeing that the characteristic values of different ground objects are obviously different, and beneficially distinguishing.
The point cloud segmentation method based on the improved PointNet is researched:
the invention provides an MSMF-PointNet terrain classification segmentation algorithm fusing multi-scale and multi-neighborhood characteristics based on an improved PointNet network, overcomes the defects of insufficient utilization of point-to-point cloud characteristics and lack of local characteristics of the original PointNet network, draws the network parameter settings of PointNet and PointNet + +, and simultaneously sets corresponding R, O and multi-scale local characteristics (xyz, RGB, R is 0.8 and 1.2) based on 16-dimensional spherical neighborhoodλ,Pλ,Lλ,Vλ) And replacing the original single-point xyz information as a new data source for training classification of the classifier.
The specific improvement idea is as follows: aiming at the problem of the increase of the dimensionality of the feature space of the fused point cloud, the number of channels is increased by adjusting the dimensionality of an input transformation matrix, so that the matrix is changed from the original processing of three-dimensional feature vectors into the processing of six-dimensional and eight-dimensional feature vectors after fusion; aiming at the increase of data volume caused by the expansion of the feature space of the fused point cloud data, the depth features of the point cloud are fully extracted by deepening the number of network layers of an MLP layer; aiming at the problem of lack of local neighborhood characteristics, point cloud characteristics calculated by spherical neighborhood are used as input, two mini-PointNet characteristic extraction networks are built, and multi-neighborhood point cloud characteristics with different scales are extracted. The MSMF-PointNet network model architecture is shown in FIG. 3.
Experimental validation and analysis
And (3) measuring area data: the city test data set of Vaihingen, germany, provided by isps was used. The dataset is a collection of ALS (aerodynamic Laser scanning) point clouds, consisting of 10 strips captured by the Leica ALS50 system, with an average overlap between two adjacent strips of around 30%. The ground resolution of the multispectral aerial image is 8cm, the size of each image is 7680 pixels × 13824 pixels, and the medial and lateral orientation elements of the image are provided. At present, the point cloud marked by the data is divided into 9 categories as algorithm evaluation standards, which contain abundant geographic environments, urban environments and building types, and can fully verify the algorithm. Aerial image data and airborne LiDAR point cloud data corresponding to the aerial image data are provided in the Vaihingen dataset, and the average point distance of the LiDAR point cloud data is 4points/m 2. The classification categories of LiDAR point cloud data are labeled in reference data provided by the Vaihingen data set, after relevant building features are extracted, a classifier can be directly trained to classify LiDAR point cloud test data, and finally accurate extraction of building points is achieved. Training and testing was performed using the PointNet network and proposed algorithm, the semantic labels of the data concentration points contained 9 classes (power line, vehicle, low plant, impervious surface, fence, roof, wall, shrub, tree). The training data set contained 753,876 points in total, and the test data set contained 411,722 points.
TABLE 1 ISPRS Vaihingen dataset Point cloud categories and numbers
Figure BDA0003543591540000081
TABLE 2 Vaihingen data parameters
Figure BDA0003543591540000091
In order to verify the establishment of the multi-scale and multi-neighborhood feature algorithm based on PointNet, the single-scale single feature (SS) (only containing xyz information), the single-scale multi-feature (SM) and the multi-scale multi-feature (MM) are verified respectively by using the same data, and the single-scale single feature is a PointNet network only containing the xyz information; the single-scale multi-feature method comprises the steps that only one mini-PointNet is added on the basis of the original PointNet, the neighborhood feature when R is 0.8 is calculated and is input into a mini network together with xyz, and meanwhile point cloud and image fused xyz RGB information are combined to serve as input of the original PointNet network; the multi-scale multi-feature is the MSMF-PointNet algorithm proposed herein, which calculates the point cloud features when R is 0.8 and R is 1.2, respectively, and outputs them together with xyz into the mini network, and at the same time, outputs the xyz RGB information into the PointNet network instead of the point cloud single feature. The classification accuracy is shown in fig. 4.
As can be seen from fig. 4, the overall classification accuracy of the proposed PointNet-based multi-scale multi-neighborhood algorithm (MM) is the highest, the classification effect is the best, the overall accuracy reaches 88.1%, and from the quantitative evaluation result, the point cloud classification accuracy of all terrain features is greatly improved after the spectral information and the covariance-based features (SM) are fused. Wherein, the overall precision of SM is improved by 19.2 percentage points compared with SS; after the spectrum information and the covariance characteristics (MM) of two scales are fused, the precision is further improved, and compared with the SM of a single scale, the precision is improved by 8.4 percentage points. Wherein the precision of the impervious ground, the tree and the roof is improved most obviously by 19.7 percent, 30 percent and 31.3 percent respectively. Therefore, the attribute information of the point cloud of the multi-scale and multi-neighborhood can be effectively enhanced, and therefore, the more accurate classification of various ground object targets is realized. The specific classification result pair is a final classification result graph of true value, SS, SM and MM in sequence as shown in FIG. 4. On the whole, the final classification result of the MM is almost the same as the true value, and the classification effect is particularly prominent particularly on roofs, trees and impervious ground. It can be seen from SS that the ground features are classified into roofs and trees by mistake, when local features based on spherical neighborhood are added into SM, the classification precision of houses and trees is greatly improved, SS is simply classified into trees, roofs and shrubs according to terrain and elevation, and SM is classified into roofs, trees, shrubs, impervious ground, fences and cars by refining classification. The MM with multiple scales is more accurate and finer than SM, as the upper area circled on the graph, a bulge is obviously seen in the middle, a narrow ditch with larger height difference at the left side and the right side is formed on the spot, when R is equal to 0.8, the right-side ground with the higher height of the narrow ditch is classified as a roof, and when the local characteristic that R is equal to 1.2 is added, the waterproof ground is accurately classified.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various changes and modifications can be made without departing from the overall concept of the present invention, and these should also be considered as the protection scope of the present invention.

Claims (9)

1. A semantic segmentation method of a multi-scale and multi-feature algorithm based on spherical neighborhood is characterized by comprising the following steps: the method comprises the following steps:
s1: registering the acquired point cloud data with the remote sensing image to generate point cloud data fusing RGB information;
s2, performing multi-scale neighborhood design and feature extraction on the point cloud data fused with the RGB information: the method comprises the steps of obtaining local neighborhood characteristics of point cloud data fusing RGB information by researching a point cloud space index structure and selecting a spherical neighborhood, and extracting multi-scale point cloud characteristics by changing the radius of the spherical neighborhood; the point cloud features comprise basic features and multi-features based on covariance; the basic features comprise xyz coordinate information and RGB information of the point cloud; the covariance-based multi-feature comprises a 5-dimensional neighborhood feature, namely covariance-based full variance, roughness, flatness, linearity and verticality information;
s3: and combining the extracted basic features, the 5-dimensional neighborhood features of at least two scales and the xyz coordinate information of the point cloud, inputting the combination into an improved model MSMF-PointNet based on PointNet for semantic segmentation, and outputting a classification result.
2. The semantic segmentation method based on the spherical neighborhood multi-scale multi-feature algorithm of claim 1, characterized in that: the improved model MSMF-PointNet based on PointNet comprises an improved PointNet network and at least two Mini-PointNet networks; inputting xyz coordinate information and RGB information of the point cloud into an improved PointNet network, wherein 64-dimensional point features and 1024-dimensional global features are output by the improved PointNet network; the 5-dimensional neighborhood characteristics of at least two scales and the xyz coordinate information of the point cloud are combined and input into a Mini-pointet network; the Mini-PointNet outputs two 256-dimensional feature vectors, and the output data of the two parts are fully connected and input into a softmax classifier for classification.
3. The semantic segmentation method based on the spherical neighborhood multi-scale multi-feature algorithm of claim 2, characterized in that: the improved PointNet network comprises six layers, namely a first T-Net point cloud rotation transformation, a first sensor mlp, a second T-Net point cloud rotation transformation, a second sensor mlp, a third sensor mlp and a Max pooling network from input to output.
4. The semantic segmentation method based on the spherical neighborhood multi-scale multi-feature algorithm of claim 2, characterized in that: the Mini-pointenet network comprises 4 layers, namely T-Net point cloud rotation transformation, two layers of sensors mlp and a Max poiling network from input to output.
5. The semantic segmentation method based on the spherical neighborhood multi-scale multi-feature algorithm of claim 1, characterized in that: the xyz coordinate information of the point cloud is calculated according to the GPS, INS and the data recorded by the laser range finder of the system; the RGB information is acquired by the imaging device.
6. The semantic segmentation method based on the spherical neighborhood multi-scale multi-feature algorithm of claim 1, characterized in that: in the S1, the registration mainly assigns the spectral information of different pixel points of each wave band of the image to the point cloud data through the function of 'extracting values to points' in Arcigs.
7. An electronic device, comprising:
at least one processor;
and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.
8. A non-transitory computer readable storage medium storing computer instructions, wherein,
the computer instructions are for causing the computer to perform the method of any one of claims 1-6.
9. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-6.
CN202210244408.8A 2022-03-11 2022-03-11 Spherical neighborhood based multi-scale multi-feature algorithm semantic segmentation method Pending CN114694022A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210244408.8A CN114694022A (en) 2022-03-11 2022-03-11 Spherical neighborhood based multi-scale multi-feature algorithm semantic segmentation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210244408.8A CN114694022A (en) 2022-03-11 2022-03-11 Spherical neighborhood based multi-scale multi-feature algorithm semantic segmentation method

Publications (1)

Publication Number Publication Date
CN114694022A true CN114694022A (en) 2022-07-01

Family

ID=82138380

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210244408.8A Pending CN114694022A (en) 2022-03-11 2022-03-11 Spherical neighborhood based multi-scale multi-feature algorithm semantic segmentation method

Country Status (1)

Country Link
CN (1) CN114694022A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116977572A (en) * 2023-09-15 2023-10-31 南京信息工程大学 Building elevation structure extraction method for multi-scale dynamic graph convolution

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116977572A (en) * 2023-09-15 2023-10-31 南京信息工程大学 Building elevation structure extraction method for multi-scale dynamic graph convolution
CN116977572B (en) * 2023-09-15 2023-12-08 南京信息工程大学 Building elevation structure extraction method for multi-scale dynamic graph convolution

Similar Documents

Publication Publication Date Title
Xia et al. Geometric primitives in LiDAR point clouds: A review
CN110415342A (en) A kind of three-dimensional point cloud reconstructing device and method based on more merge sensors
CN109883401B (en) Method and system for measuring visual field of city mountain watching
CN103337052B (en) Automatic geometric correcting method towards wide cut remote sensing image
CN103218787B (en) Multi-source heterogeneous remote sensing image reference mark automatic acquiring method
CN110866079A (en) Intelligent scenic spot real scene semantic map generating and auxiliary positioning method
Hu et al. Research on a single-tree point cloud segmentation method based on UAV tilt photography and deep learning algorithm
CN113989797A (en) Three-dimensional dynamic target detection method and device based on voxel point cloud fusion
Karsli et al. Automatic building extraction from very high-resolution image and LiDAR data with SVM algorithm
CN114120067A (en) Object identification method, device, equipment and medium
Liang et al. A novel skyline context descriptor for rapid localization of terrestrial laser scans to airborne laser scanning point clouds
Yao et al. Automated detection of 3D individual trees along urban road corridors by mobile laser scanning systems
CN114689038A (en) Fruit detection positioning and orchard map construction method based on machine vision
CN114694022A (en) Spherical neighborhood based multi-scale multi-feature algorithm semantic segmentation method
CN114639115A (en) 3D pedestrian detection method based on fusion of human body key points and laser radar
CN113963259A (en) Street view ground object multi-dimensional extraction method and system based on point cloud data
CN112906719A (en) Standing tree factor measuring method based on consumption-level depth camera
CN111487643A (en) Building detection method based on laser radar point cloud and near-infrared image
CN113362458B (en) Three-dimensional model interpretation method for simulating multi-view imaging, terminal and storage medium
CN112146647B (en) Binocular vision positioning method and chip for ground texture
Li et al. PointLAE: A Point Cloud Semantic Segmentation Neural Network via Multifeature Aggregation for Large-Scale Application
CN113379738A (en) Method and system for detecting and positioning epidemic trees based on images
Jiao et al. Individual building rooftop and tree crown segmentation from high-resolution urban aerial optical images
Bai et al. Application of unmanned aerial vehicle multi-vision image 3D modeling in geological disasters
Cui et al. A Review of Indoor Automation Modeling Based on Light Detection and Ranging Point Clouds.

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination