CN116363099A - Fine granularity analysis method, device, medium and equipment for three-dimensional point cloud data - Google Patents
Fine granularity analysis method, device, medium and equipment for three-dimensional point cloud data Download PDFInfo
- Publication number
- CN116363099A CN116363099A CN202310332687.8A CN202310332687A CN116363099A CN 116363099 A CN116363099 A CN 116363099A CN 202310332687 A CN202310332687 A CN 202310332687A CN 116363099 A CN116363099 A CN 116363099A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- dimensional point
- cloud data
- fine
- dividing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 82
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 58
- 239000011159 matrix material Substances 0.000 claims abstract description 55
- 239000013598 vector Substances 0.000 claims abstract description 52
- 238000000034 method Methods 0.000 claims abstract description 31
- 238000012952 Resampling Methods 0.000 claims abstract description 18
- 230000002776 aggregation Effects 0.000 claims abstract description 18
- 238000004220 aggregation Methods 0.000 claims abstract description 18
- 238000005192 partition Methods 0.000 claims abstract description 13
- 238000010276 construction Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 9
- 238000011176 pooling Methods 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 8
- 238000000638 solvent extraction Methods 0.000 claims description 2
- 238000013528 artificial neural network Methods 0.000 abstract description 14
- 238000002372 labelling Methods 0.000 abstract description 10
- 238000005070 sampling Methods 0.000 description 13
- 230000009466 transformation Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 238000007726 management method Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000004931 aggregating effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a three-dimensional point cloud data fine-granularity analysis method, a device, a medium and equipment, which relate to the technical field of data processing, adopt part information in CAD data and combine a neural network to learn geometric feature priori thereof, can directly migrate to the three-dimensional point cloud data to carry out fine-granularity analysis on the three-dimensional point cloud data, do not need a special three-dimensional point cloud data set with fine-granularity labeling information to serve as network training data, and solve the problem that the traditional three-dimensional point cloud analysis algorithm needs to rely on three-dimensional point cloud training data with semantic labels during learning. The scheme is as follows: dividing three-dimensional point cloud data to be analyzed; deleting blank blocks in the dividing blocks; resampling the point set in each partition block to 512 points; extracting a feature vector for each point in each dividing block; constructing a characteristic distance matrix; performing low-rank decomposition on the characteristic distance matrix; constructing a graph model; the graph model is input into a pre-trained aggregation network. The method is used for fine granularity analysis of the three-dimensional point cloud data.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a method, an apparatus, a medium, and a device for resolving fine granularity of three-dimensional point cloud data.
Background
Three-dimensional point cloud fine-grained resolution is a fundamental problem in three-dimensional vision and graphics. The existing learning-based fine-grained three-dimensional point cloud analysis method is generally described as a semantic labeling problem.
The existing three-dimensional point cloud fine granularity analysis method mainly has the following defects: in the prior art, semantic segmentation is performed on a three-dimensional point cloud by using a data set with semantic annotation, such as: chair seats, backrests, etc., however, many application scenarios require fine-grained segmentation of three-dimensional point clouds; one key prerequisite of the existing method is that a large training dataset of three-dimensional point cloud data with fine-grained segmentation and semantic tags can be obtained, but such datasets are difficult to construct; many fine-grained parts may not even have a clear semantic label, resulting in a label that is obscured; for these reasons, there are indeed many annotation ambiguities for existing datasets; the existing three-dimensional point cloud deep learning model framework is limited by conditions such as GPU hardware and the like, and very large point cloud data is generally difficult to directly process.
Disclosure of Invention
The invention provides a three-dimensional point cloud data fine granularity analysis method, a device, a medium and equipment, comprising the following steps: dividing the three-dimensional point cloud data to be analyzed to obtain dividing blocks; deleting blank blocks in the dividing blocks; resampling the point set in each partition block to 512 points; extracting a feature vector for each point in each dividing block; constructing a characteristic distance matrix; performing low-rank decomposition on the characteristic distance matrix, wherein the decomposition result of each dividing block is the fine granularity analysis result of the point set in the dividing block; constructing a graph model; compared with the prior art, the method adopts a large amount of part information in common CAD data and combines the geometric feature priori of the part information learned by a neural network, so that the part information can be directly transferred to the three-dimensional point cloud data for carrying out fine-grained analysis, a special three-dimensional point cloud data set with fine-grained labeling information is not required to be used as network training data, and the problem that the traditional three-dimensional point cloud analysis algorithm needs to rely on the three-dimensional point cloud training data with semantic labels during learning is solved.
In order to achieve the above purpose, the invention adopts the following technical scheme:
the first aspect of the invention provides a three-dimensional point cloud data fine granularity analysis method, which comprises the following steps:
and dividing the three-dimensional point cloud data to be analyzed to obtain dividing blocks.
And deleting blank blocks in the dividing blocks.
The set of points in each of the partitioned blocks is resampled to 512 points.
For each of the points in each of the divided blocks, a feature vector is extracted.
Constructing a characteristic distance matrix; the feature distance is the distance of the feature vector between every two points.
And carrying out low-rank decomposition on the characteristic distance matrix, wherein the decomposition result of each dividing block is the fine granularity analysis result of the point set in the dividing block.
And constructing a graph model.
And inputting the graph model into a pre-trained aggregation network, wherein an output aggregation result is a three-dimensional point cloud data fine-granularity analysis result.
Further, the three-dimensional point cloud data fine-granularity analysis method for aggregating the pre-training of the network comprises the following steps:
a feature vector is extracted for each of the nodes in the graph model.
Pooling the features of the points contained within each of the nodes and noting as local geometric information encoding for that node.
And extracting feature vectors of each node by adopting a graph convolution layer, and recording the feature vectors as context geometric feature codes of the node.
The local geometric information code and the context geometric feature code of each node are taken as comprehensive features.
And constructing a node characteristic similarity matrix.
And carrying out low-rank decomposition on the node characteristic similarity matrix, wherein a decomposition result is a three-dimensional point cloud data fine-granularity analysis result.
Further, the method for resolving three-dimensional point cloud data fine granularity divides three-dimensional point cloud data to be resolved to obtain dividing blocks, and the method comprises the following steps:
and dividing the three-dimensional point cloud data to be analyzed by using the volume resolution of 7 x 7 to obtain 343 divided blocks.
Further, the three-dimensional point cloud data fine-granularity analysis method extracts a feature vector for each point in each of the dividing blocks, including:
and extracting a feature vector by taking PointNet++ as a backbone network for each point in each partitioning block.
The second aspect of the present invention provides a three-dimensional point cloud data fine-granularity analysis device, including:
the dividing unit is used for dividing the three-dimensional point cloud data to be analyzed to obtain dividing blocks.
And the deleting unit is used for deleting the blank blocks in the dividing blocks.
And the resampling unit is used for resampling the point set in each dividing block into 512 points.
And an extraction unit configured to extract a feature vector for each of the points in each of the divided blocks.
The first construction unit is used for constructing a characteristic distance matrix; the feature distance is the distance of the feature vector between every two points.
And the low-rank decomposition unit is used for carrying out low-rank decomposition on the characteristic distance matrix, and the decomposition result of each division block is the fine granularity analysis result of the point set in the division block.
And the second construction unit is used for constructing the graph model.
The input unit is used for inputting the graph model into a pre-trained aggregation network, and the output aggregation result is a three-dimensional point cloud data fine-granularity analysis result.
Further, the three-dimensional point cloud data fine granularity analysis device, the input unit includes:
and the first extraction module is used for extracting the characteristic vector for each node in the graph model.
And the pooling module is used for pooling the characteristics of the points contained in each node and recording the characteristics as local geometric information codes of the node.
And the second extraction module is used for extracting the characteristic vector of each node by adopting the graph convolution layer and recording the characteristic vector as the context geometric characteristic code of the node.
As a module for encoding said local geometry information and said contextual geometry feature of each of said nodes as its integrated features.
And the construction module is used for constructing the node characteristic similarity matrix.
And the low-rank decomposition module is used for carrying out low-rank decomposition on the node characteristic similarity matrix, and the decomposition result is a three-dimensional point cloud data fine-granularity analysis result.
A third aspect of the present invention provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the three-dimensional point cloud data fine-granularity analysis method is implemented.
A fourth aspect of the present invention provides an electronic apparatus, comprising: the system comprises a memory and one or more processors, wherein the memory stores a computer program, and the computer program realizes the three-dimensional point cloud data fine-granularity analysis method when being executed by the one or more processors.
The invention provides a three-dimensional point cloud data fine granularity analysis method, a device, a medium and equipment, comprising the following steps: dividing the three-dimensional point cloud data to be analyzed to obtain dividing blocks; deleting blank blocks in the dividing blocks; resampling the point set in each partition block to 512 points; extracting a feature vector for each point in each dividing block; constructing a characteristic distance matrix; performing low-rank decomposition on the characteristic distance matrix, wherein the decomposition result of each dividing block is the fine granularity analysis result of the point set in the dividing block; constructing a graph model; compared with the prior art, the method adopts a large amount of part information in common CAD data and combines the geometric feature priori of the part information learned by a neural network, so that the part information can be directly transferred to the three-dimensional point cloud data for carrying out fine-grained analysis, a special three-dimensional point cloud data set with fine-grained labeling information is not required to be used as network training data, and the problem that the traditional three-dimensional point cloud analysis algorithm needs to rely on the three-dimensional point cloud training data with semantic labels during learning is solved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are used in the description of the embodiments will be briefly described below, which are only for the purpose of illustrating the embodiments and are not to be construed as limiting the present invention.
FIG. 1 is a flow chart of a three-dimensional point cloud data fine-granularity analysis method according to an embodiment of the invention;
FIG. 2 is a flow chart of another method for fine-granularity analysis of three-dimensional point cloud data according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a three-dimensional point cloud data fine-granularity analysis device according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a structure of another three-dimensional point cloud data fine-granularity analysis device according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a composition structure of a three-dimensional point cloud data fine-granularity analysis electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs; the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention and the terms "comprising" and "having" and any variations thereof, as described in the specification and claims of the invention and the above description of the drawings, are intended to cover a non-exclusive inclusion.
In the description of embodiments of the present invention, the technical terms "first," "second," and the like are used merely to distinguish between different objects and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated, a particular order or a primary or secondary relationship. In the description of the embodiments of the present invention, the meaning of "plurality" is two or more unless explicitly defined otherwise.
In the description of the embodiments of the present invention, the term "and/or" is merely an association relationship describing an association object, and indicates that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship.
In the description of the embodiments of the present invention, the term "plurality" means two or more (including two), and similarly, "plural sets" means two or more (including two), and "plural sheets" means two or more (including two).
In the description of the embodiments of the present invention, the orientation or positional relationship indicated by the technical terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. are based on the orientation or positional relationship shown in the drawings, and are merely for convenience of description and simplification of the description, and do not indicate or imply that the apparatus or element referred to must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the embodiments of the present invention.
In the description of the embodiments of the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured" and the like should be construed broadly and may be, for example, fixedly connected, detachably connected, or integrally formed; or may be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communicated with the inside of two elements or the interaction relationship of the two elements. The specific meaning of the above terms in the examples of the present invention will be understood by those skilled in the art according to the specific circumstances.
Example 1
An embodiment of the present invention provides a three-dimensional point cloud data fine-granularity analysis method, as shown in fig. 1, including:
101. and dividing the three-dimensional point cloud data to be analyzed to obtain dividing blocks.
The point cloud is a set of points formed by acquiring the space coordinates of each sampling point on the surface of an object, and is usually scanned by a laser radar and comprises information such as the three-dimensional coordinates, the intensity and the like of the points.
What needs to be explained here is: in this embodiment, the tool, method, number of divided tools and methods for dividing the three-dimensional point cloud data to be resolved are not limited, and an implementer may divide the three-dimensional point cloud data according to actual needs, for example: the division is performed using a volume resolution of 7 x 7, 343 divided blocks are obtained.
102. And deleting blank blocks in the dividing blocks.
The blank block, i.e. the block does not contain any point cloud data.
103. The set of points in each of the partitioned blocks is resampled to 512 points.
Because the number of points in each partitioned block is different, the set of points in each partitioned block is resampled to 512 points using the furthest point sampling method for neural network training. The neural network training refers to training an artificial neural network, inputting enough samples into the network, and adjusting the structure of the network through a certain algorithm to enable the output of the network to be consistent with an expected value, wherein the process is the neural network training.
What needs to be explained here is: in this embodiment, the resampling method of the point set in each partition block, the number of resampled points and the like are not limited, and the "furthest point sampling method" and the "512" are only one simple example, and when the present invention is implemented, an implementer can select according to actual situations and own requirements.
104. For each of the points in each of the divided blocks, a feature vector is extracted.
Feature vector: it is a term in mathematical disciplines that the eigenvector (eigenvector) of a linear transformation is a non-degenerate vector whose direction is unchanged under the transformation, and the scale of the vector scaled under this transformation is called its eigenvalue (eigenvalue). A linear transformation can be generally described entirely by its eigenvalues and eigenvectors, and a set of eigenvectors of the same eigenvalue is called an eigenvalue space.
105. Constructing a characteristic distance matrix; the feature distance is the distance of the feature vector between every two points.
Matrix: refers to a two-dimensional data table arranged longitudinally and horizontally.
106. And carrying out low-rank decomposition on the characteristic distance matrix, wherein the decomposition result of each dividing block is the fine granularity analysis result of the point set in the dividing block.
Rank: to remove redundant equations from the system of equations, a "matrix rank" is introduced. The rank of a matrix measures the correlation between the rows and columns of the matrix.
Low rank: if X is a matrix of values of m rows and n columns, rank (X) is the rank of X, and if rank (X) is much smaller than m and n, we call X a low rank matrix.
Low rank decomposition: the purpose is to remove redundancy and reduce weight parameters. The method comprises the following steps: two K x 1 convolution kernels are used instead of one K x K convolution kernel.
107. And constructing a graph model.
Graph model: refers to a graph consisting of points and lines to describe the system.
108. And inputting the graph model into a pre-trained aggregation network, wherein an output aggregation result is a three-dimensional point cloud data fine-granularity analysis result.
The invention provides a three-dimensional point cloud data fine granularity analysis method, which comprises the following steps: dividing the three-dimensional point cloud data to be analyzed to obtain dividing blocks; deleting blank blocks in the dividing blocks; resampling the point set in each partition block to 512 points; extracting a feature vector for each point in each dividing block; constructing a characteristic distance matrix; performing low-rank decomposition on the characteristic distance matrix, wherein the decomposition result of each dividing block is the fine granularity analysis result of the point set in the dividing block; constructing a graph model; compared with the prior art, the method adopts a large amount of part information in common CAD data and combines the geometric feature priori of the part information learned by a neural network, so that the part information can be directly transferred to the three-dimensional point cloud data for carrying out fine-grained analysis, a special three-dimensional point cloud data set with fine-grained labeling information is not required to be used as network training data, and the problem that the traditional three-dimensional point cloud analysis algorithm needs to rely on the three-dimensional point cloud training data with semantic labels during learning is solved.
Example 2
An embodiment of the present invention provides a three-dimensional point cloud data fine-granularity analysis method, as shown in fig. 2, including:
201. and dividing the three-dimensional point cloud data to be analyzed to obtain dividing blocks.
Specifically, the three-dimensional point cloud data to be analyzed is divided by using the volume resolution of 7 multiplied by 7, 343 divided blocks are obtained.
202. And deleting blank blocks in the dividing blocks.
203. The set of points in each of the partitioned blocks is resampled to 512 points.
Because the number of points in each partitioned block is different, the set of points in each partitioned block is resampled to 512 points using the furthest point sampling method for neural network training.
Among them, the furthest point sampling (FarthestPointSampling, FPS) is a very commonly used sampling algorithm, and is widely used, because uniform sampling of samples can be ensured, and samples points are subjected to FPS sampling reclustering like PointNet++ in a 3D point cloud deep learning framework.
Principle of FPS algorithm:
(1) The input point cloud has N points, and one point P is selected from the point cloud 0 As a starting point, a set of sampling points s= { P is obtained 0 }。
(2) Calculate all points to P 0 Is used to form N-dimensional array L, from which the point corresponding to maximum value is selected as P 1 Update sampling point set s= { P 0 ，P 1 }。
(3) Calculate all points to P 1 For each point P i Distance P of it 1 If the distance of (2) is smaller than L [ i ]]Then update L [ i ]]＝d(P i ,P 1 ) Therefore, stored in array L is always the closest distance of each point to the set of sampling points S.
(4) Selecting the point corresponding to the maximum value in L as P 2 Update sampling point set s= { P 0 ，P 1 ，P 2 }；
(5) Repeating the steps 2-4 until N' target sampling points are obtained.
204. For each of the points in each of the divided blocks, a feature vector is extracted.
Extracting a feature vector for each of the points in each of the divided blocks using PointNet++ as a backbone network.
205. Constructing a characteristic distance matrix; the feature distance is the distance of the feature vector between every two points.
And constructing a characteristic distance matrix, wherein each element in the matrix is a characteristic distance between every two of 512 points in the dividing block, and the characteristic distance is a distance of a characteristic vector between every two points.
206. And carrying out low-rank decomposition on the characteristic distance matrix, wherein the decomposition result of each dividing block is the fine granularity analysis result of the point set in the dividing block.
According to the characteristic that the characteristic among the points contained in the same component has stronger similarity, but the characteristic among the points of different components has obvious dissimilarity, therefore, a low-rank loss function is preset to carry out low-rank decomposition on the characteristic distance matrix, and the decomposition result of each dividing block is the fine-granularity analysis result of the point set in the dividing block.
207. And constructing a graph model.
For the result of the low rank decomposition, n parts, called n segments, are analyzed for each partition block, and a graph model is constructed by using all segments analyzed for non-empty partition blocks. Wherein each node in the graph represents a segment, an edge in the graph represents an adjacency relationship between segments, and for each node, its axial bounding box is calculated, and if the bounding boxes between the nodes intersect, the two nodes are adjacent.
208. And inputting the graph model into a pre-trained aggregation network, wherein an output aggregation result is a three-dimensional point cloud data fine-granularity analysis result.
2081. A feature vector is extracted for each of the nodes in the graph model.
2082. Pooling the features of the points contained within each of the nodes and noting as local geometric information encoding for that node.
2083. And extracting feature vectors of each node by adopting a graph convolution layer, and recording the feature vectors as context geometric feature codes of the node.
2084. The local geometric information code and the context geometric feature code of each node are taken as comprehensive features.
2085. And constructing a node characteristic similarity matrix.
2086. And carrying out low-rank decomposition on the node characteristic similarity matrix, wherein a decomposition result is a three-dimensional point cloud data fine-granularity analysis result.
What needs to be explained here is: the detailed description of each part of this embodiment may refer to the corresponding parts of other embodiments, and will not be repeated here.
The invention provides a three-dimensional point cloud data fine granularity analysis method, which comprises the following steps: dividing the three-dimensional point cloud data to be analyzed to obtain dividing blocks; deleting blank blocks in the dividing blocks; resampling the point set in each partition block to 512 points; extracting a feature vector for each point in each dividing block; constructing a characteristic distance matrix; performing low-rank decomposition on the characteristic distance matrix, wherein the decomposition result of each dividing block is the fine granularity analysis result of the point set in the dividing block; constructing a graph model; compared with the prior art, the method adopts a large amount of part information in common CAD data and combines the geometric feature priori of the part information learned by a neural network, so that the part information can be directly transferred to the three-dimensional point cloud data for carrying out fine-grained analysis, a special three-dimensional point cloud data set with fine-grained labeling information is not required to be used as network training data, and the problem that the traditional three-dimensional point cloud analysis algorithm needs to rely on the three-dimensional point cloud training data with semantic labels during learning is solved.
Example 3
An embodiment of the present invention provides a three-dimensional point cloud data fine granularity analysis device, as shown in fig. 3, including:
the dividing unit 31 is configured to divide the three-dimensional point cloud data to be resolved, so as to obtain dividing blocks.
And a deleting unit 32, configured to delete a blank block in the divided blocks.
A resampling unit 33, configured to resample the point set in each of the partition blocks to 512 points.
An extracting unit 34 for extracting a feature vector for each of the points in each of the divided blocks.
A first construction unit 35 for constructing a feature distance matrix; the feature distance is the distance of the feature vector between every two points.
And a low-rank decomposition unit 36, configured to perform low-rank decomposition on the feature distance matrix, where a decomposition result of each of the division blocks is a fine-granularity analysis result of the point set in the division block.
A second construction unit 37 for constructing a graph model.
The input unit 38 is configured to input the graph model into a pre-trained aggregation network, and the output aggregation result is a three-dimensional point cloud data fine-granularity analysis result.
What needs to be explained here is: the detailed description of each part of this embodiment may refer to the corresponding parts of other embodiments, and will not be repeated here.
The invention provides a three-dimensional point cloud data fine granularity analysis device, which comprises: the dividing unit divides the three-dimensional point cloud data to be analyzed to obtain dividing blocks; the deleting unit deletes the blank block in the dividing block; resampling the point set in each dividing block into 512 points by a resampling unit; the extraction unit extracts a feature vector for each point in each divided block; the first construction unit constructs a characteristic distance matrix; the low-rank decomposition unit carries out low-rank decomposition on the characteristic distance matrix, and the decomposition result of each dividing block is the fine granularity analysis result of the point set in the dividing block; the second construction unit constructs a graph model; compared with the prior art, the method adopts a large amount of part information in common CAD data and combines the geometric feature priori of the part information learned by a neural network, so that the method can directly migrate to the three-dimensional point cloud data to carry out fine-grained analysis, does not need a special three-dimensional point cloud data set with fine-grained labeling information as network training data, and avoids the problem that the traditional three-dimensional point cloud analysis algorithm needs to rely on the three-dimensional point cloud training data with semantic tags during learning.
Example 4
An embodiment of the present invention provides a three-dimensional point cloud data fine granularity analysis device, as shown in fig. 4, including:
the dividing unit 41 is configured to divide the three-dimensional point cloud data to be resolved, and obtain dividing blocks.
And a deleting unit 42, configured to delete a blank block in the divided blocks.
A resampling unit 43, configured to resample the point set in each of the partition blocks to 512 points.
An extracting unit 44 for extracting a feature vector for each of the points in each of the divided blocks.
A first construction unit 45 for constructing a feature distance matrix; the feature distance is the distance of the feature vector between every two points.
The low-rank decomposition unit 46 is configured to perform low-rank decomposition on the feature distance matrix, where a decomposition result of each of the division blocks is a fine-granularity analysis result of the point set in the division block.
A second construction unit 47 for constructing the graph model.
The input unit 48 is configured to input the graph model into a pre-trained aggregation network, and the output aggregation result is a three-dimensional point cloud data fine-granularity analysis result.
Wherein the input unit 48 includes:
a first extraction module 481 is configured to extract a feature vector for each of the nodes in the graph model.
A pooling module 482 for pooling the features of the points contained within each of the nodes and recording as local geometric information encoding for that node.
A second extracting module 483 is configured to extract each node feature vector by using the graph convolution layer, and record the node feature vector as a context geometric feature code of the node.
As a module 484, the local geometry information code and the contextual geometry feature code of each of the nodes are used as their integrated features.
And a construction module 485, configured to construct a node feature similarity matrix.
The low-rank decomposition module 486 is configured to perform low-rank decomposition on the node feature similarity matrix, where the decomposition result is a three-dimensional point cloud data fine-granularity analysis result.
What needs to be explained here is: the detailed description of each part of this embodiment may refer to the corresponding parts of other embodiments, and will not be repeated here.
The invention provides a three-dimensional point cloud data fine granularity analysis device, which comprises: the dividing unit divides the three-dimensional point cloud data to be analyzed to obtain dividing blocks; the deleting unit deletes the blank block in the dividing block; resampling the point set in each dividing block into 512 points by a resampling unit; the extraction unit extracts a feature vector for each point in each divided block; the first construction unit constructs a characteristic distance matrix; the low-rank decomposition unit carries out low-rank decomposition on the characteristic distance matrix, and the decomposition result of each dividing block is the fine granularity analysis result of the point set in the dividing block; the second construction unit constructs a graph model; compared with the prior art, the method adopts a large amount of part information in common CAD data and combines the geometric feature priori of the part information learned by a neural network, so that the method can directly migrate to the three-dimensional point cloud data to carry out fine-grained analysis, does not need a special three-dimensional point cloud data set with fine-grained labeling information as network training data, and avoids the problem that the traditional three-dimensional point cloud analysis algorithm needs to rely on the three-dimensional point cloud training data with semantic tags during learning.
Example 5
The embodiment of the invention provides a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and when the computer program is executed by one or more processors, the three-dimensional point cloud data fine granularity analysis method is realized.
The computer readable storage medium may be a non-transitory readable storage medium, and when program instructions in the computer readable storage medium are executed by an electronic device, the electronic device implements a function of a management apparatus in the AI application task management method provided by the present invention. The computer-readable storage medium includes, but is not limited to, volatile memory, such as: random access memory, nonvolatile memory, for example: flash memory, hard disk (HDD), solid State Drive (SSD).
The invention provides a three-dimensional point cloud data fine-granularity analysis medium, which comprises the following components: dividing the three-dimensional point cloud data to be analyzed to obtain dividing blocks; deleting blank blocks in the dividing blocks; resampling the point set in each partition block to 512 points; extracting a feature vector for each point in each dividing block; constructing a characteristic distance matrix; performing low-rank decomposition on the characteristic distance matrix, wherein the decomposition result of each dividing block is the fine granularity analysis result of the point set in the dividing block; constructing a graph model; compared with the prior art, the method adopts a large amount of part information in common CAD data and combines the geometric feature priori of the part information learned by a neural network, so that the part information can be directly transferred to the three-dimensional point cloud data for carrying out fine-grained analysis, a special three-dimensional point cloud data set with fine-grained labeling information is not required to be used as network training data, and the problem that the traditional three-dimensional point cloud analysis algorithm needs to rely on the three-dimensional point cloud training data with semantic labels during learning is solved.
Example 6
An embodiment of the present invention provides an electronic device 110, as shown in fig. 5, including: a memory 1101 and one or more processors 1102, the memory 1101 having stored thereon a computer program that when executed by the one or more processors 1102 implements the three-dimensional point cloud data fine-grained parsing method described above.
The invention provides a three-dimensional point cloud data fine granularity analysis device, which comprises: dividing the three-dimensional point cloud data to be analyzed to obtain dividing blocks; deleting blank blocks in the dividing blocks; resampling the point set in each partition block to 512 points; extracting a feature vector for each point in each dividing block; constructing a characteristic distance matrix; performing low-rank decomposition on the characteristic distance matrix, wherein the decomposition result of each dividing block is the fine granularity analysis result of the point set in the dividing block; constructing a graph model; compared with the prior art, the method adopts a large amount of part information in common CAD data and combines the geometric feature priori of the part information learned by a neural network, so that the part information can be directly transferred to the three-dimensional point cloud data for carrying out fine-grained analysis, a special three-dimensional point cloud data set with fine-grained labeling information is not required to be used as network training data, and the problem that the traditional three-dimensional point cloud analysis algorithm needs to rely on the three-dimensional point cloud training data with semantic labels during learning is solved.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limited thereto; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the invention, and are intended to be included within the scope of the appended claims and description. In particular, the technical features mentioned in the respective embodiments may be combined in any manner as long as there is no structural conflict. The present invention is not limited to the specific embodiments disclosed herein, but encompasses all technical solutions falling within the scope of the claims.
Claims (8)
1. The three-dimensional point cloud data fine granularity analysis method is characterized by comprising the following steps of:
dividing the three-dimensional point cloud data to be analyzed to obtain dividing blocks;
deleting blank blocks in the divided blocks;
resampling the point set in each of the partitioned blocks to 512 points;
extracting a feature vector for each of the points in each of the divided blocks;
constructing a characteristic distance matrix; the feature distance is the distance of the feature vector between every two points;
performing low-rank decomposition on the characteristic distance matrix, wherein the decomposition result of each dividing block is the fine granularity analysis result of the point set in the dividing block;
constructing a graph model;
and inputting the graph model into a pre-trained aggregation network, wherein an output aggregation result is a three-dimensional point cloud data fine-granularity analysis result.
2. The method of fine-grained resolution of three-dimensional point cloud data according to claim 1, wherein the pre-training of the aggregation network comprises:
extracting a feature vector for each of the nodes in the graph model;
pooling the characteristics of the points contained in each node and recording as local geometric information codes of the node;
extracting feature vectors of each node by adopting a graph convolution layer, and recording the feature vectors as context geometric feature codes of the node;
encoding the local geometric information and the contextual geometric features of each of the nodes as integrated features thereof;
constructing a node characteristic similarity matrix;
and carrying out low-rank decomposition on the node characteristic similarity matrix, wherein a decomposition result is a three-dimensional point cloud data fine-granularity analysis result.
3. The method for fine-granularity analysis of three-dimensional point cloud data according to claim 1, wherein dividing the three-dimensional point cloud data to be analyzed to obtain divided blocks comprises:
and dividing the three-dimensional point cloud data to be analyzed by using the volume resolution of 7×7x7 to obtain 343 dividing blocks.
4. The three-dimensional point cloud data fine-granularity parsing method according to claim 1, wherein extracting a feature vector for each of the points in each of the divided blocks, comprises:
and extracting a feature vector by taking PointNet++ as a backbone network for each point in each partitioning block.
5. The utility model provides a three-dimensional point cloud data fine granularity analytical equipment which characterized in that includes:
the dividing unit is used for dividing the three-dimensional point cloud data to be analyzed to obtain dividing blocks;
a deleting unit configured to delete a blank block in the divided blocks;
a resampling unit, configured to resample a point set in each of the partition blocks to 512 points;
an extracting unit configured to extract a feature vector for each of the points in each of the divided blocks;
the first construction unit is used for constructing a characteristic distance matrix; the feature distance is the distance of the feature vector between every two points;
the low-rank decomposition unit is used for carrying out low-rank decomposition on the characteristic distance matrix, and the decomposition result of each division block is the fine granularity analysis result of the point set in the division block;
a second construction unit for constructing a graph model;
the input unit is used for inputting the graph model into a pre-trained aggregation network, and the output aggregation result is a three-dimensional point cloud data fine-granularity analysis result.
6. The three-dimensional point cloud data fine-granularity parsing apparatus according to claim 5, wherein the input unit includes:
a first extraction module for extracting a feature vector for each of the nodes in the graph model;
the pooling module is used for pooling the characteristics of the points contained in each node and recording the characteristics as local geometric information codes of the nodes;
the second extraction module is used for extracting the characteristic vectors of each node by adopting a graph convolution layer and recording the characteristic vectors as the context geometric characteristic codes of the node;
as a module for encoding said local geometric information and said contextual geometric features of each of said nodes as integrated features thereof;
the construction module is used for constructing a node characteristic similarity matrix;
and the low-rank decomposition module is used for carrying out low-rank decomposition on the node characteristic similarity matrix, and the decomposition result is a three-dimensional point cloud data fine-granularity analysis result.
7. A computer-readable storage medium, having stored thereon a computer program which, when executed by one or more processors, implements the three-dimensional point cloud data fine-grained parsing method according to any of claims 1 to 4.
8. An electronic device, comprising: a memory and one or more processors, the memory having stored thereon a computer program which, when executed by the one or more processors, implements the three-dimensional point cloud data fine-grained resolution method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310332687.8A CN116363099A (en) | 2023-03-31 | 2023-03-31 | Fine granularity analysis method, device, medium and equipment for three-dimensional point cloud data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310332687.8A CN116363099A (en) | 2023-03-31 | 2023-03-31 | Fine granularity analysis method, device, medium and equipment for three-dimensional point cloud data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116363099A true CN116363099A (en) | 2023-06-30 |
Family
ID=86941495
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310332687.8A Pending CN116363099A (en) | 2023-03-31 | 2023-03-31 | Fine granularity analysis method, device, medium and equipment for three-dimensional point cloud data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116363099A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113674403A (en) * | 2021-08-26 | 2021-11-19 | 上海交通大学 | Three-dimensional point cloud up-sampling method, system, equipment and medium |
| CN114120110A (en) * | 2021-11-22 | 2022-03-01 | 中国科学院紫金山天文台 | Multi-granularity calculation method for airborne laser point cloud classification of hybrid scene |
-
2023
- 2023-03-31 CN CN202310332687.8A patent/CN116363099A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113674403A (en) * | 2021-08-26 | 2021-11-19 | 上海交通大学 | Three-dimensional point cloud up-sampling method, system, equipment and medium |
| WO2023025030A1 (en) * | 2021-08-26 | 2023-03-02 | 上海交通大学 | Three-dimensional point cloud up-sampling method and system, device, and medium |
| CN114120110A (en) * | 2021-11-22 | 2022-03-01 | 中国科学院紫金山天文台 | Multi-granularity calculation method for airborne laser point cloud classification of hybrid scene |
Non-Patent Citations (2)
| Title |
|---|
| XIAOGANG WANG 等: "Learning Fine-Grained Segmentation of 3D Shapes without Part Labels", 2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 31 December 2021 (2021-12-31), pages 10271 - 10280 * |
| 顾军华 等: "基于点云数据的分割方法综述", 燕山大学学报, no. 02, 31 March 2020 (2020-03-31) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110738207B (en) | Character detection method for fusing character area edge information in character image | |
| US11443192B2 (en) | Machine-learning for 3D modeled object inference | |
| US20170124711A1 (en) | Universal correspondence network | |
| EP3674984A1 (en) | Set of neural networks | |
| KR101623860B1 (en) | Method for calculating similarity between document elements | |
| US11816188B2 (en) | Weakly supervised one-shot image segmentation | |
| US20230043026A1 (en) | Learning-based active surface model for medical image segmentation | |
| US20220108478A1 (en) | Processing images using self-attention based neural networks | |
| CN113239818B (en) | Table cross-modal information extraction method based on segmentation and graph convolution neural network | |
| CN113222964B (en) | Method and device for generating coronary artery central line extraction model | |
| US11288538B2 (en) | Object functionality predication methods, computer device, and storage medium | |
| WO2023277974A1 (en) | Systems and methods of hierarchical implicit representation in octree for 3d modeling | |
| EP4009277A1 (en) | Methods and systems for generating end-to-end model to estimate 3-dimensional(3-d) pose of object | |
| CN113516019B (en) | Hyperspectral image unmixing method and device and electronic equipment | |
| JP7116969B2 (en) | 2D map generation device, 2D map generation method, and 2D map generation program | |
| CN116363099A (en) | Fine granularity analysis method, device, medium and equipment for three-dimensional point cloud data | |
| Dzwinel et al. | 2-d embedding of large and high-dimensional data with minimal memory and computational time requirements | |
| CN113850811B (en) | Three-dimensional point cloud instance segmentation method based on multi-scale clustering and mask scoring | |
| CN114913330A (en) | Point cloud component segmentation method and device, electronic equipment and storage medium | |
| CN110826488B (en) | Image identification method and device for electronic document and storage equipment | |
| CN114764746A (en) | Super-resolution method and device for laser radar, electronic device and storage medium | |
| JP6942330B2 (en) | Image features and 3D shape search system using them | |
| CN115063299B (en) | Image preprocessing method and device, electronic equipment and storage medium | |
| CN116306575B (en) | Document analysis method, document analysis model training method and device and electronic equipment | |
| JP7485354B2 (en) | Statistical distance matrix calculation method, statistical distance matrix visualization method, device, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |