CN116258835A - Method and system for 3D reconstruction of point cloud data based on deep learning - Google Patents
Method and system for 3D reconstruction of point cloud data based on deep learning Download PDFInfo
- Publication number
- CN116258835A CN116258835A CN202310483537.7A CN202310483537A CN116258835A CN 116258835 A CN116258835 A CN 116258835A CN 202310483537 A CN202310483537 A CN 202310483537A CN 116258835 A CN116258835 A CN 116258835A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- point
- points
- vector
- cloud data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000013135 deep learning Methods 0.000 title claims abstract description 23
- 230000009466 transformation Effects 0.000 claims abstract description 16
- 238000007781 pre-processing Methods 0.000 claims abstract description 6
- 238000005457 optimization Methods 0.000 claims abstract description 5
- 238000009877 rendering Methods 0.000 claims abstract description 5
- 239000013598 vector Substances 0.000 claims description 117
- 230000006870 function Effects 0.000 claims description 32
- 230000008569 process Effects 0.000 claims description 28
- 238000012545 processing Methods 0.000 claims description 21
- 238000013528 artificial neural network Methods 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 9
- 238000013519 translation Methods 0.000 claims description 9
- 238000012546 transfer Methods 0.000 claims description 8
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000011084 recovery Methods 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 description 10
- 230000008859 change Effects 0.000 description 6
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 5
- 238000005259 measurement Methods 0.000 description 4
- 238000013527 convolutional neural network Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000007792 addition Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 101001121408 Homo sapiens L-amino-acid oxidase Proteins 0.000 description 1
- 102100026388 L-amino-acid oxidase Human genes 0.000 description 1
- 101100233916 Saccharomyces cerevisiae (strain ATCC 204508 / S288c) KAR5 gene Proteins 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- JEIPFZHSYJVQDO-UHFFFAOYSA-N iron(III) oxide Inorganic materials O=[Fe]O[Fe]=O JEIPFZHSYJVQDO-UHFFFAOYSA-N 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003647 oxidation Effects 0.000 description 1
- 238000007254 oxidation reaction Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Chemical compound O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Computational Linguistics (AREA)
- Computer Graphics (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Geometry (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
Description
技术领域Technical Field
本发明涉及信息技术领域,主要面向实物类非物质文化遗产,提出一种运用深度学习方法,通过去噪、配准和表面重构三步,将点云数据构建为三维建模的自动化方法和系统。The present invention relates to the field of information technology, and is mainly aimed at physical intangible cultural heritage. It proposes an automated method and system for constructing point cloud data into three-dimensional modeling through three steps of denoising, registration and surface reconstruction using a deep learning method.
背景技术Background Art
由于文物具有易破损性,且大部分文物都存在自然损坏现象(氧化、腐蚀、生锈等),对文物的实物保护需要投入大量的资源。近年来,随着激光扫描技术和计算机技术的不断发展,数字化文物保护得到了越来越多的应用,如文物三维建模、虚拟现实、数字文物修复等。其中,文物三维建模将文物的实物转化为可以永久存储在计算机中的数字模型,具有成本低、效率高、易展示等优势,是目前主流的数字化文化保护方式。文物三维建模需要用到点云,点云是激光扫描得到的原始数据,它是空间中点的集合,每个点除了包含点的三维坐标信息,还包含颜色、法向量、强度值、时间等信息。Since cultural relics are fragile and most of them are subject to natural damage (oxidation, corrosion, rust, etc.), the physical protection of cultural relics requires a lot of resources. In recent years, with the continuous development of laser scanning technology and computer technology, digital cultural relic protection has been increasingly applied, such as cultural relic 3D modeling, virtual reality, digital cultural relic restoration, etc. Among them, cultural relic 3D modeling converts the physical objects of cultural relics into digital models that can be permanently stored in computers. It has the advantages of low cost, high efficiency, and easy display. It is the current mainstream digital cultural protection method. Cultural relic 3D modeling requires point cloud, which is the raw data obtained by laser scanning. It is a collection of points in space. In addition to the 3D coordinate information of each point, each point also contains information such as color, normal vector, intensity value, time, etc.
在扫描点云数据建模方面,现有的点云LOD模型建立技术存在着一些不足之处,比如内存消耗大、处理时间长和稳定性差等问题,难以处理海量点云数据,且生成的三维模型效果不佳。为了解决这些问题,需要采用一种更高效、更稳定的点云处理技术。在这方面,深度学习技术提供了一种很好的解决方案。深度学习具有处理大型数据集的高效性、提取特征的自主性,非常适合点云数据处理。深度学习技术在图像、语音、自然语言处理等领域的成功应用,也为点云处理提供了一种新的思路。基于深度学习的点云处理技术能够实现点云的高效处理和建模,为非物质文化遗产的保护和传承提供了有力的支持。In terms of scanning point cloud data modeling, the existing point cloud LOD model building technology has some shortcomings, such as high memory consumption, long processing time and poor stability. It is difficult to process massive point cloud data, and the generated 3D model is not good. In order to solve these problems, a more efficient and stable point cloud processing technology is needed. In this regard, deep learning technology provides a good solution. Deep learning has the efficiency of processing large data sets and the autonomy of extracting features, which is very suitable for point cloud data processing. The successful application of deep learning technology in image, speech, natural language processing and other fields also provides a new idea for point cloud processing. Point cloud processing technology based on deep learning can realize efficient processing and modeling of point clouds, providing strong support for the protection and inheritance of intangible cultural heritage.
然而点云数据的特殊性使得传统的深度学习模型不能直接应用于点云数据的处理。点云数据是一种特殊的三维数据表示方法,它由大量的离散点组成,每个点包含三个坐标值以及其他可能的属性值,如颜色、法向量等。与传统的二维图像数据相比,点云数据的特殊性主要表现在两个方面,它具有数据无序性和仿射变化无关性。首先,一团点云数据中有很多个点数据,这些点在点云文件里无论以什么顺序出现,它们指代的信息并不改变,相反,一张图片里的点在图像里已经按照固有的顺序排列好了。其次,点云数据存在于三维空间中,具有仿射变化无关性即旋转不变性,这导致用于二维图像的传统的卷积神经网络无法用于点云数据。由于这些特殊性质,导致传统的卷积神经网络模型难以直接处理点云数据,并且也一直没有深度处理点云数据的标准流程。However, the particularity of point cloud data makes it impossible for traditional deep learning models to be directly applied to the processing of point cloud data. Point cloud data is a special three-dimensional data representation method, which consists of a large number of discrete points, each of which contains three coordinate values and other possible attribute values, such as color, normal vector, etc. Compared with traditional two-dimensional image data, the particularity of point cloud data is mainly manifested in two aspects. It has data disorder and affine change independence. First, there are many point data in a group of point cloud data. No matter in what order these points appear in the point cloud file, the information they refer to does not change. On the contrary, the points in a picture are arranged in the image according to the inherent order. Secondly, point cloud data exists in three-dimensional space and has affine change independence, that is, rotation invariance, which makes the traditional convolutional neural network used for two-dimensional images unable to be used for point cloud data. Due to these special properties, it is difficult for traditional convolutional neural network models to directly process point cloud data, and there has been no standard process for deep processing of point cloud data.
发明内容Summary of the invention
为了解决现有技术的不足之处,有效对3D点云数据进行建模,本发明提出了一套点云数据三维建模方法,包括了去噪、配准和表面重构三个步骤流程,并通过改进的神经网络模型,实现用深度学习技术对点云数据进行处理以得到最终的三维网格模型。In order to address the shortcomings of the prior art and effectively model 3D point cloud data, the present invention proposes a set of point cloud data three-dimensional modeling methods, including three steps of denoising, alignment and surface reconstruction, and through an improved neural network model, it realizes the use of deep learning technology to process point cloud data to obtain the final three-dimensional mesh model.
为了达到上述目的,本发明提供的技术方案是:基于深度学习的点云数据三维重建方法,包括如下步骤:In order to achieve the above object, the technical solution provided by the present invention is: a three-dimensional reconstruction method of point cloud data based on deep learning, comprising the following steps:
步骤1,针对被扫描物体,利用三维激光扫描仪从多个不同的角度进行多次扫描,得到对于同一个待扫描物体的多幅点云数据;Step 1: Scan the object to be scanned multiple times from multiple different angles using a three-dimensional laser scanner to obtain multiple point cloud data for the same object to be scanned;
步骤2,对采集的多幅点云数据进行预处理,包括去除离散点和将噪声点恢复到正确的位置上;Step 2: preprocess the collected multiple point cloud data, including removing discrete points and restoring noise points to the correct position;
步骤3,对于任意两幅点云数据,根据源点云和目标点云计算一个变换关系,将源点云对齐至目标点云所处的坐标系下,从而实现点云的配准,最终得到一个完整的点云图像;Step 3: For any two point cloud data, a transformation relationship is calculated based on the source point cloud and the target point cloud, and the source point cloud is aligned to the coordinate system of the target point cloud, so as to achieve point cloud registration and finally obtain a complete point cloud image;
步骤4,对非结构化的点云图像进行表面重构,得到点云数据的三角网格模型,再经过优化和渲染,得到逼近于实物的三维模型。Step 4: reconstruct the surface of the unstructured point cloud image to obtain a triangular mesh model of the point cloud data, and then optimize and render it to obtain a three-dimensional model that is close to the real object.
进一步的,步骤2中,首先进行局部划分,将每幅点云数据分成多个局部patch,每个局部patch以一个中心点为中心,包含距离中心r范围内的所有点;Furthermore, in step 2, local division is first performed to divide each point cloud data into multiple local patches, each local patch is centered on a central point and contains all points within a range r from the center;
其次,将每个局部patch输入到四元空间转移网络,学习到四维向量,通过这个四维向量,将原始点云迁移到一个新的位姿状态,然后通过一个全连接层改变位姿状态的维度,使用多层感知机网络提取逐个点云的特征,并对每个点云的特征向量做逆变换,还原到原本的位姿;在获得处理完的逐点特征向量后,对一个patch中的每个点的特征向量进行融合得到一个局部patch的特征向量;Secondly, each local patch is input into the quaternion space transfer network to learn a four-dimensional vector. Through this four-dimensional vector, the original point cloud is transferred to a new posture state, and then the dimension of the posture state is changed through a fully connected layer. The multi-layer perceptron network is used to extract the features of each point cloud, and the feature vector of each point cloud is inversely transformed to restore it to the original posture. After obtaining the processed point-by-point feature vector, the feature vector of each point in a patch is fused to obtain the feature vector of a local patch.
最后对融合的特征向量使用若干全连接层进行回归处理,获得最终结果。Finally, the fused feature vector is regressed using several fully connected layers to obtain the final result.
进一步的,通过改变最后一个全连接层的通道数量实现离散点的去除和噪声点的恢复,对离群点的判定:最后一个全连接层的通道数量设定为1,过程描述函数,,表示该点为离群点的概率,自行设置阈值,即时,判定为离群点,将其去除,所有离群点的集合记为;对噪声点判定:最后一个全连接层的通道数量设定为3,过程描述函数,,表示学习到的每个点的偏移向量,修正后的点即为,表示原始的噪声点云,表示原始的噪声点云中的一个点。Furthermore, by changing the number of channels in the last fully connected layer, the removal of discrete points and the recovery of noise points are achieved. For the determination of outliers, the number of channels in the last fully connected layer is set to 1, and the process description function , , Indicates the probability that the point is an outlier, and sets the threshold by yourself ,Right now When is an outlier, remove it, and the set of all outliers is recorded as ; For noise point determination: the number of channels in the last fully connected layer is set to 3, and the process description function , , Represents the offset vector of each learned point, and the corrected point is , represents the original noisy point cloud, Represents a point in the original noisy point cloud.
进一步的,步骤3中任意两幅点云数据进行配准的具体实现方式如下;Furthermore, the specific implementation method of registering any two point cloud data in step 3 is as follows;
首先,分别对源点云和目标点云进行特征提取,得到两个全局特征,然后,将这两个全局特征进行拼接得到两个点云的合并特征,所述拼接操作是为了让源点云和目标点云产生关联,以便在后续的处理中能够更好地进行匹配和对齐;First, feature extraction is performed on the source point cloud and the target point cloud respectively to obtain two global features. Then, the two global features are spliced to obtain the merged features of the two point clouds. The splicing operation is to associate the source point cloud and the target point cloud so that they can be better matched and aligned in subsequent processing.
接下来,将合并特征作为输入,通过五个全连接层来计算点云之间的变换关系,最后的输出是一个七维的向量,其中前三位是平移向量,后四位是单位四元数,将单位四元数转换为旋转矩阵,根据输出的平移向量和旋转矩阵将源点云进行平移和旋转操作,即将源点云对齐至目标点云所处的坐标系下;Next, the merged features are used as input, and the transformation relationship between point clouds is calculated through five fully connected layers. The final output is a seven-dimensional vector, of which the first three are translation vectors and the last four are unit quaternions. The unit quaternion is converted into a rotation matrix. The source point cloud is translated and rotated according to the output translation vector and rotation matrix, that is, the source point cloud is aligned to the coordinate system of the target point cloud.
最后,将对齐后的源点云和目标点云进行融合。Finally, the aligned source point cloud and target point cloud are fused.
进一步的,步骤4中,通过神经网络分辨出点云的表面形状点,形状外点和形状内点,而后对表面形状点上的每一帧点云进行体素表面重构,得到点云数据的三角网格模型,形状外点和形状内点不参与重构;表面重构的具体实现方式如下;Furthermore, in step 4, the surface shape points, shape external points and shape internal points of the point cloud are distinguished by a neural network, and then the voxel surface is reconstructed for each frame of the point cloud on the surface shape points to obtain a triangular mesh model of the point cloud data, and the shape external points and shape internal points do not participate in the reconstruction; the specific implementation method of the surface reconstruction is as follows;
首先初始化一个空三角形网格,然后执行以下操作:First initialize an empty triangle mesh, then do the following:
41,获取当前体素在(x, y, z)处的立方体cube,包含8个顶点;41. Get the cube of the current voxel at (x, y, z), which contains 8 vertices;
42,获取当前立方体cube的边界集合,该边界集合为所有占据概率在[0.5-tau,0.5+tau]区间内的点的集合,tau为设置的阈值;42. Get the boundary set of the current cube cube, which is the set of all points whose occupancy probability is in the interval [0.5-tau, 0.5+tau], where tau is the set threshold;
43,对于集合中的每个点,执行以下操作:43. For each point in the set, do the following:
i.选取顶点v1和v2,其中p1<0.5, p2>0.5;i. Select vertices v1 and v2, where p1<0.5, p2>0.5;
ii.通过插值函数插值(v1, v2, p1, p2)获得一个新的占据概率等于0.5的插值点vertex;ii. Use the interpolation function to interpolate (v1, v2, p1, p2) to obtain a new interpolation point vertex with an occupation probability equal to 0.5;
iii.将插值点vertex添加到三角形网格M中;iii. Add the interpolation point vertex to the triangle mesh M;
44,构建当前立方体cube的三角形面列表,该列表包含当前cube中占据概率为0.5的点和刚生成的vertex点;44. Build a triangular face list of the current cube, which contains points with an occupation probability of 0.5 in the current cube and the newly generated vertex points;
45,对于每个三角形面,将其添加到三角形网格M中,当所有三角形面都已经添加到三角形网格中,算法结束;45. For each triangular face, add it to the triangular mesh M. When all triangular faces have been added to the triangular mesh, the algorithm ends.
进一步的,神经网络学习过程如下;Further, the neural network learning process is as follows;
(1)潜在向量编码:使用点卷积方法首先为每个输入点产生一个潜在向量,编码器由点云卷积网络实现,只需要改变最后一层的通道数来控制向量的维度,其中为经过配准后的点云中的点,n表示输入点的个数;(1) Latent vector encoding: Use the point convolution method to first encode each input point Generate a latent vector , encoder Implemented by the point cloud convolutional network, only the number of channels in the last layer needs to be changed to control the vector The dimensions of is the point in the registered point cloud, and n represents the number of input points;
(2)相对潜在向量编码:给定一个任意查询点,查询点来自输入点,构造一组点集邻居,它包括离最近的k个点,设点集邻居中的点为,而后用查询点相对于的局部坐标,对每个点集邻居中的点的潜在向量进行增强,这些增强的潜在向量经过处理得到相对潜在向量,其中为级联操作;通过以上操作对每一个输入点都完成一次查询;(2) Relative latent vector encoding: Given an arbitrary query point , query point From the input point, construct a set of point set neighbors , which includes The nearest k points, let the points in the point set neighbors be , and then use the query point Relative to The local coordinates of , for each point set neighbor Points in The latent vector These enhanced latent vectors are enhanced by Processing to obtain the relative potential vector ,in It is a cascade operation; through the above operation, a query is completed for each input point;
(3)特征加权:通过学习个独立的线性层,由个相应的权重向量参数化,产生个相对权重,表示点乘操作,权重总和为1;最后通过softmax将相对权重变为权重,并平均为,查询点处的特征向量由相邻点的相对潜在向量加权求和得到:;(3) Feature weighting: through learning independent linear layers, consisting of The corresponding weight vector Parameterization, generating Relative weight , represents the dot multiplication operation, the sum of the weights is 1; finally, the relative weights are converted to Becomes weight , and the average is , query point The eigenvector at By adjacent points The relative potential vector The weighted summation yields: ;
(4)解码为占据概率:线性层将特征向量解码为占据得分,这是一个维度为2的全连接层输出,然后通过softmax转换为占用概率;(4) Decoding into occupancy probability: linear layer The feature vector Decoded as occupancy score , which is a fully connected layer output with a dimension of 2, and then converted to occupancy probability through softmax ;
(5)体素表面重构:得到各个点的占用概率后,将占用概率为0.5tau的点云当作一个体素格子进行重构。(5) Voxel surface reconstruction: obtain the occupancy probability of each point After that, the occupation probability 0.5 The point cloud of tau is reconstructed as a voxel grid.
进一步的,步骤1中采用的三维激光扫描仪包括激光脉冲扫描仪和相位测距扫描仪。Furthermore, the three-dimensional laser scanner used in step 1 includes a laser pulse scanner and a phase ranging scanner.
本发明还提供基于深度学习的点云数据三维重建系统,包括如下模块:The present invention also provides a point cloud data 3D reconstruction system based on deep learning, comprising the following modules:
点云获取模块,针对被扫描物体,利用三维激光扫描仪从多个不同的角度进行多次扫描,得到对于同一个待扫描物体的多幅点云数据;The point cloud acquisition module uses a 3D laser scanner to perform multiple scans from multiple different angles on the object to be scanned, thereby obtaining multiple point cloud data for the same object to be scanned;
预处理模块,用于对采集的多幅点云数据进行预处理,包括去除离散点和将噪声点恢复到正确的位置上;The preprocessing module is used to preprocess the collected multiple point cloud data, including removing discrete points and restoring noise points to the correct position;
配准模块,对于任意两幅点云数据,根据源点云和目标点云计算一个变换关系,将源点云对齐至目标点云所处的坐标系下,从而实现点云的配准,最终得到一个完整的点云图像;The registration module calculates a transformation relationship between the source point cloud and the target point cloud for any two point cloud data, aligns the source point cloud to the coordinate system of the target point cloud, thereby realizing point cloud registration and finally obtaining a complete point cloud image;
重构模块,用于对非结构化的点云图像进行表面重构,得到点云数据的三角网格模型,再经过优化和渲染,得到逼近于实物的三维模型。The reconstruction module is used to reconstruct the surface of unstructured point cloud images to obtain a triangular mesh model of the point cloud data, and then obtain a three-dimensional model that is close to the real object through optimization and rendering.
进一步的,预处理模块中,首先进行局部划分,将每幅点云数据分成多个局部patch,每个局部patch以一个中心点为中心,包含距离中心r范围内的所有点;Furthermore, in the preprocessing module, local division is first performed to divide each point cloud data into multiple local patches, each local patch is centered on a central point and contains all points within a range r from the center;
其次,将每个局部patch输入到四元空间转移网络,学习到四维向量,通过这个四维向量,将原始点云迁移到一个新的位姿状态,然后通过一个全连接层改变位姿状态的维度,使用多层感知机网络提取逐个点云的特征,并对每个点云的特征向量做逆变换,还原到原本的位姿;在获得处理完的逐点特征向量后,对一个patch中的每个点的特征向量进行融合得到一个局部patch的特征向量;Secondly, each local patch is input into the quaternion space transfer network to learn a four-dimensional vector. Through this four-dimensional vector, the original point cloud is transferred to a new posture state, and then the dimension of the posture state is changed through a fully connected layer. The multi-layer perceptron network is used to extract the features of each point cloud, and the feature vector of each point cloud is inversely transformed to restore it to the original posture. After obtaining the processed point-by-point feature vector, the feature vector of each point in a patch is fused to obtain the feature vector of a local patch.
最后对融合的特征向量使用若干全连接层进行回归处理,获得最终结果,具体实现方式为:Finally, the fused feature vector is regressed using several fully connected layers to obtain the final result. The specific implementation method is as follows:
通过改变最后一个全连接层的通道数量实现离散点的去除和噪声点的恢复,对离群点的判定:最后一个全连接层的通道数量设定为1,过程描述函数,,表示该点为离群点的概率,自行设置阈值,即时,判定为离群点,将其去除,所有离群点的集合记为;对噪声点判定:最后一个全连接层的通道数量设定为3,过程描述函数,,表示学习到的每个点的偏移向量,修正后的点即为,表示原始的噪声点云,表示原始的噪声点云中的一个点。By changing the number of channels in the last fully connected layer, the discrete points can be removed and the noise points can be restored. For the determination of outliers, the number of channels in the last fully connected layer is set to 1. The process description function , , Indicates the probability that the point is an outlier, and sets the threshold by yourself ,Right now When is an outlier, remove it, and the set of all outliers is recorded as ; For noise point determination: the number of channels in the last fully connected layer is set to 3, and the process description function , , Represents the offset vector of each learned point, and the corrected point is , represents the original noisy point cloud, Represents a point in the original noisy point cloud.
进一步的,重构模块中,通过神经网络分辨出点云的表面形状点,形状外点和形状内点,而后对表面形状点上的每一帧点云进行体素表面重构,得到点云数据的三角网格模型,形状外点和形状内点不参与重构;表面重构的具体实现方式如下;Furthermore, in the reconstruction module, the surface shape points, shape external points and shape internal points of the point cloud are distinguished by a neural network, and then the voxel surface is reconstructed for each frame of the point cloud on the surface shape points to obtain a triangular mesh model of the point cloud data, and the shape external points and shape internal points do not participate in the reconstruction; the specific implementation method of the surface reconstruction is as follows;
首先初始化一个空三角形网格,然后执行以下操作:First initialize an empty triangle mesh, then do the following:
41,获取当前体素在(x, y, z)处的立方体cube,包含8个顶点;41. Get the cube of the current voxel at (x, y, z), which contains 8 vertices;
42,获取当前立方体cube的边界集合,该边界集合为所有占据概率在[0.5-tau,0.5+tau]区间内的点的集合,tau为设置的阈值;42. Get the boundary set of the current cube cube, which is the set of all points whose occupancy probability is in the interval [0.5-tau, 0.5+tau], where tau is the set threshold;
43,对于集合中的每个点,执行以下操作:43. For each point in the set, do the following:
i.选取顶点v1和v2,其中p1<0.5, p2>0.5;i. Select vertices v1 and v2, where p1<0.5, p2>0.5;
ii.通过插值函数插值(v1, v2, p1, p2)获得一个新的占据概率等于0.5的插值点vertex;ii. Use the interpolation function to interpolate (v1, v2, p1, p2) to obtain a new interpolation point vertex with an occupation probability equal to 0.5;
iii.将插值点vertex添加到三角形网格M中;iii. Add the interpolation point vertex to the triangle mesh M;
44,构建当前立方体cube的三角形面列表,该列表包含当前cube中占据概率为0.5的点和刚生成的vertex点;44. Build a triangular face list of the current cube, which contains points with an occupation probability of 0.5 in the current cube and the newly generated vertex points;
45,对于每个三角形面,将其添加到三角形网格M中,当所有三角形面都已经添加到三角形网格中,算法结束;45. For each triangular face, add it to the triangular mesh M. When all triangular faces have been added to the triangular mesh, the algorithm ends.
其中神经网络学习过程如下;The neural network learning process is as follows:
(1)潜在向量编码:使用点卷积方法首先为每个输入点产生一个潜在向量,编码器由点云卷积网络实现,只需要改变最后一层的通道数来控制向量的维度,其中为经过配准后的点云中的点,n表示输入点的个数;(1) Latent vector encoding: Use the point convolution method to first encode each input point Generate a latent vector , encoder Implemented by the point cloud convolutional network, only the number of channels in the last layer needs to be changed to control the vector The dimensions of is the point in the registered point cloud, and n represents the number of input points;
(2)相对潜在向量编码:给定一个任意查询点,查询点来自输入点,构造一组点集邻居,它包括离最近的k个点,设点集邻居中的点为,而后用查询点相对于的局部坐标,对每个点集邻居中的点的潜在向量进行增强,这些增强的潜在向量经过处理得到相对潜在向量,其中为级联操作;通过以上操作对每一个输入点都完成一次查询;(2) Relative latent vector encoding: Given an arbitrary query point , query point From the input point, construct a set of point set neighbors , which includes The nearest k points, let the points in the point set neighbors be , and then use the query point Relative to The local coordinates of , for each point set neighbor Points in The latent vector These enhanced latent vectors are enhanced by Processing to obtain the relative potential vector ,in It is a cascade operation; through the above operation, a query is completed for each input point;
(3)特征加权:通过学习个独立的线性层,由个相应的权重向量参数化,产生个相对权重,表示点乘操作,权重总和为1;最后通过softmax将相对权重变为权重,并平均为,查询点处的特征向量由相邻点的相对潜在向量加权求和得到:;(3) Feature weighting: through learning independent linear layers, consisting of The corresponding weight vector Parameterization, generating Relative weight , represents the dot multiplication operation, the sum of the weights is 1; finally, the relative weights are converted to Becomes weight , and the average is , query point The eigenvector at By adjacent points The relative potential vector The weighted summation yields: ;
(4)解码为占据概率:线性层将特征向量解码为占据得分,这是一个维度为2的全连接层输出,然后通过softmax转换为占用概率;(4) Decoding into occupancy probability: linear layer The feature vector Decoded as occupancy score , which is a fully connected layer output with a dimension of 2, and then converted to occupancy probability through softmax ;
(5)体素表面重构:得到各个点的占用概率后,将占用概率为0.5tau的点云当作一个体素格子进行重构。(5) Voxel surface reconstruction: obtain the occupancy probability of each point After that, the occupation probability 0.5 The point cloud of tau is reconstructed as a voxel grid.
与现有技术相比,本发明的优点和有益效果:Compared with the prior art, the advantages and beneficial effects of the present invention are:
(1)通过局部划分的思路,在卷积神经网络模型中引入了对称函数和空间转移网络,使得深度学习模型能够适应点云数据的无序性和仿射变化无关性,使其适用于点云数据的去噪,提升了点云数据的质量,以实现更好的三维模型构建效果。(1) Through the idea of local division, symmetric functions and spatial transfer networks are introduced into the convolutional neural network model, so that the deep learning model can adapt to the disorder and affine change independence of point cloud data, making it suitable for point cloud data denoising and improving the quality of point cloud data to achieve better 3D model construction effect.
(2)通过多层感知机MLP和点的潜在向量相对编码方法对点云数据进行特征提取,使得点云数据的稀疏性对特征提取的影响减少,能更精准地提取点云的局部形状信息。(2) Feature extraction of point cloud data is performed through a multi-layer perceptron (MLP) and a point latent vector relative encoding method, which reduces the impact of the sparsity of point cloud data on feature extraction and enables more accurate extraction of the local shape information of the point cloud.
(3)引入了占据概率概念以及基于attention机制的特征加权方法进行点云表面形状确定,解决传统三维模型构建方法的效率低、可扩展性不强和生成精度不高等问题。(3) The concept of occupancy probability and a feature weighting method based on the attention mechanism are introduced to determine the surface shape of the point cloud, solving the problems of low efficiency, poor scalability and low generation accuracy of traditional 3D model construction methods.
本发明创建了一套利用深度学习技术把点云数据建模为三维模型的自动化系统,将实物类非物质文化遗产转化为可以永久存储在计算机中的数字模型,以实现非物质文化遗产的保护和传承。The present invention has created an automated system that uses deep learning technology to model point cloud data into three-dimensional models, converting physical intangible cultural heritage into digital models that can be permanently stored in computers, so as to achieve the protection and inheritance of intangible cultural heritage.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1为本发明的流程图。FIG. 1 is a flow chart of the present invention.
图2为本发明中去噪模块的架构图。FIG. 2 is a schematic diagram of the denoising module of the present invention.
图3为本发明中表面重构模块的示意图。FIG. 3 is a schematic diagram of a surface reconstruction module in the present invention.
具体实施方式DETAILED DESCRIPTION
下面结合附图和实施例对本发明的技术方案作进一步说明。The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.
如图1所示,本发明提供的基于深度学习的点云数据三维重建方法,包括如下步骤:As shown in FIG1 , the method for three-dimensional reconstruction of point cloud data based on deep learning provided by the present invention comprises the following steps:
步骤一,点云数据获取Step 1: Point cloud data acquisition
获取点云,实际上就是获取物体表面采样点的三维坐标。获取点云数据最常用的设备,根据测距原理的不同,可将三维激光扫描仪分为:Acquiring point cloud actually means obtaining the three-dimensional coordinates of sampling points on the surface of an object. The most commonly used equipment for acquiring point cloud data can be divided into three types according to different distance measurement principles:
(1)激光脉冲扫描仪,基于脉冲测距法。激光发生器发出光脉冲,打在被测物体表面并反射回来,由扫描仪上的接收器所接收。光速是已知的,由传播时间可以计算得出扫描仪和采样点之间的距离,再结合扫描仪的位置以及扫描的角度,就可以准确计算出每一个采样点的三维坐标。这种基于时间测量的脉冲测距法,需要精度很高的时钟系统。它的特点是测距范围广、使用场景灵活但精度相对较低,多用于远距离的扫描测量中。(1) Laser pulse scanner, based on pulse ranging method. The laser generator emits a light pulse, which hits the surface of the object to be measured and reflects back, and is received by the receiver on the scanner. The speed of light is known, and the distance between the scanner and the sampling point can be calculated from the propagation time. Combined with the position of the scanner and the scanning angle, the three-dimensional coordinates of each sampling point can be accurately calculated. This pulse ranging method based on time measurement requires a very accurate clock system. It is characterized by a wide ranging range and flexible usage scenarios, but relatively low accuracy, and is mostly used in long-distance scanning measurements.
(2)相位测距扫描仪,基于相位测距法。相位测距法本质上也是一种脉冲测距法,只不过它对于距离的计算是基于光波的相位差而不是基于时间的。由于光也是一种波,发射光和接收光之间存在相位差,通过这个相位差可以间接地计算光的传播时间,从而计算出距离。它有更高的扫描精度,一般用于中等距离或者近距离的扫描测量中,精度可达毫米级。(2) Phase ranging scanner, based on phase ranging method. Phase ranging method is essentially a pulse ranging method, except that its distance calculation is based on the phase difference of light waves rather than time. Since light is also a wave, there is a phase difference between the emitted light and the received light. Through this phase difference, the propagation time of light can be indirectly calculated, and thus the distance can be calculated. It has higher scanning accuracy and is generally used for medium-distance or short-distance scanning measurements, with an accuracy of up to millimeter level.
在本发明的应用场景中,扫描对象为非遗文物实体或举办非遗文化活动的场所,采用的是地面固定激光扫描系统和相位测距扫描仪结合使用的扫描方式。In the application scenario of the present invention, the scanning object is an intangible cultural heritage entity or a place for holding intangible cultural heritage activities, and a scanning method that combines a ground-fixed laser scanning system and a phase ranging scanner is adopted.
在实际的扫描场景中,由于待扫描物体体积较大(如建筑物)、存在遮挡物或是扫描仪本身的扫描角度有限,单次扫描并不能获取完整的物体表面的点云数据,需要使用多个扫描站在多个不同的角度进行多次扫描,于是就得到了对于同一个待扫描物体的多幅点云数据。In actual scanning scenarios, due to the large size of the object to be scanned (such as a building), the presence of obstructions, or the limited scanning angle of the scanner itself, a single scan cannot obtain the point cloud data of the complete object surface. Multiple scanning stations are required to perform multiple scans at multiple different angles, thus obtaining multiple point cloud data for the same object to be scanned.
步骤二,去噪模块Step 2: Denoising module
在点云采集的原始过程中,由于受到采集环境(光线、空气中的水汽、设备抖动、遮挡物)、扫描设备(设备故障、扫描精度低)和被扫描物体(物体表面有细小的凹凸)的影响,扫描得到的点云中通常存在一些噪声点,这些噪声点不仅增加了点云的数据量,还影响了后期点云处理步骤的精确度,从而影响到最终的建模效果。因此处理工作的第一步就是要去除这些噪声点。In the original process of point cloud acquisition, due to the influence of the acquisition environment (light, water vapor in the air, equipment jitter, obstructions), scanning equipment (equipment failure, low scanning accuracy) and scanned objects (small bumps on the surface of the object), there are usually some noise points in the scanned point cloud. These noise points not only increase the amount of point cloud data, but also affect the accuracy of the subsequent point cloud processing steps, thereby affecting the final modeling effect. Therefore, the first step in the processing work is to remove these noise points.
本模块的初始输入为一帧点云数据,点云数据在被输入到网络之前,首先需要进行局部划分,将整个点云分成多个局部patch,每个局部patch以一个中心点为中心,包含距离中心r范围内的所有点。这样可以将点云数据划分为多个较小的块,方便网络学习。The initial input of this module is a frame of point cloud data. Before the point cloud data is input into the network, it needs to be locally divided first, dividing the entire point cloud into multiple local patches. Each local patch is centered on a central point and contains all points within a range of r from the center. In this way, the point cloud data can be divided into multiple smaller blocks, which is convenient for network learning.
接下来,使用四元空间转移网络,学习到了四维向量,通过这个向量,将原始点云迁移到一个新的位姿状态,以保证模型对特定空间的仿射变化无关性。然后,使用多层感知机网络提取逐个点的特征,并对每个点的特征向量做类似的逆变换,到达原本的位姿。为了保证模型能够适应点云数据的无序性,使用对称函数对各点特征进行特征融合,将各点特征合并成为一个局部patch的特征向量。Next, the quaternion space transfer network is used to learn a four-dimensional vector, through which the original point cloud is transferred to a new posture state to ensure that the model is independent of affine changes in a specific space. Then, a multi-layer perceptron network is used to extract the features of each point, and a similar inverse transformation is performed on the feature vector of each point to reach the original posture. In order to ensure that the model can adapt to the disorder of the point cloud data, a symmetric function is used to fuse the features of each point and merge the features of each point into a feature vector of a local patch.
得到了融合的结果后,使用若干全连接层处理融合的局部特征向量,获得网络输出的最终结果。需要注意的是,由于点云数据中常常包含离群点和噪声点,可以通过改变最后一个回归层的通道数量,以适应所需输出的大小,同时增强模型的鲁棒性。After obtaining the fusion result, several fully connected layers are used to process the fused local feature vectors to obtain the final result of the network output. It should be noted that since point cloud data often contains outliers and noise points, the number of channels in the last regression layer can be changed to adapt to the required output size and enhance the robustness of the model.
上述过程用公式表示为:,是一帧点云数据中的每一个点。是希望通过训练得到的目标函数,是对称函数,是特征提取函数。基本思路就是对各个元素(即点云中的各个点)使用分别处理,再送入对称函数中处理,以实现排列不变性。The above process is expressed as follows: , It is each point in a frame of point cloud data. is the objective function we hope to obtain through training, is a symmetric function, is a feature extraction function. The basic idea is to use Process them separately and then send them to the symmetric function to achieve permutation invariance.
本发明所要去除的点包括两类,一种称为离群点,它是远离点云主体部分的点,可以将它直接去除,另一种称为噪声点,实际上是由正确点发生偏移而形成的,因此去除它并不是将其删除,而是在对它上施加一个逆偏移向量,将其恢复到正确的位置上。整个去噪过程: The points to be removed by the present invention include two types: one is called outlier points, which are points far away from the main part of the point cloud and can be directly removed; the other is called noise points, which are actually formed by the deviation of the correct points. Therefore, removing them does not mean deleting them, but applying an inverse deviation vector to them to restore them to the correct position. The whole denoising process:
其中表示通过去噪模块后预测的点云结果,表示去噪模块预测出的离群点的集合,表示原始的噪声点云,表示原始的噪声点云中的一个点,表示学习到的每个点的偏移向量。具体过程如下:in Represents the predicted point cloud result after passing through the denoising module, represents the set of outliers predicted by the denoising module, represents the original noisy point cloud, represents a point in the original noise point cloud, Represents the offset vector of each point learned. The specific process is as follows:
(1)局部划分。为了能够把握点云数据的局部特征,需要将一整个点云分为多个patch,给定一个点云,一个局部patch以点为中心,包含距离中心r范围内的所有点,记为,假设其中有个点。(1) Local division. In order to grasp the local features of point cloud data, it is necessary to divide the entire point cloud into multiple patches. , a local patch is composed of points As the center, it includes all points within the range of r from the center, recorded as , assuming that there is points.
(2)逐点特征提取。初始输入为一帧的全部点云数据的集合,每一个点云表示为一个维向量,其中N代表点的数量,3对应XYZ坐标。如图2所示,首先将数据按照上一步分好的patch,共有个,每次输入一个patch,通过一个预训练好的四元空间转移网络(STN1),输出一个位姿变换的四维向量(旋转)的参数:,通过这个操作,保证了模型对特定空间的仿射变化无关性,并且将原始点云迁移到一个新的有利于网络学习的位姿状态:,而后通过一个全连接层(FNN1)更改维度便于特征提取,然后使用多层感知机网络提取逐个点的特征并处理:,最后也要对逐点的特征向量做类似的逆变换,到达原本的位姿,因此还需要引入一个四元空间转移网络(STN2)和一个全连接层(FNN2):,注意这里的与都共有个,因为他们是中个点的特征向量,但是与的维度不同。(2) Point-by-point feature extraction. The initial input is a set of all point cloud data of a frame, and each point cloud is represented by a dimensional vector, where N represents the number of points and 3 corresponds to the XYZ coordinates. As shown in Figure 2, first divide the data into patches according to the previous step. Each time a patch is input, it is passed through a pre-trained four-element spatial transfer network (STN 1 ) to output the parameters of a four-dimensional vector (rotation) of pose transformation: Through this operation, the model is guaranteed to be independent of affine changes in a specific space, and the original point cloud is migrated to a new pose state that is conducive to network learning: , and then a fully connected layer (FNN 1 ) is used to change the dimension for feature extraction, and then a multi-layer perceptron network is used to extract and process the features of each point: Finally, a similar inverse transformation is performed on the point-by-point feature vector to reach the original position, so a quaternion space transfer network (STN 2 ) and a fully connected layer (FNN 2 ) are also introduced: , note that and All in common because they are middle The eigenvectors of points, but and The dimensions are different.
(3)特征融合。网络在获得处理完的逐点特征向量后,对一个patch中的每个点的特征向量求和(即通过对称函数实现),这样就合并成为一个patch的特征向量,不论输入点的顺序如何,加法的交换律保证了合并后的特征向量都是一样的,即保证模型能够适应点云数据的无序性,同时它保留了更多的信息:,与的维度相同。(3) Feature fusion. After obtaining the processed point-by-point feature vectors, the network sums the feature vectors of each point in a patch (i.e., implemented through a symmetric function), thus merging them into a feature vector of a patch. Regardless of the order of the input points, the commutative law of addition ensures that the merged feature vectors are the same, which ensures that the model can adapt to the disorder of point cloud data, while retaining more information: , and The dimensions are the same.
(4)回归模块。得到了特征融合的结果后,对融合的特征向量使用若干全连接层()处理,获得网络输出的最终结果。可以通过改变最后一个回归层(即全连接层)的通道数量,以适应所需输出的大小。对离群点判定,回归层的通道数量设定为1,表示该点为离群点的概率,过程描述函数,,可自行设置阈值,即时,判定为离群点,将其去除,所有离群点的集合记为;对噪声点判定,回归层的通道数量设定为3,表示应对该点施加的逆偏移向量,过程描述函数,,修正后的点即为。(4) Regression module. After obtaining the result of feature fusion, the fused feature vector is processed using several fully connected layers ( ) to obtain the final result of the network output. The number of channels in the last regression layer (i.e., the fully connected layer) can be changed to adapt to the required output size. For outlier determination, the number of channels in the regression layer is set to 1, indicating the probability that the point is an outlier. The process description function , , you can set the threshold yourself ,Right now When is an outlier, remove it, and the set of all outliers is recorded as ; For noise point determination, the number of channels in the regression layer is set to 3, indicating the inverse offset vector to be applied to the point, and the process description function , , the corrected point is .
步骤三,配准模块Step 3: Registration module
由于每次扫描得到的点云所处的坐标系并不相同,不能简单地将点云进行相加,而是要经过一定的旋转平移操作,将不同点云置于同一坐标系下,才可以进行不同点云的拼接,这个过程称之为点云配准。Since the coordinate systems of the point clouds obtained from each scan are not the same, the point clouds cannot be simply added together. Instead, they must go through certain rotation and translation operations to place different point clouds in the same coordinate system before they can be spliced. This process is called point cloud registration.
本模块的初始输入为上一模块处理好的经过去噪的多帧点云数据,为方便介绍这里假设为两帧点云数据,这里将它们分为源点云和目标点云,本模块的任务是将两帧点云数据对齐到同一坐标系下,使得它们能够在后续的处理中被有效地融合。为了实现这个目标,我们需要计算一个变换关系,将源点云对齐至目标点云所处的坐标系下。The initial input of this module is the denoised multi-frame point cloud data processed by the previous module. For the convenience of introduction, it is assumed that there are two frames of point cloud data. Here they are divided into source point cloud and target point cloud. The task of this module is to align the two frames of point cloud data to the same coordinate system so that they can be effectively fused in subsequent processing. To achieve this goal, we need to calculate a transformation relationship to align the source point cloud to the coordinate system of the target point cloud.
为了提取点云的全局特征,我们使用多层感知机MLP方法。首先,我们分别对源点云和目标点云进行特征提取,得到两个全局特征。然后,我们将这两个特征拼接起来得到两个点云的合并特征。这个拼接操作是为了让源点云和目标点云产生关联,以便在后续的处理中能够更好地进行匹配和对齐。In order to extract the global features of the point cloud, we use the multi-layer perceptron MLP method. First, we extract features from the source point cloud and the target point cloud respectively to obtain two global features. Then, we concatenate these two features to obtain the merged features of the two point clouds. This concatenation operation is to associate the source point cloud and the target point cloud so that they can be better matched and aligned in subsequent processing.
接下来,我们将合并特征作为输入,通过五个全连接层来计算点云之间的变换关系。最后的输出是一个七维的向量,其中前三位是平移向量,后四位是单位四元数。单位四元数可以转换为旋转矩阵,因此我们可以通过这个向量来表示源点云相对于目标点云的平移和旋转变换。将源点云进行平移和旋转操作,就可以将其对齐至目标点云所处的坐标系下。Next, we use the merged features as input and calculate the transformation relationship between point clouds through five fully connected layers. The final output is a seven-dimensional vector, where the first three are the translation vector and the last four are the unit quaternion. The unit quaternion can be converted to a rotation matrix, so we can use this vector to represent the translation and rotation transformation of the source point cloud relative to the target point cloud. By translating and rotating the source point cloud, it can be aligned to the coordinate system of the target point cloud.
最后,将对齐后的源点云和目标点云进行融合,得到一个完整的点云图像,在后续的处理中,我们可以使用这个点云图像来进行三维重建任务。Finally, the aligned source point cloud and target point cloud are fused to obtain a complete point cloud image. In subsequent processing, we can use this point cloud image to perform 3D reconstruction tasks.
本模块的功能是根据源点云和目标点云,计算一个变换关系,将源点云对齐 至目标点云所处的坐标系下,然后将两幅点云相加得到一个合并点云(对于多幅点云即两两相加后的结果再相加)。在具体的配准任务中,待配准的两幅点云必须包含重叠的部分,否则无法进行配准。所以在选取待配准点云时,通常会选择扫描站点比较靠近的两幅点云进行配准,这样两幅点云重叠部分多,易找到点之间的匹配关系。具体过程如下:The function of this module is to calculate a transformation relationship based on the source point cloud and the target point cloud, align the source point cloud to the coordinate system of the target point cloud, and then add the two point clouds to obtain a merged point cloud (for multiple point clouds, the results of adding two by two are added together). In the specific registration task, the two point clouds to be registered must contain overlapping parts, otherwise the registration cannot be performed. Therefore, when selecting the point clouds to be registered, two point clouds with relatively close scanning sites are usually selected for registration, so that the two point clouds have more overlapping parts and it is easy to find the matching relationship between the points. The specific process is as follows:
(1)特征提取。针对源点云和目标点云分别进行,使用多层感知机方法和最大池化层提取点云全局特征:,。(1) Feature extraction. This is done for the source point cloud and the target point cloud respectively, using the multi-layer perceptron method and the maximum pooling layer to extract the global features of the point cloud: , .
(2)拼接。将两个全局特征拼接得到两个点云的合并特征,作为后面五个全连接层的输入,进行拼接操作是为了让源点云和目标点云产生关联,这样回归函数才是综合考虑二者的结构信息预测出变换:+。(2) Splicing. The two global features are spliced to obtain the merged features of the two point clouds, which are used as the input of the following five fully connected layers. The purpose of the splicing operation is to associate the source point cloud with the target point cloud, so that the regression function can comprehensively consider the structural information of the two and predict the transformation: + .
(3)全连接层。将拼接得到的合并特征输入全连接层,最后的输出是一个七维的向量,该向量的前三位是平移向量,后四位是单位四元数,单位四元数可以转换为旋转矩阵。源点云经过平移和旋转与目标点云处于同一坐标系下。(3) Fully connected layer. The combined features obtained by concatenation are input into the fully connected layer. The final output is a seven-dimensional vector. The first three digits of the vector are the translation vector, and the last four digits are the unit quaternion. The unit quaternion can be converted into a rotation matrix. After translation and rotation, the source point cloud is in the same coordinate system as the target point cloud.
多幅点云配准也可以看成两两配准,先把1号和2号配准为一幅,再去和3号配准,最终多幅点云配准后的图像合成完整的点云图像。The registration of multiple point clouds can also be viewed as pairwise registration. First, register point clouds 1 and 2 into one image, and then register them with point cloud 3. Finally, the images after multiple point cloud registrations are synthesized into a complete point cloud image.
步骤四,表面重构模块Step 4: Surface reconstruction module
点云表面重构是从非结构化的点云中重构出曲面或者多边形网格,使其尽可能地拟合于原本的物体表面,经过表面重构得到的曲面模型或多边形网格模型,再经过一定的优化和渲染,就可以得到逼近于实物的三维模型。本模块的初始输入为上一模块拼接好的一帧完整点云数据,其功能是使用设计的神经网络分辨出点云的表面形状点,形状外点和形状内点,而后对表面形状点上的每一帧点云进行体素表面重构,得到点云数据的三角网格模型,形状外点和形状内点不参与重构。Point cloud surface reconstruction is to reconstruct a surface or polygonal mesh from an unstructured point cloud so that it fits the original object surface as closely as possible. The surface model or polygonal mesh model obtained by surface reconstruction, after certain optimization and rendering, can obtain a three-dimensional model close to the real object. The initial input of this module is a frame of complete point cloud data spliced by the previous module. Its function is to use the designed neural network to distinguish the surface shape points, shape external points and shape internal points of the point cloud, and then perform voxel surface reconstruction on each frame of point cloud on the surface shape points to obtain a triangular mesh model of the point cloud data. The shape external points and shape internal points do not participate in the reconstruction.
如图3所示,本发明引入了占据概率的概念,给定一组经过配准的3D点云作为输入,本模块的目标是构建一个隐式函数,指示在任意给定查询点处的占用概率,使用包含在整个空间中的点云并带有标签的数据来使用神经网络学习这个函数,标签值为0表示这个点在物体形状的外部,即形状外点,可以把它当作背景或者噪声,标签值为1表示这个点在物体形状的内部,即形状内点,不参与物体的表面重构,这样大大提升了表面重构的效率。而后,可以将形状的表面提取为占用水平为0.5的隐式函数的等值面,占用水平0.5的最可能是表面,构成的等值面即都是0.5的点构成的表面。As shown in FIG3 , the present invention introduces the occupation probability Given a set of registered 3D point clouds as input, the goal of this module is to construct an implicit function , indicating that at any given query point Occupancy probability , using the point cloud contained in the entire space and labeled data to use the neural network to learn this function. The label value of 0 means that the point is outside the shape of the object, that is, the shape outside point, which can be regarded as background or noise. The label value of 1 means that the point is inside the shape of the object, that is, the shape inside point, and does not participate in the surface reconstruction of the object. This greatly improves the efficiency of surface reconstruction. Then, the surface of the shape can be extracted as an implicit function with an occupancy level of 0.5 The isosurface of the occupancy level 0.5 is most likely the surface, and the isosurface is composed of points with a value of 0.5.
设计的神经网络学习过程如下:The designed neural network learning process is as follows:
(1)潜在向量编码:使用点卷积方法首先为每个输入点产生一个潜在向量,编码器可以由任何点云卷积网络实现,只需要改变最后一层的通道数来控制向量的维度,其中为经过配准后的点云中的点,n表示输入点的个数。(1) Latent vector encoding: Use the point convolution method to first encode each input point Generate a latent vector , encoder It can be implemented by any point cloud convolutional network, just need to change the number of channels in the last layer to control the vector The dimensions of is the point in the point cloud after registration, and n represents the number of input points.
(2)相对潜在向量编码。给定一个任意查询点,查询点来自输入点,构造一组点集邻居,它包括离最近的k个点,这里的k可根据实际情况自行设置,设点集邻居中的点为。而后用查询点相对于的局部坐标,对每个点集邻居中的点的潜在向量进行增强,这些增强的潜在向量经过处理得到相对潜在向量,其中为级联操作。其中查询点只是说明从这个输入点开始查询,最终需要对每一个输入点都完成一次查询。相对潜在向量的引入,使得查询点的特征与其邻居点集内的点产生联系,使得神经网络对于物体的局部形状特征的把握更加精确,对于复杂形状的生成,同样可以利用相对潜在向量将其结构为局部形状的组合,这样做大大提升了模型重构的可扩展性。(2) Relative latent vector encoding. Given an arbitrary query point , query point From the input point, construct a set of point set neighbors , which includes The nearest k points, where k can be set according to the actual situation, and the points in the point set neighbors are Then use the query point Relative to The local coordinates of , for each point set neighbor Points in The latent vector These enhanced latent vectors are enhanced by Processing to obtain the relative potential vector ,in It is a cascade operation. The query point only indicates that the query starts from this input point, and eventually a query needs to be completed for each input point. The introduction of relative latent vectors connects the features of the query point with the points in its neighbor point set, making the neural network more accurate in grasping the local shape features of the object. For the generation of complex shapes, relative latent vectors can also be used to structure them into a combination of local shapes, which greatly improves the scalability of model reconstruction.
(3)特征加权。相对潜在向量的范数与输入点对于确定查询点的占用的重要性相关,由此可以用它来推断相对潜变量的重要性权重。由此引入了一个注意力机制,相对潜在向量经过由权重向量参数化的线性层,产生相对权重,表示点乘操作,这些权重通过softmax函数在内进行归一化,得到各自的插值权重,这些插值权重总和为1。实际上特征加权是通过学习个独立的线性层,由个相应的权重向量参数化,产生个相对权重,最后通过softmax将变为权重,并平均为。查询点处的特征向量由相邻点的相对潜在向量加权求和得到:。(3) Feature weighting. Relative latent vector The norm and input point To determine the query point The importance of occupancy is related to the relative latent variable The importance weight of . This introduces an attention mechanism, relative to the potential vector Through the weight vector Parameterized linear layer, generating relative weights , Represents the dot product operation, these weights are passed through the softmax function in Normalize them to get their respective interpolation weights , the sum of these interpolation weights is 1. In fact, feature weighting is learned by independent linear layers, consisting of The corresponding weight vector Parameterization, generating Relative weight , and finally through softmax Becomes weight , and the average is Query point The eigenvector at By adjacent points The relative potential vector The weighted summation yields: .
(4)解码为占据概率。线性层将特征向量解码为占据得分,这是一个维度为2的全连接层输出,然后通过softmax转换为占用概率,标签值为0表示这个点在物体形状的外部,可以把它当作背景或者噪声,标签值为1表示这个点在物体形状的内部,不参与物体的表面重构。(4) Decoded into occupancy probability. Linear layer The feature vector Decoded as occupancy score , which is a fully connected layer output with a dimension of 2, and then converted to occupancy probability through softmax , a label value of 0 means that the point is outside the shape of the object and can be regarded as background or noise. A label value of 1 means that the point is inside the shape of the object and does not participate in the surface reconstruction of the object.
(5)体素表面重构。得到各个点的占用概率后,把点云当作一个体素格子进行重构,首先初始化一个空三角形网格,然后执行以下操作:(5) Voxel surface reconstruction. Obtain the occupancy probability of each point Finally, reconstruct the point cloud as a voxel grid. First, initialize an empty triangle mesh and then do the following:
1、获取当前体素在(x, y, z)处的立方体cube,包含8个顶点。1. Get the cube of the current voxel at (x, y, z), which contains 8 vertices.
2、获取当前立方体cube的边界集合,该边界为所有在占据概率在[0.5-tau,0.5+tau]区间内的点的集合。2. Get the boundary set of the current cube, which is the set of all points whose occupancy probability is in the interval [0.5-tau, 0.5+tau].
3、对于集合中的每个点,执行以下操作:3. For each point in the set, do the following:
i.选取顶点v1和v2,其中p1和p2分别为顶点v1和v2的占据概率,p1<0.5, p2>0.5。i. Select vertices v1 and v2, where p1 and p2 are the occupancy probabilities of vertices v1 and v2 respectively, p1<0.5, p2>0.5.
ii. 通过插值函数插值(v1, v2, p1, p2)获得一个新的占据概率等于0.5的插值点vertex。ii. Use the interpolation function to interpolate (v1, v2, p1, p2) to obtain a new interpolation point vertex with an occupation probability equal to 0.5.
iii. 将插值点vertex添加到三角形网格M中。iii. Add the interpolation point vertex to the triangle mesh M.
4、构建当前立方体cube的三角形面列表faces该列表包含当前cube中占据概率为0.5的点和刚生成的vertex点。4. Construct the triangle face list faces of the current cube. This list contains the points with an occupation probability of 0.5 in the current cube and the vertex points that have just been generated.
5、对于每个三角形面,将其添加到三角形网格M中,当所有三角形面都已经添加到三角形网格中,算法结束。5. For each triangular face, add it to the triangular mesh M. When all triangular faces have been added to the triangular mesh, the algorithm ends.
伪代码如下:The pseudo code is as follows:
输入:3D体素格子V(尺寸为nx, ny, nz),占用概率阈值tauInput: 3D voxel grid V (size nx, ny, nz), occupancy probability threshold tau
输出:三角形网格MOutput: triangular mesh M
M = 空网格M = Empty grid
for x in range(nx-1):for x in range(nx-1):
for y in range(ny-1):for y in range(ny-1):
for z in range(nz-1):for z in range(nz-1):
cube = 获取立方体(x, y, z, V)cube = Get Cube(x, y, z, V)
set = 获取立方体集合(cube, tau)set = Get cube set (cube, tau)
for point in set:for point in set:
if p1<0.5 and p2>0.5if p1<0.5 and p2>0.5
vertex = 插值(v1, v2, p1, p2)vertex = interpolate(v1, v2, p1, p2)
M.添加顶点(vertex)M. Adding vertices
faces = 构建三角形面列表(cube, set)faces = build triangle face list (cube, set)
for face in faces:for face in faces:
M.添加三角形(face)M. Add triangle (face)
当所有三角形面都已经添加到三角形网格中,算法结束When all triangle faces have been added to the triangle mesh, the algorithm ends.
返回M其中,获取立方体函数“获取立方体(x, y, z, V) ”根据体素坐标返回包含八个顶点的立方体,获取立方体边界函数“获取立方体集合(cube, tau) ”返回一个包含所有在占据概率[0.5-tau,0.5+tau]区间内的点的集合,“插值 (v1, v2, p1, p2) ”返回一个占据概率等于0.5的插值点,“构建三角形函数构建三角形(cube, set) ”返回一个包含所有在占据概率等于0.5点的三角形面的列表。Returns M, where the cube get function "Get Cube (x, y, z, V)" returns a cube with eight vertices according to the voxel coordinates, the cube boundary get function "Get Cube Set (cube, tau)" returns a set of all points in the occupation probability interval [0.5-tau,0.5+tau], "Interpolation (v1, v2, p1, p2)" returns an interpolation point with an occupation probability equal to 0.5, and the "Build Triangle Function Build Triangle (cube, set)" returns a list of all triangular faces with an occupation probability equal to 0.5.
本发明还提供基于深度学习的点云数据三维重建系统,包括如下模块:The present invention also provides a point cloud data 3D reconstruction system based on deep learning, comprising the following modules:
点云获取模块,针对被扫描物体,利用三维激光扫描仪从多个不同的角度进行多次扫描,得到对于同一个待扫描物体的多幅点云数据;The point cloud acquisition module uses a 3D laser scanner to perform multiple scans from multiple different angles on the object to be scanned, thereby obtaining multiple point cloud data for the same object to be scanned;
预处理模块,用于对采集的多幅点云数据进行预处理,包括去除离散点和将噪声点恢复到正确的位置上;The preprocessing module is used to preprocess the collected multiple point cloud data, including removing discrete points and restoring noise points to the correct position;
配准模块,对于任意两幅点云数据,根据源点云和目标点云计算一个变换关系,将源点云对齐至目标点云所处的坐标系下,从而实现点云的配准,最终得到一个完整的点云图像;The registration module calculates a transformation relationship between the source point cloud and the target point cloud for any two point cloud data, aligns the source point cloud to the coordinate system of the target point cloud, thereby realizing point cloud registration and finally obtaining a complete point cloud image;
重构模块,用于对非结构化的点云图像进行表面重构,得到点云数据的三角网格模型,再经过优化和渲染,得到逼近于实物的三维模型。The reconstruction module is used to reconstruct the surface of unstructured point cloud images to obtain a triangular mesh model of the point cloud data, and then obtain a three-dimensional model that is close to the real object through optimization and rendering.
各模块的具体实现方式与各步骤相应,本发明不予撰述。The specific implementation method of each module corresponds to each step and is not described in detail in the present invention.
本文中所描述的具体实施例仅仅是对本发明精神作举例说明。本发明所属技术领域的技术人员可以对所描述的具体实施例做各种各样的修改或补充或采用类似的方式替代,但并不会偏离本发明的精神或者超越所附权利要求书所定义的范围。The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310483537.7A CN116258835B (en) | 2023-05-04 | 2023-05-04 | Method and system for 3D reconstruction of point cloud data based on deep learning |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310483537.7A CN116258835B (en) | 2023-05-04 | 2023-05-04 | Method and system for 3D reconstruction of point cloud data based on deep learning |
Publications (2)
Publication Number | Publication Date |
---|---|
CN116258835A true CN116258835A (en) | 2023-06-13 |
CN116258835B CN116258835B (en) | 2023-07-28 |
Family
ID=86679625
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310483537.7A Active CN116258835B (en) | 2023-05-04 | 2023-05-04 | Method and system for 3D reconstruction of point cloud data based on deep learning |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116258835B (en) |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN116781837A (en) * | 2023-08-25 | 2023-09-19 | 中南大学 | An automated laser three-dimensional scanning system |
CN116817771A (en) * | 2023-08-28 | 2023-09-29 | 南京航空航天大学 | Aerospace part coating thickness measurement method based on cylindrical voxel characteristics |
CN117078912A (en) * | 2023-07-10 | 2023-11-17 | 中国地质大学(武汉) | Laser point cloud and image alignment method and device based on deep learning |
CN117292067A (en) * | 2023-11-24 | 2023-12-26 | 中影年年(北京)文化传媒有限公司 | Virtual 3D model method and system based on scanning real object acquisition |
CN117473105A (en) * | 2023-12-28 | 2024-01-30 | 浪潮电子信息产业股份有限公司 | Three-dimensional content generation method based on multi-mode pre-training model and related components |
CN117690095A (en) * | 2024-02-03 | 2024-03-12 | 成都坤舆空间科技有限公司 | Intelligent community management system based on three-dimensional scene |
CN117710977A (en) * | 2024-02-02 | 2024-03-15 | 西南石油大学 | Dam BIM three-dimensional model semantic quick extraction method and system based on point cloud data |
CN117953167A (en) * | 2024-03-27 | 2024-04-30 | 贵州道坦坦科技股份有限公司 | Expressway auxiliary facility modeling method and system based on point cloud data |
CN118212368A (en) * | 2024-05-22 | 2024-06-18 | 堆知(北京)科技集团有限公司 | Method and system for reconstructing three-dimensional image based on incomplete contour point cloud data |
CN118570395A (en) * | 2024-07-29 | 2024-08-30 | 深圳大学 | Building group structured reconstruction method, apparatus, device, medium and program product |
CN118967972A (en) * | 2024-07-24 | 2024-11-15 | 山东省果树研究所 | Method and system for establishing 3D model of chestnut germplasm |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2011085433A1 (en) * | 2010-01-14 | 2011-07-21 | The University Of Sydney | Acceptation/rejection of a classification of an object or terrain feature |
CN105574929A (en) * | 2015-12-15 | 2016-05-11 | 电子科技大学 | Single vegetation three-dimensional modeling method based on ground LiDAR point cloud data |
CN109118582A (en) * | 2018-09-19 | 2019-01-01 | 东北大学 | A kind of commodity three-dimensional reconstruction system and method for reconstructing |
WO2020146547A1 (en) * | 2019-01-08 | 2020-07-16 | Apple Inc. | Auxiliary information signaling and reference management for projection-based point cloud compression |
CN111833449A (en) * | 2020-06-30 | 2020-10-27 | 南京航空航天大学 | Three-dimensional reconstruction of subway tunnel interior environment and intelligent identification method of disease |
CN111899328A (en) * | 2020-07-10 | 2020-11-06 | 西北工业大学 | Point cloud three-dimensional reconstruction method based on RGB data and generation countermeasure network |
CN112967219A (en) * | 2021-03-17 | 2021-06-15 | 复旦大学附属华山医院 | Two-stage dental point cloud completion method and system based on deep learning network |
CN113358061A (en) * | 2021-05-31 | 2021-09-07 | 东南大学 | Single stripe three-dimensional point cloud measuring method for end-to-end calibration of deep learning network |
WO2021175050A1 (en) * | 2020-03-04 | 2021-09-10 | 华为技术有限公司 | Three-dimensional reconstruction method and three-dimensional reconstruction device |
WO2021179593A1 (en) * | 2020-03-10 | 2021-09-16 | 山东大学 | Deep learning-based three-dimensional pipeline reconstruction method, system, medium, and apparatus |
-
2023
- 2023-05-04 CN CN202310483537.7A patent/CN116258835B/en active Active
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2011085433A1 (en) * | 2010-01-14 | 2011-07-21 | The University Of Sydney | Acceptation/rejection of a classification of an object or terrain feature |
CN105574929A (en) * | 2015-12-15 | 2016-05-11 | 电子科技大学 | Single vegetation three-dimensional modeling method based on ground LiDAR point cloud data |
CN109118582A (en) * | 2018-09-19 | 2019-01-01 | 东北大学 | A kind of commodity three-dimensional reconstruction system and method for reconstructing |
WO2020146547A1 (en) * | 2019-01-08 | 2020-07-16 | Apple Inc. | Auxiliary information signaling and reference management for projection-based point cloud compression |
WO2021175050A1 (en) * | 2020-03-04 | 2021-09-10 | 华为技术有限公司 | Three-dimensional reconstruction method and three-dimensional reconstruction device |
WO2021179593A1 (en) * | 2020-03-10 | 2021-09-16 | 山东大学 | Deep learning-based three-dimensional pipeline reconstruction method, system, medium, and apparatus |
CN111833449A (en) * | 2020-06-30 | 2020-10-27 | 南京航空航天大学 | Three-dimensional reconstruction of subway tunnel interior environment and intelligent identification method of disease |
CN111899328A (en) * | 2020-07-10 | 2020-11-06 | 西北工业大学 | Point cloud three-dimensional reconstruction method based on RGB data and generation countermeasure network |
CN112967219A (en) * | 2021-03-17 | 2021-06-15 | 复旦大学附属华山医院 | Two-stage dental point cloud completion method and system based on deep learning network |
CN113358061A (en) * | 2021-05-31 | 2021-09-07 | 东南大学 | Single stripe three-dimensional point cloud measuring method for end-to-end calibration of deep learning network |
Non-Patent Citations (2)
Title |
---|
陈辉;马世伟;ANDREAS NUECHTER;: "基于激光扫描和SFM的非同步点云三维重构方法", 仪器仪表学报, no. 05, pages 1148 - 1157 * |
黄明伟;方莉娜;唐丽玉;王思洁;: "改进泊松算法的图像三维重建点云模型网格化", 测绘科学, no. 04, pages 23 - 28 * |
Cited By (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN117078912A (en) * | 2023-07-10 | 2023-11-17 | 中国地质大学(武汉) | Laser point cloud and image alignment method and device based on deep learning |
CN116781837B (en) * | 2023-08-25 | 2023-11-14 | 中南大学 | An automated laser three-dimensional scanning system |
CN116781837A (en) * | 2023-08-25 | 2023-09-19 | 中南大学 | An automated laser three-dimensional scanning system |
CN116817771A (en) * | 2023-08-28 | 2023-09-29 | 南京航空航天大学 | Aerospace part coating thickness measurement method based on cylindrical voxel characteristics |
CN116817771B (en) * | 2023-08-28 | 2023-11-17 | 南京航空航天大学 | Aerospace part coating thickness measurement method based on cylindrical voxel characteristics |
CN117292067A (en) * | 2023-11-24 | 2023-12-26 | 中影年年(北京)文化传媒有限公司 | Virtual 3D model method and system based on scanning real object acquisition |
CN117292067B (en) * | 2023-11-24 | 2024-03-05 | 中影年年(北京)科技有限公司 | Virtual 3D model method and system based on scanning real object acquisition |
CN117473105A (en) * | 2023-12-28 | 2024-01-30 | 浪潮电子信息产业股份有限公司 | Three-dimensional content generation method based on multi-mode pre-training model and related components |
CN117473105B (en) * | 2023-12-28 | 2024-04-05 | 浪潮电子信息产业股份有限公司 | Three-dimensional content generation method based on multi-mode pre-training model and related components |
CN117710977B (en) * | 2024-02-02 | 2024-04-26 | 西南石油大学 | Dam BIM three-dimensional model semantic quick extraction method and system based on point cloud data |
CN117710977A (en) * | 2024-02-02 | 2024-03-15 | 西南石油大学 | Dam BIM three-dimensional model semantic quick extraction method and system based on point cloud data |
CN117690095A (en) * | 2024-02-03 | 2024-03-12 | 成都坤舆空间科技有限公司 | Intelligent community management system based on three-dimensional scene |
CN117690095B (en) * | 2024-02-03 | 2024-05-03 | 成都坤舆空间科技有限公司 | Intelligent community management system based on three-dimensional scene |
CN117953167A (en) * | 2024-03-27 | 2024-04-30 | 贵州道坦坦科技股份有限公司 | Expressway auxiliary facility modeling method and system based on point cloud data |
CN117953167B (en) * | 2024-03-27 | 2024-05-28 | 贵州道坦坦科技股份有限公司 | Expressway auxiliary facility modeling method and system based on point cloud data |
CN118212368A (en) * | 2024-05-22 | 2024-06-18 | 堆知(北京)科技集团有限公司 | Method and system for reconstructing three-dimensional image based on incomplete contour point cloud data |
CN118212368B (en) * | 2024-05-22 | 2024-08-13 | 堆知(北京)科技集团有限公司 | Method and system for reconstructing three-dimensional image based on incomplete contour point cloud data |
CN118967972A (en) * | 2024-07-24 | 2024-11-15 | 山东省果树研究所 | Method and system for establishing 3D model of chestnut germplasm |
CN118570395A (en) * | 2024-07-29 | 2024-08-30 | 深圳大学 | Building group structured reconstruction method, apparatus, device, medium and program product |
Also Published As
Publication number | Publication date |
---|---|
CN116258835B (en) | 2023-07-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN116258835B (en) | Method and system for 3D reconstruction of point cloud data based on deep learning | |
Zorzi et al. | Polyworld: Polygonal building extraction with graph neural networks in satellite images | |
Meuleman et al. | Progressively optimized local radiance fields for robust view synthesis | |
Chen et al. | Point-based multi-view stereo network | |
Li et al. | Beyond grids: Learning graph representations for visual recognition | |
CN111627065B (en) | Visual positioning method and device and storage medium | |
Hu et al. | Structure‐aware 3D reconstruction for cable‐stayed bridges: A learning‐based method | |
CN108734728A (en) | A kind of extraterrestrial target three-dimensional reconstruction method based on high-resolution sequence image | |
GB2573170A (en) | 3D Skeleton reconstruction from images using matching 2D skeletons | |
CN114782634B (en) | Method and system for reconstructing clothed human body from monocular image based on surface implicit function | |
Dipanda et al. | 3-D shape reconstruction in an active stereo vision system using genetic algorithms | |
CN116682021A (en) | A Method for Extracting Building Vector Outline Data from High Resolution Remote Sensing Image | |
Chen et al. | Research on 3D reconstruction based on multiple views | |
Correia et al. | 3D reconstruction of human bodies from single-view and multi-view images: A systematic review | |
Li et al. | Unsupervised category-specific partial point set registration via joint shape completion and registration | |
Zhang et al. | ElasticMVS: Learning elastic part representation for self-supervised multi-view stereopsis | |
Wang et al. | RSBuilding: Towards General Remote Sensing Image Building Extraction and Change Detection with Foundation Model | |
CN113723294B (en) | Data processing method and device and object recognition method and device | |
Hu et al. | Learning structural graph layouts and 3D shapes for long span bridges 3D reconstruction | |
CN118314006A (en) | Semantic-combined indoor plane layout reconstruction method and device and electronic equipment | |
Niu et al. | Overview of image-based 3D reconstruction technology | |
Lei et al. | Geometric feature learning for 3D meshes | |
CN112837420B (en) | Shape complement method and system for terracotta soldiers and horses point cloud based on multi-scale and folding structure | |
CN115471651A (en) | 4D target segmentation method based on point cloud space-time memory network | |
Yang et al. | Learning to reconstruct and understand indoor scenes from sparse views |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant |