WO2022252274A1 - 基于PointNet网络点云分割及虚拟环境生成方法和装置 - Google Patents

基于PointNet网络点云分割及虚拟环境生成方法和装置 Download PDF

Info

Publication number
WO2022252274A1
WO2022252274A1 PCT/CN2021/099276 CN2021099276W WO2022252274A1 WO 2022252274 A1 WO2022252274 A1 WO 2022252274A1 CN 2021099276 W CN2021099276 W CN 2021099276W WO 2022252274 A1 WO2022252274 A1 WO 2022252274A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
point
segmentation
virtual environment
pointnet network
Prior art date
Application number
PCT/CN2021/099276
Other languages
English (en)
French (fr)
Inventor
姚寿文
兰泽令
王瑀
栗丽辉
孔若思
Original Assignee
北京理工大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京理工大学 filed Critical 北京理工大学
Publication of WO2022252274A1 publication Critical patent/WO2022252274A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/10—Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Definitions

  • the invention belongs to the technical field of virtual presentation, and in particular relates to a PointNet network-based point cloud segmentation and virtual environment generation method and device.
  • sensors based on lidar and depth cameras are widely used in the fields of automatic driving, remote operation and virtual reality. Since the 3D depth information can capture the depth information of the environment, the environment presentation based on the 3D point cloud data is very helpful for the operator to understand the surrounding environment (for example, the surrounding environment of the vehicle). Reconstructing the 3D environment through point clouds can improve the operator's perception of the environment, but due to the large number of point clouds, real-time data transmission and environment reconstruction become more difficult, and in the point cloud environment, the operator's perception of the environment Objects may appear difficult to discern.
  • the present invention provides a method and device for point cloud segmentation and virtual environment generation based on the PointNet network.
  • the present invention adopts the following technical solutions:
  • a method for point cloud segmentation and virtual environment generation based on PointNet network comprising:
  • Step S1 obtaining the point cloud in the data set to be processed in the virtual environment
  • Step S2 using the improved PointNet network to perform point cloud semantic segmentation on the point cloud;
  • Step S3 according to the semantically segmented point cloud, replace the object with a virtual model with physical attributes in the virtual environment, and generate a virtual object including all physical attributes.
  • the data set includes six indoor scenes in three buildings, and there are eleven room types in total, including conference room, lounge, auditorium, toilet, copy room, lounge, storage room, corridor, storage room, etc. room, office, lobby, and open space;
  • the semantic categories of the dataset are ceiling, chair, door, floor, table, wall, beam, column, window, sofa, bookshelf, board, and debris;
  • the point cloud in the dataset contains coordinates Position information XYZ and color information RGB.
  • the structure of the PointNet network is as shown in Figure 2, including: a first T-Net layer, a second T-Net layer, a plurality of perceptron MLPs and a feature fusion layer.
  • the improved PointNet network structure is divided into two parts: feature extraction and point cloud semantic segmentation, wherein,
  • the feature extraction process is: realize the global feature extraction by extracting the local features of the point cloud, specifically: obtain the d-dimensional features of n point clouds, which include spatial coordinate values, color information and point normal information; through continuous Classify and learn the local features of the point cloud, and finally obtain the global feature extraction through the maximum pooling process;
  • the point cloud semantic segmentation process is: splicing the local features and the global features, and performing dimensionality reduction processing through multi-layer MLP, and finally predicting the category of the point cloud, thereby realizing the segmentation of the point cloud.
  • realizing point cloud semantic segmentation through the improved PointNet network structure includes the following steps:
  • Step 21 aligning the position of the point cloud in the data set to be processed through the first T-Net layer
  • Step 22 Raise the dimension of the point cloud local features from 3 dimensions to 64 dimensions by MLP;
  • Step 23 performing feature alignment on the point cloud through the second T-Net layer
  • Step 24 through MLP, the dimension of the local features of the point cloud is raised from 64 dimensions to 128 dimensions, and then to 1024 dimensions;
  • Step 25 pooling the point cloud through the maximum value symmetric function to obtain the global feature of the point cloud;
  • Step 26 splicing the point cloud global features and point cloud local features through the feature fusion layer
  • Step 27 Perform dimensionality reduction processing on the spliced point cloud features through MLP to realize semantic segmentation of point clouds.
  • the process of extracting the local features of the point cloud is: after obtaining the spatial position information of the point cloud, the number n of point clouds, and the dimension d; sampling the farthest point of the point cloud, and indexing the obtained multiple center point clouds, The position information and the number of point clouds are obtained; all point clouds are grouped with the center point cloud as the center through the ball query method, and local features of the point cloud are extracted.
  • the farthest point sampling of the point cloud is specifically: randomly initialize a point as the farthest point, after obtaining its spatial position coordinates, compare the Euclidean distances between all remaining points and the current point, and obtain the coordinates of the farthest point And the distance, and store the distance value in the distance matrix, then use the obtained point as the query point, calculate the distance from each remaining point to the current point, and obtain the maximum value, repeat this step until the i most Further.
  • the method of the ball query grouping the point cloud is specifically: calculating the Euclidean distance L between the S center points determined after sampling and all points, and setting the distance threshold R, and selecting the distance from the center point is the point cloud in the spherical area of R, if L ⁇ R 2 , then keep the corresponding point M, if the value of M is less than the required point cloud number NS, then take the point at the maximum distance and add NS-M points , to meet the required number of point clouds, and then perform feature extraction.
  • the invention provides a point cloud segmentation and virtual environment generation device based on the PointNet network, comprising:
  • An acquisition module configured to acquire the point cloud of the data set to be processed in the virtual environment
  • Segmentation module for adopting PointNet network to carry out point cloud semantic segmentation to described point cloud
  • the generating module is used to replace the object with a virtual model having physical attributes in the virtual environment according to the segmented point cloud, and generate a virtual object including all physical attributes.
  • the data set includes six indoor scenes in three buildings, and there are eleven room types in total, including conference room, lounge, auditorium, toilet, copy room, lounge, storage room, corridor, storage room, etc. room, office, lobby, and open space;
  • the semantic categories of the dataset are ceiling, chair, door, floor, table, wall, beam, column, window, sofa, bookshelf, board, and debris;
  • the point cloud in the dataset contains coordinates Position information XYZ and color information RGB.
  • the present invention designs a point cloud segmentation processing algorithm, and considers the extraction of local point cloud features, so as to realize the semantic segmentation processing of the point cloud. Expand the point cloud data set, train the designed neural network model on the expanded data set, analyze the segmentation results, and transmit the point cloud segmentation results to the virtual environment, so that the target object corresponding to the point cloud can be simulated in the virtual environment. Model generation in the environment. To solve the problem that real-time data transmission and environment reconstruction become more difficult due to the large number of point clouds, and in the point cloud environment, the operator may have difficulty identifying objects in the environment.
  • Fig. 1 is a flow chart of point cloud segmentation and virtual environment generation method based on PointNet network
  • Figure 2 is a schematic structural diagram of the PointNet network
  • Fig. 3 is a schematic diagram of the T-Net conversion process
  • Figure 4 is a schematic diagram of the structural framework of the PointNet network
  • Fig. 5 is a schematic flow chart of local point cloud feature extraction
  • Fig. 6 is a schematic diagram of the farthest point sampling process
  • Fig. 7 is a schematic diagram of a ball query process
  • Fig. 8 is a schematic diagram of local point cloud feature extraction
  • Fig. 9 is a schematic diagram of generating a target object (indoor scene) model in a virtual environment.
  • the present invention provides a kind of method based on PointNet network point cloud segmentation and virtual environment generation, comprising:
  • Step S1 obtaining the point cloud in the data set to be processed in the virtual environment
  • Step S2 using the improved PointNet network to perform point cloud semantic segmentation on the point cloud;
  • Step S3 according to the semantically segmented point cloud, replace the object with a virtual model with physical attributes in the virtual environment, and generate a virtual object including all physical attributes.
  • the data set includes six different large-scale indoor scenes in three buildings, and there are eleven types of rooms in total, including conference room, lounge, auditorium, toilet, copy room, lounge, storage room, corridor , storage room, office, lobby and open space.
  • There are thirteen semantic categories in the dataset namely ceiling, chair, door, floor, table, wall, beam, column, window, sofa, bookshelf, board, and sundries.
  • the point cloud in the dataset contains coordinate position information XYZ and color information RGB.
  • step S2 the structure of the PointNet network is as shown in Figure 2, including: a first T-Net layer, a second T-Net layer, a plurality of perceptrons (Multilayer Perception, MLP) and a feature fusion layer.
  • MLP Multilayer Perception
  • the point cloud semantic segmentation through the PointNet network includes the following steps:
  • Step 21 aligning the position of the point cloud in the data set to be processed through the first T-Net layer
  • Step 22 Raise the dimension of the point cloud local features from 3 dimensions to 64 dimensions by MLP;
  • Step 23 performing feature alignment on the point cloud through the second T-Net layer
  • Step 24 through MLP, the dimension of the local features of the point cloud is raised from 64 dimensions to 128 dimensions, and then to 1024 dimensions;
  • Step 25 pooling the point cloud through the maximum value symmetric function to obtain the global feature of the point cloud;
  • Step 26 splicing the point cloud global features and point cloud local features through the feature fusion layer
  • Step 27 Perform dimensionality reduction processing on the spliced point cloud features through MLP to realize semantic segmentation of point clouds.
  • the PointNet network structure there are two T-Net layers, namely the first T-Net layer and the second T-Net layer.
  • the first T-Net layer is located after the point cloud input, and the position of the input point cloud is aligned. Make point clouds rigid body invariant.
  • the second T-Net layer is located after the first MLP, which is to perform feature alignment on the point cloud after the MLP.
  • First obtain a transformation matrix (Transformation matrix) through the T-Net layer, and perform matrix multiplication with the input point cloud matrix set to obtain the transformed alignment data, as shown in Figure 3, where d is the point cloud data dimension.
  • the input data is a point cloud space matrix
  • the output is an aligned point cloud space matrix.
  • the matrix of the first T-Net layer is a 3*3 or 6*6 matrix.
  • the input data is the high-dimensional point cloud feature matrix after MLP, and the output is the aligned point cloud feature matrix. Due to the multi-layer perceptron MLP structure, the dimension of the point cloud changes from 3 to 64. , the matrix of the first T-Net layer is a 64*64 matrix.
  • T-Net is actually a small PointNet network structure. Its internal network structure is the same as that of PointNet, except that the final output result is a transformation matrix, which is convenient for subsequent operations.
  • the role of MLP in PointNet is to upgrade the point cloud data, so as to ensure the maximum number of point cloud features in the subsequent point cloud processing and segmentation as much as possible.
  • the weights of all neurons in the MLP layer in the PointNet network are the same value. Due to the disordered nature of the point cloud, for a point cloud with n points, there are n types of input methods of the point cloud, so it is required that the model output the same result for n types of arrangements.
  • the PointNet network realizes this function through the maximum symmetric function, and the output result is the feature of the point cloud, as shown in the following formula,
  • f represents the function of extracting features
  • h represents the feature extraction layer of each layer of MLP
  • g is the maximum symmetric function.
  • the PointNet network After feature extraction, the PointNet network obtains a 1024-dimensional global feature.
  • the feature fusion layer splices and fuses the global feature with the specified 64-dimensional point feature to obtain a global feature based on local and The new features of the global features are then obtained through a multi-layer MLP network to predict the class of the point cloud.
  • the PointNet network structure is divided into feature extraction and point cloud semantic segmentation.
  • Feature extraction is the process of realizing global feature extraction by extracting local features of the point cloud.
  • the input is the d-dimensional features of n point clouds, where the d-dimension is the original feature of the point cloud, including spatial coordinate values, color information, and point Normal information; by continuously classifying and learning the local features of the point cloud, and finally extracting the global features through the maximum pooling process.
  • Point cloud semantic segmentation splices the intermediate features (local features) in feature extraction with the final global features, and performs dimensionality reduction processing through multi-layer MLP, and finally predicts the category of point clouds, so as to realize the classification of point clouds. segmentation.
  • the farthest point of the point cloud is first sampled, and the obtained multiple center point clouds are indexed , to obtain its location information new_xyz and the number of point clouds new_n, and then use the ball query method to take the obtained center point cloud as the center of the circle, group all input point clouds, and extract local point cloud features to obtain nSample feature information , when a new feature dimension appears, the features are concatenated to preserve the features of the point cloud as much as possible.
  • the input point cloud is grouped, that is, sampled.
  • the farthest point sampling has a better coverage of the entire point set. Therefore, this paper uses the farthest point sampling (FPS) sampling method to select the center point of the point cloud, and the number of center points finally obtained is the number of groups, so that the points in the point cloud can establish a certain relationship.
  • the specific method is to first randomly select a center i from the entire point set as the farthest point and obtain the coordinate value of the point, then compare the Euclidean distances from all points to the center point, obtain the point with the maximum distance, and obtain the obtained
  • the points are stored in the distance matrix, and the distance between all point clouds and the points stored in the matrix is compared.
  • the points in the matrix are updated to ensure that the points stored in the matrix is the closest value of the distance from each point to the sampling point, and then select the point with the largest distance again, and iterate again until the target point is collected, as shown in Figure 6.
  • the input point cloud randomly initialize a point as the farthest point. After obtaining its spatial position coordinates, compare the Euclidean distance between all remaining points and the current point, obtain the coordinates and distance of the farthest point, and set the distance value Store in the distance matrix, then use the obtained points as query points, calculate the distance from each remaining point to the current point, and obtain the maximum value, repeat this step until the i furthest points are sampled.
  • K nearest neighbor search is a machine learning method. According to this method, K number of adjacent points can be found to complete point cloud grouping.
  • Ball query is based on the premise of setting the upper limit of query points, according to the set radius range, at the query point Find points around. Compared with the K nearest neighbor search method, the local neighborhood of the ball query ensures a fixed range of area sizes, making the features of the local area more versatile in the entire space. Therefore, the present invention uses the ball query method to group point clouds. As shown in Figure 7.
  • For the input point cloud first calculate the Euclidean distance L between the S center points determined after sampling and all points, and set the distance threshold R, and select the point cloud in the spherical area whose distance from the center point is R, If L ⁇ R 2 , keep the corresponding point M, if the value of M is less than the required number of point clouds NS, then take the point at the maximum distance and add NS-M points to meet the required number of point clouds, Feature extraction is then performed.
  • Global feature extraction of the input point cloud data is an important step in point cloud segmentation. After grouping the point cloud through the farthest point sampling and ball query in the previous section, it is necessary to calculate the global feature of the local point cloud , and regroup the points obtained after grouping, and then learn, and then realize the global feature extraction of all input point clouds.
  • the feature extraction of the grouped local point cloud data is a key step.
  • the input in the network is all the point cloud information of each group after grouping, that is, each grouped point cloud is regarded as a whole, and the global feature is extracted for each group of point clouds.
  • This step is the same as that in the PointNet structure.
  • the whole point cloud carries out the same step of feature extraction, so in the present invention, the train of thought to the feature extraction of local point cloud learns from the train of thought in the PointNet network structure, specifically:
  • make f is a continuous function of point cloud feature distance for any ⁇ R, for Any continuous function h and a symmetric function g(x 1 ,x 2 ,x 3 ,...,x n ), such that for Have,
  • x 1 , x 2 , x 3 ,..., x n are all elements in S
  • ⁇ is a continuous function
  • MAX means to perform max pooling operation, that is, input n vectors, and output a new vector with the largest value for each element.
  • the continuous function h is fitted by a multi-layer perceptron MLP, and the ⁇ function is the activation function, as shown in Figure 8.
  • the input number is the three-dimensional coordinate information (x, y, z) of the point cloud, and then the input point cloud is converted from three-dimensional to high-dimensional through MLP, and then processed by the maximum symmetric function g and the gamma activation function to extract the local point cloud features.
  • Semantic segmentation refers to segmenting the object category corresponding to each point cloud to identify objects in the point cloud environment.
  • semantic segmentation first, use the multi-layer perceptron MLP to perform dimensionality reduction processing on the obtained point cloud global features; then, classify the point cloud through the softmax function to obtain the probability score of each point in each category; finally, Carry out label classification to realize semantic segmentation processing of point cloud.
  • step S3 since the huge data volume of the point cloud will increase the burden of data transmission and processing, after using the improved PointNet network to segment and recognize the point cloud, replace the object in the virtual environment according to the point cloud data It is a virtual model with physical attributes to better represent the surrounding environment, as shown in FIG. 9 .
  • the present invention also provides a point cloud segmentation and virtual environment generation device based on the PointNet network to realize the point cloud segmentation and virtual environment generation method based on the PointNet network, including:
  • An acquisition module configured to acquire the point cloud of the data set to be processed in the virtual environment
  • Segmentation module for adopting PointNet network to carry out point cloud semantic segmentation to described point cloud
  • the generating module is used to replace the object with a virtual model having physical attributes in the virtual environment according to the segmented point cloud, and generate a virtual object including all physical attributes.
  • the data set includes six indoor scenes in three buildings, and there are eleven room types in total, including conference room, lounge, auditorium, toilet, copy room, lounge, storage room, corridor, storage room , office, lobby, and open space;
  • the semantic categories of the dataset are ceiling, chair, door, floor, table, wall, beam, column, window, sofa, bookshelf, board, and sundries;
  • the point cloud in the dataset contains coordinate positions Information XYZ and color information RGB.
  • the present invention solves the problem that real-time data transmission and environment reconstruction become more difficult due to the large number of point clouds, and in the point cloud environment, it is difficult for operators to distinguish objects in the environment.
  • a deep learning point cloud segmentation network model is designed to realize the direct segmentation processing of the collected point cloud data, without the need to process the point cloud before processing
  • the data is transformed.
  • the point cloud is grouped and sampled, and then the local features of the point cloud are obtained, and then the global feature extraction is performed on the point cloud, and finally the semantic segmentation of the point cloud is realized.
  • the deep learning point cloud segmentation is improved. Processing precision.
  • the present invention expands the data set and enriches the object categories.
  • the object model presentation method based on the point cloud segmentation results in the virtual environment is studied, the segmented object type and the corresponding space coordinates are transmitted to the virtual environment, and the point cloud corresponding to the point cloud is transferred by calling the established model library in the virtual environment.
  • the object model is presented in the virtual environment, realizing the display mode of the object model in the virtual environment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Geometry (AREA)
  • Computer Graphics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本发明公开一种基于PointNet网络点云分割及虚拟环境生成方法和装置，包括：获取虚拟环境中待处理数据集中的点云；采用改进的PointNet网络对所述点云进行点云语义分割；根据语义分割后的点云，在虚拟环境中将物体替换为具有物理属性的虚拟模型，生成包含所有物理属性的虚拟对象。采用本发明的技术方案，以解决由于点云数量庞大，使得实时数据传输及环境重建变得比较困难，且在点云环境中，操作者对环境中的物体辨别困难的问题。

Description

基于PointNet网络点云分割及虚拟环境生成方法和装置 技术领域
本发明属于虚拟呈现技术领域，尤其涉及一种基于PointNet网络点云分割及虚拟环境生成方法和装置。
背景技术
随着传感器技术的发展，基于激光雷达、深度相机等传感器被广泛应用于自动驾驶、远程操作及虚拟现实等领域。由于三维深度信息能够捕获环境的深度信息，因此基于三维点云数据的环境呈现对于操作者理解周围环境(譬如，车辆周围环境)有很大的帮助。通过点云进行三维环境重构可以提高操作者对环境的感知能力，但由于点云数量庞大，使得实时数据传输及环境重建变得比较困难，且在点云环境中，操作者对环境中的物体可能会出现辨别困难的情况。
发明内容
本发明针对以上技术问题是，提供一种基于PointNet网络点云分割及虚拟环境生成方法和装置。
为实现上述目的，本发明采用如下的技术方案：
一种基于PointNet网络点云分割及虚拟环境生成方法，包括：
步骤S1、获取虚拟环境中待处理数据集中的点云；
步骤S2、采用改进的PointNet网络对所述点云进行点云语义分割；
步骤S3、根据语义分割后的点云，在虚拟环境中将物体替换为具有物理属性的虚拟模型，生成包含所有物理属性的虚拟对象。
作为优选，所述数据集为三栋建筑物中的六个室内场景，共有十一种房间类型，分别为会议室、休息室、礼堂、厕所、复印室、休息室、存储室、走廊、储藏室、办公室、大堂以及开放空间；数据集的语义类别分别为天花板、椅子、门、地板、桌子、墙、梁、柱、窗、沙发、书架、木板、杂物；数据集中的点云包含坐标位置信息XYZ和颜色信息RGB。
作为优选，所述PointNet网络的结构如图2所示，包括：第一 T-Net层、第二T-Net层、多个感知机MLP以及特征融合层。
作为优选，改进的PointNet网络结构分为特征提取和点云语义分割两个部分，其中，
所述特征提取过程为：通过对点云局部特征的提取实现全局特征提取，具体为：获取n个点云的d维特征，其包含空间坐标值、色彩信息以及点的法线信息；通过不断地对点云的局部特征进行分类和学习，最终经过最大池化处理获得全局特征的提取；
所述点云语义分割过程为：将所述局部特征与所述全局特征进行拼接，并通过多层的MLP进行降维处理，最终对点云的类别做出预测，从而实现点云的分割。
作为优选，通过改进的PointNet网络结构实现点云语义分割包括以下步骤：
步骤21、通过第一T-Net层对待处理数据集中点云进行位置对齐；
步骤22、通过MLP将点云局部特征的维度从3维升到64维；
步骤23、通过第二T-Net层对点云进行特征对齐；
步骤24、通过MLP将点云局部特征的维度从64维升至128维，再升为1024维；
步骤25、通过最大值对称函数对点云进行池化处理，获得点云全局特征；
步骤26、通过特征融合层将点云全局特征与点云局部特征进行拼接；
步骤27、通过MLP对拼接后的点云特征进行降维处理，实现点云的语义分割。
作为优选，提取点云局部特征的过程为：获取点云的空间位置信息及点云个数n、维度d后；对点云进行最远点采样，对得到的多个中心点云进行索引，获得其位置信息及点云个数；通过球查询方法以所述中心点云为圆心，对所有点云进行分组，并提取点云局部特征。
作为优选，对点云进行最远点采样具体为：随机初始化一个点作 为最远点，在取得其空间位置坐标后，比较剩余所有点与当前点的欧式距离，取得距离最远的点的坐标及距离，并将距离值存储至距离distance矩阵中，之后将取得的点作为查询点，计算每个剩余点到当前点的距离，并取得最大值，重复此步骤后，直到采样到i个最远点。
作为优选，所述球查询的方法对点云进行分组具体为：计算经过采样之后确定的S个中心点与所有点之间的欧氏距离L，并设定距离阈值R，选取与中心点距离为R的球形区域中的点云，若L＜R 2,则保留对应的点M，若M的值小于所需的点云数NS，则取最大距离时的点，补充NS-M个点，满足所需的点云个数，之后进行特征提取。
本发明提供一种基于PointNet网络点云分割及虚拟环境生成装置，包括：
获取模块，用于获取虚拟环境中待处理数据集的点云；
分割模块，用于采用PointNet网络对所述点云进行点云语义分割；
生成模块，用于根据分割后的点云，在虚拟环境中将物体替换为具有物理属性的虚拟模型，生成包含所有物理属性的虚拟对象。
作为优选，所述数据集为三栋建筑物中的六个室内场景，共有十一种房间类型，分别为会议室、休息室、礼堂、厕所、复印室、休息室、存储室、走廊、储藏室、办公室、大堂以及开放空间；数据集的语义类别分别为天花板、椅子、门、地板、桌子、墙、梁、柱、窗、沙发、书架、木板、杂物；数据集中的点云包含坐标位置信息XYZ和颜色信息RGB。
本发明以PointNet神经网络模型为基础，设计点云分割处理算法，考虑局部点云特征的提取，以实现对点云的语义分割处理。对点云数据集进行扩展，将设计的神经网络模型在扩展后的数据集上进行训练，针对分割结果进行分析，同时将点云分割结果传输至虚拟环境中，实现点云对应目标物体在虚拟环境中的模型生成。以解决由于点云数量庞大，使得实时数据传输及环境重建变得比较困难，且在点云 环境中，操作者对环境中的物体可能会辨别困难的问题。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为基于PointNet网络点云分割及虚拟环境生成方法流程图；
图2为PointNet网络的结构示意图；
图3为T-Net变换流程的示意图；
图4为PointNet网络的结构框架示意图；
图5为局部点云特征提取的流程示意图；
图6为最远点采样流程示意图；
图7为球查询流程示意图；
图8为局部点云特征提取示意图；
图9为虚拟环境中目标物体(室内场景)模型生成示意图。
具体实施方式
以下实施例会结合附图对本发明进行详述，在附图或说明中，相似或相同的部分使用相同的标号，并且在实际应用中，各部件的形状、厚度或高度可扩大或缩小。本发明所列举的各实施例仅用以说明本发明，并非用以限制本发明的范围。对本发明所作的任何显而易知的修饰或变更都不脱离本发明的精神与范围。
如图1所示，本发明提供一种基于PointNet网络点云分割及虚拟环境生成方法，包括：
步骤S1、获取虚拟环境中待处理数据集中的点云；
步骤S2、采用改进的PointNet网络对所述点云进行点云语义分割；
步骤S3、根据语义分割后的点云，在虚拟环境中将物体替换为具有物理属性的虚拟模型，生成包含所有物理属性的虚拟对象。
进一步，所述数据集为三栋建筑物中的六个不同的大型室内场景，共有十一种房间类型，分别为会议室、休息室、礼堂、厕所、复印室、休息室、存储室、走廊、储藏室、办公室、大堂以及开放空间。数据集的语义类别共有十三种，分别为天花板、椅子、门、地板、桌子、墙、梁、柱、窗、沙发、书架、木板、杂物。数据集中的点云包含坐标位置信息XYZ和颜色信息RGB。
进一步，步骤S2中，所述PointNet网络的结构如图2所示，包括：第一T-Net层、第二T-Net层、多个感知机(Multilayer Perception，MLP)以及特征融合层。
通过PointNet网络实现点云语义分割包括以下步骤：
步骤21、通过第一T-Net层对待处理数据集中点云进行位置对齐；
步骤22、通过MLP将点云局部特征的维度从3维升到64维；
步骤23、通过第二T-Net层对点云进行特征对齐；
步骤24、通过MLP将点云局部特征的维度从64维升至128维，再升为1024维；
步骤25、通过最大值对称函数对点云进行池化处理，获得点云全局特征；
步骤26、通过特征融合层将点云全局特征与点云局部特征进行拼接；
步骤27、通过MLP对拼接后的点云特征进行降维处理，实现点云的语义分割。
在PointNet网络结构中，共有两个T-Net层，即第一T-Net层和第二T-Net层，第一T-Net层位于点云输入之后，对输入的点云进行位置对齐，使点云具有刚体不变性。第二T-Net层位于第一个MLP之后，是对经过MLP之后的点云进行特征对齐。首先通过T-Net层获得一个变换矩阵(Transformation matrix)，并与输入的点云矩阵集进行矩阵相乘，从而获得变换后的对齐数据，如图3所示，其中，d为点云数据的维度。对于第一T-Net层而言，输入数据为点云空间 矩阵，输出为对齐后的点云空间矩阵，当输入的点云只包含空间位置信息时，d为3；若包含彩色信息时，d为6，第一T-Net层的矩阵为3*3或6*6矩阵。对于第二T-Net网络，输入数据为经过MLP的高维点云特征矩阵，输出为对齐后的点云特征矩阵，由于经过多层感知机MLP结构，因此点云的维度从3变为64，第一T-Net层的矩阵为64*64矩阵。T-Net实际上就是小型的PointNet网络结构，其内部网络结构与PointNet的结构一样，只是最终的输出结果为变换矩阵，便于后续的操作过程。
PointNet中MLP作用是对点云数据进行升维处理，以尽最大可能在之后的点云处理及分割中保证最大数量的点云特征。与传统的MLP层结构不同，在PointNet网络中MLP层中所有神经元的权重为同一值。由于点云具有无序性，对于一个具有n个点的点云，点云的输入方式共有n种排列，因此需要求模型对于n种排列方式输出的结果相同。PointNet网络通过最大值对称函数实现此功能，输出结果为点云的特征，如下式所示，
f({x 1,…x n})≈g(h(x 1),…h(x n))
其中，f表示提取特征的函数，h表示每层MLP的特征提取层，g为最大值对称函数。
在经过特征提取后，PointNet网络得到一个具有1024维的全局特征，为了获得更高的点云分割精度，特征融合层将全局特征与指定的64维点特征进行拼接融合，从而得到一个基于局部与全局特征的新特征，之后通过多层MLP网络获得对点云类别的预测。
如图4所示，PointNet网络结构分为特征提取和点云语义分割。特征提取为通过对点云局部特征的提取实现全局特征提取的过程，输入为n个点云的d维特征，其中，d维为点云的原始特征，包括空间坐标值、色彩信息以及点的法线信息；通过不断地对点云的局部特征进行分类和学习，最终经过最大池化处理获得全局特征的提取。点云语义分割将特征提取中的中间特征(局部特征)与最后得到的全局特征进行拼接，并通过多层的MLP进行降维处理，最终对点云的类别做 出预测，从而实现点云的分割。
如图5所示，在获得输入点云的空间位置信息xyz及原始信息如点云个数n、维度d后，首先对点云进行最远点采样，对得到的多个中心点云进行索引，获得其位置信息new_xyz及点云个数new_n，之后通过球查询的方法以得到的中心点云为圆心，对所有输入点云进行分组，并进行局部点云特征的提取，获得nSample个特征信息，当出现新的特征维度时，对特征进行拼接，以尽最大可能保留点云的特征。
首先对输入的点云进行分组，即采样，与随机采样相比，最远点采样对整个点集的覆盖性较好。因此本文通过最远点(farthest point sampling，FPS)采样的方式选取点云中心点，最终取得的中心点个数即为分组个数，使得点云中的点建立一定的相互联系。具体做法为首先从整个点集中先随机选取一个中心i作为最远点并获得该点的坐标值，之后比较所有点到该中心点的欧氏距离，取得距离最大值的点，并将所得的点存入distance矩阵，比较所有点云与矩阵中存储的点的距离，若某一点的距离小于矩阵中已存储的点的距离，则对矩阵中的点进行更新，以确保矩阵中存储的点是每个点到采样点的距离最近值，之后再次选取距离最大的点，再次进行迭代，直到采集完目标点为止，如图6所示。
在输入的点云中，随机初始化一个点作为最远点，在取得其空间位置坐标后，比较剩余所有点与当前点的欧式距离，取得距离最远的点的坐标及距离，并将距离值存储至距离distance矩阵中，之后将取得的点作为查询点，计算每个剩余点到当前点的距离，并取得最大值，重复此步骤后，直到采样到i个最远点。
当完成对中心点个数的选取后，需要完成每组内的点云分组选取过程，以确定每组点云所包含的点云数，主要有K近邻搜索及球查询两种方法。K近邻搜索是一种机器学习方法，根据此方法能够找到K个数量的相邻点以完成点云分组，球查询是在设置查询点数上限的前提下，根据设定的半径范围，在查询点周围对点进行查找。与K近邻搜索方法相比，球查询的局部邻域确保了固定范围的区域尺寸，使局 部区域的特征在整个空间中更具有通用性，因此本发明采用球查询的方式对点云进行分组，如图7所示。对于输入的点云，首先计算经过采样之后确定的S个中心点与所有点之间的欧氏距离L，并设定距离阈值R，选取与中心点距离为R的球形区域中的点云，若L＜R 2,则保留对应的点M，若M的值小于所需的点云数NS，则取最大距离时的点，补充NS-M个点，满足所需的点云个数，之后进行特征提取。
对输入的点云数据进行全局特征提取是进行点云分割时的一个重要步骤，在上一节中通过最远点采样和球查询完成对点云的分组之后，需要计算局部点云的全局特征，并通过对分组之后得到的点进行再分组，再学习，继而实现对所有输入点云的全局特征提取。
在此过程中，对分组之后的局部点云数据进行特征提取是关键步骤。网络中的输入为经过分组之后每一组的全部点云信息，即将每一个分组后的点云都看为一个整体，对每一组点云进行全局特征的提取，此步骤与PointNet结构中对整体点云进行特征提取的步骤一样，因此本发明中对局部点云的特征提取思路借鉴PointNet网络结构中的思路，具体为：
令
Figure PCTCN2021099276-appb-000001
f为对于任意的χ→R上的点云特征距离连续函数，对于
Figure PCTCN2021099276-appb-000002
任意一个连续函数h和一个对称函数g(x 1,x 2,x 3,…,x n)，使得对
Figure PCTCN2021099276-appb-000003
有，
|f(S)-γ(MAX{h(x i)})|＜ε
其中，x 1,x 2,x 3,…,x n是S中的全部元素，γ为连续函数，MAX表示进行max pooling操作，即输入n个向量，输出一个每个元素最大的新向量。
在PointNet网络结构中，连续函数h通过多层感知机MLP进行拟合，γ函数则为激活函数，如图8所示。输入的数字为点云的三维坐标信息(x,y,z)，之后通过MLP将输入的点云从三维转为高维，然后经过最大值对称函数g与γ激活函数的处理，从而提取局部的点云特征。
语义分割是指将每个点云对应的物体类别分割出来，以对点云环境中的物体进行辨别。在语义分割中；首先，对得到的点云全局特征 使用多层感知机MLP进行降维处理；然后，通过softmax函数对点云进行分类，获得每个点在每个类别的概率分数；最后，进行标签分类，实现点云的语义分割处理。
进一步，步骤S3中，由于点云庞大的数据量会增加数据传输和处理的负担，因此，在采用改进的PointNet网络对点云进行分割识别之后，根据点云数据，在虚拟环境中将物体替换为具有物理属性的虚拟模型，以更好的对周围环境进行表示，如图9所示。
本发明还提供一种基于PointNet网络点云分割及虚拟环境生成装置，实现上述基于PointNet网络点云分割及虚拟环境生成方法，包括：
获取模块，用于获取虚拟环境中待处理数据集的点云；
分割模块，用于采用PointNet网络对所述点云进行点云语义分割；
生成模块，用于根据分割后的点云，在虚拟环境中将物体替换为具有物理属性的虚拟模型，生成包含所有物理属性的虚拟对象。
进一步，所述数据集为三栋建筑物中的六个室内场景，共有十一种房间类型，分别为会议室、休息室、礼堂、厕所、复印室、休息室、存储室、走廊、储藏室、办公室、大堂以及开放空间；数据集的语义类别分别为天花板、椅子、门、地板、桌子、墙、梁、柱、窗、沙发、书架、木板、杂物；数据集中的点云包含坐标位置信息XYZ和颜色信息RGB。
本发明具有以下有益效果：
(1)、本发明解决由于点云数量庞大，使得实时数据传输及环境重建变得比较困难，且在点云环境中，操作者对环境中的物体辨别困难的问题。
(2)、针对点云处理的难点与挑战，基于PointNet网络结构模型，设计深度学习的点云分割网络模型，实现对采集的点云数据直接进行分割处理，而不需在处理之前对点云数据进行转换。首先对点云进行分组采样处理，之后得到点云的局部特征，再对点云进行全局特 征提取，最终实现点云的语义分割处理，通过对网络模型进行训练，提升了深度学习点云分割的处理精度。
(3)、为了包含更多的物体类别，本发明对数据集进行了扩展，丰富了物体类别。同时研究了虚拟环境下基于点云分割结果的物体模型呈现方法，将分割后的物体类型及对应空间坐标传输至虚拟环境中，通过调用虚拟环境中的已建立的模型库，将点云对应的物体模型呈现至虚拟环境中，实现了虚拟环境中物体模型的显示方式。
应当理解，虽然本说明书按照实施方式加以描述，但并非每个实施方式仅包含一个独立的技术方案，说明书的这种叙述方式仅仅是为清楚起见，本领域技术人员应当将说明书作为一个整体，各实施例中的技术方案也可以经适当组合，形成本领域技术人员可以理解的其他实施方式。

Claims (10)

  1. 一种基于PointNet网络点云分割及虚拟环境生成方法，其特征在于，包括：
    步骤S1、获取虚拟环境中待处理数据集中的点云；
    步骤S2、采用改进的PointNet网络对所述点云进行点云语义分割；
    步骤S3、根据语义分割后的点云，在虚拟环境中将物体替换为具有物理属性的虚拟模型，生成包含所有物理属性的虚拟对象。
  2. 如权利要求1所述的基于PointNet网络点云分割及虚拟环境生成方法，所述数据集为三栋建筑物中的六个室内场景，共有十一种房间类型，分别为会议室、休息室、礼堂、厕所、复印室、休息室、存储室、走廊、储藏室、办公室、大堂以及开放空间；数据集的语义类别分别为天花板、椅子、门、地板、桌子、墙、梁、柱、窗、沙发、书架、木板、杂物；数据集中的点云包含坐标位置信息XYZ和颜色信息RGB。
  3. 如权利要求1素所述的基于PointNet网络点云分割及虚拟环境生成方法，其特征在于，所述PointNet网络包括：第一T-Net层、第二T-Net层、多个感知机MLP以及特征融合层。
  4. 如权利要求1素所述的基于PointNet网络点云分割及虚拟环境生成方法，其特征在于，PointNet网络结构分为特征提取和点云语义分割两个部分，其中，
    所述特征提取过程为：通过对点云局部特征的提取实现全局特征提取，具体为：获取n个点云的d维特征，其包含空间坐标值、色彩信息以及点的法线信息；通过不断地对点云的局部特征进行分类和学习，经过最大池化处理获得全局特征的提取；
    所述点云语义分割过程为：将所述局部特征与所述全局特征进行拼接，并通过多层的MLP进行降维处理，最终对点云的类别做出预测，从而实现点云的分割。
  5. 如权利要求3素所述的基于PointNet网络点云分割及虚拟环境生成方法，其特征在于，通过改进的PointNet网络结构实现点云 语义分割包括以下步骤：
    步骤21、通过第一T-Net层对待处理数据集中点云进行位置对齐；
    步骤22、通过MLP将点云局部特征的维度从3维升到64维；
    步骤23、通过第二T-Net层对点云空间进行特征对齐；
    步骤24、通过MLP将点云局部特征的维度从64维升至128维，再升为1024维；
    步骤25、通过最大值对称函数对点云进行池化处理，获得点云全局特征；
    步骤26、通过特征融合层将点云全局特征与点云局部特征进行拼接；
    步骤27、通过MLP对拼接后的点云特征进行降维处理，实现点云的语义分割。
  6. 如权利要求4所述的基于改进的PointNet网络点云分割及虚拟环境生成方法，其特征在于，提取点云局部特征的过程为：获取点云的空间位置信息及点云个数n、维度d后；对点云进行最远点采样，对得到的多个中心点云进行索引，获得其位置信息及点云个数；通过球查询方法以所述中心点云为圆心，对所有点云进行分组，并提取点云局部特征。
  7. 如权利要求6所述的基于改进的PointNet网络点云分割及虚拟环境生成方法，其特征在于，对点云进行最远点采样具体为：随机初始化一个点作为最远点，在取得其空间位置坐标后，比较剩余所有点与当前点的欧式距离，取得距离最远的点的坐标及距离，并将距离值存储至距离distance矩阵中，之后将取得的点作为查询点，计算每个剩余点到当前点的距离，并取得最大值，重复此步骤后，直到采样到i个最远点。
  8. 如权利要求7所述的基于改进的PointNet网络点云分割及虚拟环境生成方法，其特征在于，所述球查询的方法对点云进行分组具体为：计算经过采样之后确定的S个中心点与所有点之间的欧氏距离 L，并设定距离阈值R，选取与中心点距离为R的球形区域中的点云，若L＜R 2,则保留对应的点M，若M的值小于所需的点云数NS，则取最大距离时的点，补充NS-M个点，满足所需的点云个数，之后进行特征提取。
  9. 一种基于PointNet网络点云分割及虚拟环境生成装置，其特征在于，包括：
    获取模块，用于获取虚拟环境中待处理数据集的点云；
    分割模块，用于采用改进的PointNet网络对所述点云进行点云语义分割；
    生成模块，用于根据分割后的点云，在虚拟环境中将物体替换为具有物理属性的虚拟模型，生成包含所有物理属性的虚拟对象。
  10. 如权利要求9所述的基于PointNet网络点云分割及虚拟环境生成装置，其特征在于，所述数据集为三栋建筑物中的六个室内场景，共有十一种房间类型，分别为会议室、休息室、礼堂、厕所、复印室、休息室、存储室、走廊、储藏室、办公室、大堂以及开放空间；数据集的语义类别分别为天花板、椅子、门、地板、桌子、墙、梁、柱、窗、沙发、书架、木板、杂物；数据集中的点云包含坐标位置信息XYZ和颜色信息RGB。
PCT/CN2021/099276 2021-05-31 2021-06-10 基于PointNet网络点云分割及虚拟环境生成方法和装置 WO2022252274A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110603532.4A CN113256640B (zh) 2021-05-31 2021-05-31 基于PointNet网络点云分割及虚拟环境生成方法和装置
CN202110603532.4 2021-05-31

Publications (1)

Publication Number Publication Date
WO2022252274A1 true WO2022252274A1 (zh) 2022-12-08

Family

ID=77185510

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/099276 WO2022252274A1 (zh) 2021-05-31 2021-06-10 基于PointNet网络点云分割及虚拟环境生成方法和装置

Country Status (2)

Country Link
CN (1) CN113256640B (zh)
WO (1) WO2022252274A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115937043A (zh) * 2023-01-04 2023-04-07 南京邮电大学 一种触觉辅助点云补全的方法
CN115984489A (zh) * 2023-03-21 2023-04-18 广东数字生态科技有限责任公司 一种输电线路的三维重建方法、装置及处理设备
CN116704137A (zh) * 2023-07-27 2023-09-05 山东科技大学 一种海上石油钻井平台点云深度学习逆向建模方法
CN116824188A (zh) * 2023-06-05 2023-09-29 腾晖科技建筑智能(深圳)有限公司 一种基于多神经网络集成学习的吊物类型识别方法及系统
CN116882031A (zh) * 2023-09-01 2023-10-13 临沂大学 一种基于点云的建筑模型构建方法及系统
CN117132501A (zh) * 2023-09-14 2023-11-28 武汉纺织大学 一种基于深度相机的人体点云空洞修复方法及系统
CN117315146A (zh) * 2023-09-22 2023-12-29 武汉大学 基于跨尺度多源数据的三维模型的重建方法及存储方法
CN117496161A (zh) * 2023-12-29 2024-02-02 武汉理工大学 一种点云分割方法及装置

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113810736B (zh) * 2021-08-26 2022-11-01 北京邮电大学 一种ai驱动的实时点云视频传输方法及系统
CN114511682B (zh) * 2022-04-19 2022-07-15 清华大学 一种基于激光雷达的三维场景重建方法、装置及电子设备

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110660062A (zh) * 2019-08-31 2020-01-07 南京理工大学 一种基于PointNet的点云实例分割方法及系统
CN111192270A (zh) * 2020-01-03 2020-05-22 中山大学 一种基于点全局上下文关系推理的点云语义分割方法
CN112287939A (zh) * 2020-10-29 2021-01-29 平安科技（深圳）有限公司 三维点云语义分割方法、装置、设备及介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111815776A (zh) * 2020-02-04 2020-10-23 山东水利技师学院 综合机载和车载三维激光点云及街景影像的三维建筑物精细几何重建方法
CN112712589A (zh) * 2021-01-08 2021-04-27 浙江工业大学 一种基于激光雷达和深度学习的植株3d建模的方法和系统
CN112785694A (zh) * 2021-02-05 2021-05-11 希盟泰克(重庆)实业发展有限公司 一种基于深度学习的bim三维重建方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110660062A (zh) * 2019-08-31 2020-01-07 南京理工大学 一种基于PointNet的点云实例分割方法及系统
CN111192270A (zh) * 2020-01-03 2020-05-22 中山大学 一种基于点全局上下文关系推理的点云语义分割方法
CN112287939A (zh) * 2020-10-29 2021-01-29 平安科技（深圳）有限公司 三维点云语义分割方法、装置、设备及介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
WANG X H, GAO X, ZANG K, MENG H J: "Ontology Based Semantic Understanding for 3D Indoor Scenes", JOURNAL OF PHYSICS: CONFERENCE SERIES, vol. 1607, no. 1, 1 August 2020 (2020-08-01), GB , pages 012103, XP093009839, ISSN: 1742-6588, DOI: 10.1088/1742-6596/1607/1/012103 *

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115937043A (zh) * 2023-01-04 2023-04-07 南京邮电大学 一种触觉辅助点云补全的方法
CN115984489A (zh) * 2023-03-21 2023-04-18 广东数字生态科技有限责任公司 一种输电线路的三维重建方法、装置及处理设备
CN115984489B (zh) * 2023-03-21 2023-09-19 广东数字生态科技有限责任公司 一种输电线路的三维重建方法、装置及处理设备
CN116824188B (zh) * 2023-06-05 2024-04-09 腾晖科技建筑智能(深圳)有限公司 一种基于多神经网络集成学习的吊物类型识别方法及系统
CN116824188A (zh) * 2023-06-05 2023-09-29 腾晖科技建筑智能(深圳)有限公司 一种基于多神经网络集成学习的吊物类型识别方法及系统
CN116704137B (zh) * 2023-07-27 2023-10-24 山东科技大学 一种海上石油钻井平台点云深度学习逆向建模方法
CN116704137A (zh) * 2023-07-27 2023-09-05 山东科技大学 一种海上石油钻井平台点云深度学习逆向建模方法
CN116882031A (zh) * 2023-09-01 2023-10-13 临沂大学 一种基于点云的建筑模型构建方法及系统
CN116882031B (zh) * 2023-09-01 2023-11-17 临沂大学 一种基于点云的建筑模型构建方法及系统
CN117132501A (zh) * 2023-09-14 2023-11-28 武汉纺织大学 一种基于深度相机的人体点云空洞修复方法及系统
CN117132501B (zh) * 2023-09-14 2024-02-23 武汉纺织大学 一种基于深度相机的人体点云空洞修复方法及系统
CN117315146A (zh) * 2023-09-22 2023-12-29 武汉大学 基于跨尺度多源数据的三维模型的重建方法及存储方法
CN117315146B (zh) * 2023-09-22 2024-04-05 武汉大学 基于跨尺度多源数据的三维模型的重建方法及存储方法
CN117496161A (zh) * 2023-12-29 2024-02-02 武汉理工大学 一种点云分割方法及装置
CN117496161B (zh) * 2023-12-29 2024-04-05 武汉理工大学 一种点云分割方法及装置

Also Published As

Publication number Publication date
CN113256640B (zh) 2022-05-24
CN113256640A (zh) 2021-08-13

Similar Documents

Publication Publication Date Title
WO2022252274A1 (zh) 基于PointNet网络点云分割及虚拟环境生成方法和装置
CN111489358B (zh) 一种基于深度学习的三维点云语义分割方法
WO2021017606A1 (zh) 视频处理方法、装置、电子设备及存储介质
CN110175596B (zh) 基于双流卷积神经网络的虚拟学习环境微表情识别与交互方法
CN110163110B (zh) 一种基于迁移学习和深度特征融合的行人重识别方法
US9639746B2 (en) Systems and methods of detecting body movements using globally generated multi-dimensional gesture data
CN108038420B (zh) 一种基于深度视频的人体行为识别方法
CN110008842A (zh) 一种基于深度多损失融合模型的行人重识别方法
CN112907602B (zh) 一种基于改进k-近邻算法的三维场景点云分割方法
EP4002161A1 (en) Image retrieval method and apparatus, storage medium, and device
CN107871106A (zh) 人脸检测方法和装置
JP2023501574A (ja) 仮想および拡張現実のためのシステムおよび方法
CN109063649B (zh) 基于孪生行人对齐残差网络的行人重识别方法
CN109190561B (zh) 一种视频播放中的人脸识别方法及系统
CN112132197A (zh) 模型训练、图像处理方法、装置、计算机设备和存储介质
CN110674741A (zh) 一种基于双通道特征融合的机器视觉中手势识别方法
KR20210108044A (ko) 디지털 트윈 기술을 위한 영상 분석 시스템
CN105868706A (zh) 一种基于稀疏自编码的三维模型识别方法
CN112069336B (zh) 一种基于场景草图的细粒度图像检索方法及系统
CN115995039A (zh) 用于全向地点识别的提升语义图嵌入
JP2022082493A (ja) ノイズチャネルに基づくランダム遮蔽回復の歩行者再識別方法
CN110942110A (zh) 一种三维模型的特征提取方法和装置
Yao [Retracted] Application of Higher Education Management in Colleges and Universities by Deep Learning
CN117115917A (zh) 基于多模态特征融合的教师行为识别方法、设备以及介质
CN116682178A (zh) 密集场景下的多人姿态检测方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21943633

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21943633

Country of ref document: EP

Kind code of ref document: A1