WO2022000522A1 - 基于显著性特征的模拟残缺点云的遮罩生成方法 - Google Patents

基于显著性特征的模拟残缺点云的遮罩生成方法 Download PDF

Info

Publication number
WO2022000522A1
WO2022000522A1 PCT/CN2020/100817 CN2020100817W WO2022000522A1 WO 2022000522 A1 WO2022000522 A1 WO 2022000522A1 CN 2020100817 W CN2020100817 W CN 2020100817W WO 2022000522 A1 WO2022000522 A1 WO 2022000522A1
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
point
center
points
mask
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2020/100817
Other languages
English (en)
French (fr)
Inventor
李革
张若楠
晏玮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Peking University Shenzhen Graduate School
Original Assignee
Peking University Shenzhen Graduate School
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Peking University Shenzhen Graduate School filed Critical Peking University Shenzhen Graduate School
Publication of WO2022000522A1 publication Critical patent/WO2022000522A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds

Definitions

  • the invention relates to the field of point cloud and 3D data preprocessing, and more particularly to a mask generation method for simulating residual defect clouds based on saliency features.
  • the steps of the PointCloud Saliency Map method in the prior art are: 1. Input loss function L(X, y; ⁇ ), point cloud input X, label y, model weight ⁇ , hyperparameter ⁇ , discarded points n, and cycle times T; 2. In 0-T cycles, perform steps 3-8; 3. Calculate the gradient; 4. Calculate the center coordinate point through the median value; 5. Calculate the inner product; 6. Calculate the saliency map score; 7. If If it is a high loss rate: sort from low to high, and lose the first n/T; 8. If it is a low loss rate: sort from high to low, and lose the first n/T; 9. Output the point cloud after pruning data Y.
  • Input loss function L(X, y; ⁇ ), point cloud input X, label y, model weight ⁇ , hyperparameter ⁇ , discarded points n, and cycle times T 2. In 0-T cycles, perform steps 3-8; 3. Calculate the gradient; 4. Calculate the center coordinate point through the median value; 5. Calculate
  • the existing mask generation methods all use random sampling, that is, randomly select a point from the original data, and then remove the fixed number of points within a certain range of the point, so as to simulate the point cloud with the corresponding missing rate; therefore, the disadvantages are: 1.
  • the generated incomplete mask is generally a single area, as shown in the figure below, the blue part is the incomplete part, a single area, not multiple areas, so it cannot well describe the diversity of the incomplete situation, such as missing multiple areas. Therefore, it cannot well reflect the lack of point clouds in real life. 2.
  • the diversity of input data cannot be well guaranteed, which is not conducive to the robustness of training.
  • the point cloud data obtained by using the existing point cloud acquisition device or depth perception may exist due to the limited scanning area, incomplete angle, physical environment light influence, limitations of the laser scanner itself, or the collection structure of objects is too complex and changeable, etc. , which inevitably lead to the presence of missing areas in the scan results.
  • the existing simulation method is simple and practical with random sampling mode, which cannot measure the importance of the point in the point cloud to the whole point cloud, and cannot satisfy the diversity of the residual defect clouds that appear in various real scenes. condition. Therefore, it is particularly important how to simulate diverse residual defect clouds and generate residual defect clouds according to the attributes of the midpoints of the point cloud.
  • the present invention proposes a mask generation method for simulating residual defect clouds based on saliency features, which can well simulate and represent point clouds with different missing rates in real scenes. It is flexible, convenient, and widely used to measure the salient features of each point in the cloud, and to judge whether it is proposed by scoring. It provides a basic guarantee for intelligent processing tasks such as intelligent point cloud completion and repair generation in the later stage.
  • the present invention adopts the following technical solutions to achieve the purpose:
  • a mask generation method for simulating residual defect clouds based on saliency features includes the following steps:
  • Step 1 Obtain the current number of point cloud points, the measurement method of the loss function, the discard rate and the number of cycles;
  • Step 2 Outside the number of cycles, jump out of the cycle and output the point cloud mask M;
  • Step 3 Within the number of cycles, calculate the coordinates x c of the center of the point cloud. You can use the median or average of all coordinates to roughly measure the center of the sphere, or find the coordinates of the center of the sphere through finer grid division. ;
  • Step 4 Calculate the current gradient g of each point cloud according to the loss function
  • Step 5 Calculate the rate of change ⁇ of each point in the current point cloud relative to the position of the center of the sphere;
  • Step 7 Sort all the points from high to low, delete the first [pN/T] points, that is, these points in the M mask are set to 0.
  • This method proposes a mask generation method for simulating residual defect clouds based on saliency features. By scoring each point in the point cloud to measure the degree of its influence on the shape of the entire point cloud, some fixed proportion point sets are then eliminated. , forming the required mask that can simulate residual defect clouds.
  • This method provides pre-order diversified residual defect cloud data for point cloud completion, repair and reconstruction tasks, and can generate various proportions of missing point clouds in simulated real scenes, making up for the fact that the existing random sampling cannot be based on the importance of points. degree of susceptibility to mask defects.
  • the present invention has the following advantages and effects: Diversity: it can generate missing point clouds in multiple areas, a simpler random downsampling method, more diversity, and more flexible and complex masks can be generated. .
  • Fig. 1 is the flow chart of the present invention
  • FIG. 2 is an effect diagram of the present invention.
  • Step 1 S1 obtain the current number of point cloud points, the measurement method of the loss function, the discard rate and the number of cycles;
  • the present invention measures the saliency score by estimating each point based on the saliency feature, and then inputs the loss function in the input part to further control the position of the points to be eliminated.
  • Step 2 S2 outside the number of cycles, jump out of the cycle, and output the point cloud mask M;
  • This step is used to control whether the loop is set to end, if not, continue to S3-S7, otherwise, output the final point cloud masking result.
  • the mask consists of 0 and 1, 1 means the point is retained, and 0 means the point is eliminated;
  • Step 3 S3 within the number of cycles, calculate the point cloud sphere center position coordinate x c , use the median or average value of all coordinates to roughly measure the sphere center position, or find the sphere center through finer grid division coordinate.
  • the coordinates of the spherical center of the point cloud can be measured using the spherical coordinate system, or the median or average value in the Cartesian coordinate system to roughly measure, or it can be measured through a finer mesh. Grid division to find the coordinates of the center of the sphere.
  • Step 4 S4 calculate the current gradient change g of each point cloud
  • This step is to measure the rate of change of the loss function affected by each point cloud.
  • the loss function here can be the classification cross-entropy loss function of the point cloud, or it can be a segmentation XOR loss function calculated based on the point cloud features; it can be set depending on the task requirements.
  • Step 5 S5 calculate the rate of change ⁇ of each point in the current point cloud relative to the position of the center of the sphere
  • This step provides pre-order support for calculating the saliency score in the next step.
  • the w weight value depends on the user settings, which can be simply set to all 1.
  • the w weight value range is [0, 1]. The larger the value, the more significant its gradient density is, and the smaller the value, the less affected by the gradient. . This part is conducive to the consistent value of the region and the preservation of locality. It can also be obtained or learned by learning the adjacency matrix in the graph structure.
  • Step 7 S7 Sort all the points from high to low, and delete the [pN/T] points before the sorting, that is, these points in the M mask are set to 0.
  • step 1 S1 it can be seen from step 1 S1 that the points that need to be eliminated in each cycle, the points that will have a greater impact on the shape are preferentially eliminated. On the one hand, it increases the recognition difficulty of the output mask, and on the other hand provides the diversity of missing point clouds. Express.
  • This embodiment discloses a mask generation method for simulating residual defect clouds based on salient features, which specifically includes the following steps:
  • Step S1 obtain the current number of point cloud points, the loss function measurement method, the discard rate and the number of cycles;
  • the point cloud is X
  • the current number of point clouds is N
  • the loss function is L(x,y)
  • the discard rate is p
  • the number of cycles is T.
  • the loss function can be the loss function of point cloud classification, such as the loss function of the point cloud classification model obtained through point cloud network training such as PointNet[1], PointNet++[2], DGCNN[3].
  • Step S2 outside the number of cycles, jump out of the cycle, and output the point cloud mask M;
  • M is a label containing 0 or 1, 0 means that the point cloud has been eliminated, 1 means retained; it can also be a collection of point clouds after elimination; output storage is performed according to different tasks;
  • Step S3 within the number of cycles, calculate the point cloud sphere center position coordinate x c , and use the median or average value of all the coordinates to roughly measure the sphere center position.
  • Step S4 Calculate the current gradient g of each point cloud according to the loss function, and measure the contribution of each point in the point cloud to the shape of the entire point cloud through the gradient change;
  • Step S5 Calculate the rate of change ⁇ of each point in the current point cloud relative to the position of the center of the sphere, that is, the distance of each point cloud relative to the core position is measured.
  • the physical meaning of this value is expressed as, it is relative to the center of the sphere.
  • the salient feature score expresses the degree of influence of each point on the overall shape, so it is a measure of the product of the change in position relative to the center of the sphere and the weight.
  • w can be simply set to 1, or a custom empirical value can also be used.
  • the value range is generally [0,1]. The larger the value, the more significant its gradient density is, and the smaller the value, the less affected by the gradient. , this part is conducive to the consistent value of the region and preserves locality, and can also be obtained or learned by learning the adjacency matrix in the graph structure;
  • Step S7 sort all the point scores from high to low, delete [pN/T] points before the sorting, that is, these points in the M mask are set to 0.
  • the scores of all points can be sorted by sorting algorithms, such as quick sort, heap sort, etc., and the proposed points or masks can be obtained by proposing [pN/T] points before sorting.
  • Fig. 2 is the effect drawing of the present invention, mask: namely the light-colored part in Fig. 2, the darker color is the original image, in order to simulate the situation of the realistic residual defect cloud, a mask needs to be generated to obtain the image in the pink image; the mask can be Talk about different types of disabilities. Different defect areas and shapes will have different effects on subsequent point cloud reconstruction and other processing.
  • the four pictures in the first row are: random seed sampling, the missing part of random seed is a continuous area, because it is a point within a certain radius of a seed area, so you can see that the blue part is an area, not a different area ;
  • the missing area can be several pieces, which increases the difficulty of repairing to a certain extent.
  • this embodiment discloses a mask generation method for simulating residual defect clouds based on saliency features.
  • Most of the existing simulation methods are only based on random sampling, which cannot well represent their contribution to the difficulty of the task, and cannot meet the missing situation in the real point cloud scene.
  • There are few datasets, which cannot provide a large number of real cases for further in-depth learning and exploration. Therefore, the mask generation method for simulating residual defect clouds based on saliency features proposed by the present invention measures its contribution to the shape of the entire point cloud by scoring the midpoint of each point cloud by introducing a scoring mode of salient features. degree, to eliminate or retain according to the proportion, and can simulate the incomplete situation under different deletion proportions. It provides data sources and diverse simulation masks for subsequent completion and reconstruction tasks.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Geometry (AREA)
  • Multimedia (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

一种基于显著性特征的模拟残缺点云的遮罩生成方法，包括步骤一、获得当前点云点个数，损失函数衡量方式，以及丢弃率及循环次数；步骤二、在循环次数外，则跳出循环，输出点云遮罩M；步骤三、循环次数内，计算球心坐标x c（S3），可使用所有坐标的中值或平均值来进行粗略衡量其球心位置；步骤四、根据损失函数，计算当前点云的梯度g（S4）；步骤五、计算当前点云中每个点相对球心的变化率δ（S5）；步骤六、计算当前点云中每个点得分s＝-w*δ（S6）；步骤七、将得分从低到高进行排序，从点云中删除排序前[pN/T]个点（S7）。所述方法能够生成多区域缺失的点云情况，产生更加灵活多变复杂的遮罩。

Description

基于显著性特征的模拟残缺点云的遮罩生成方法 技术领域
本发明涉及点云及3D数据预处理领域，更具体地涉及一种基于显著性特征的模拟残缺点云的遮罩生成方法。
背景技术
与本发明的现有技术请参见参考附录[1]、[2]、[3]和[4]。
现有技术的PointCloud Saliency Map方法步骤是：1、输入损失函数L(X,y；θ),点云输入X，标签y，模型的权重θ，超参数α，丢弃点数n，循环次数T；2、在0-T个循环内，执行步骤3-8；3、计算梯度；4、通过中位值来计算中心坐标点；5、计算内积；6、计算显著性地图得分；7、如果是高丢失率：则从低到高排序，丢失前n/T个；8、如果是低丢失率：则从高到低排序，丢失前n/T个；9、输出剪枝后的点云数据Y。
参考算法图
Figure PCTCN2020100817-appb-000001
现有类似技术，如PointCloud Saliency Map方法，其主要用于点云的特征 地图构建，请参见参考附录[4]。
现有生成遮罩方式均采用随机采样方式，也即：从原始数据中随机一个点，然后再去除该点一定范围内的固定点数，以此来模拟相应缺失率的点云；故缺点为：1.生成残缺遮罩一般为单个区域，如下图所示，蓝色部分为残缺部分，单个区域，而非多个区域，故不能够很好的描述残缺情况的多样性，例如多区域缺失的情况，因而不能很好的反应现实生活中点云的缺失情况。2.对于后续点云补全等生成任务的训练而言，不能够很好的保障输入数据的多样性，因而不利于训练鲁棒性。
使用现有点云采集装置或深度感知获取的点云数据，由于可能存在扫描区域有限、角度不完全、物理环境光线影响、激光扫描仪本身的局限性、或者物体的集合结构过于复杂多变等情况，都不可避免的致使扫描结果中存在缺失区域。但是由于现阶段点云数据集规模有限，此类缺失点云数据集过少，大部分为模型化或完整点云，缺少模拟残缺点云生成方式。同时，现有模拟方法简单实用随机采样模式，不能够针对点云中点对整个点云的重要性来进行衡量输出，且无法满足多样性的模拟各类真实场景中所出现的残缺点云的情况。因此，如何模拟多样化的残缺点云且根据点云中点的属性来生成残缺点云就显得尤为重要。
发明内容
针对上述问题和相关方法的缺陷，本发明提出了一种基于显著性特征的模拟残缺点云的遮罩生成方法，能够很好地对现实场景中不同缺失率的点云进行模拟表示，根据点云中每个点的显著性特征来进行衡量，通过打分判断是否提出，灵活方便，应用广泛，为后期智能化点云补全、修复生成等智能化处理任务提供了基础保障。
为解决上述技术问题，本发明采用如下技术方案来达到目的：
一种基于显著性特征的模拟残缺点云的遮罩生成方法。所述的方法包括以下 步骤：
步骤一、获得当前点云点个数，损失函数衡量方式，以及丢弃率及循环次数；
步骤二、在循环次数外，则跳出循环，输出点云遮罩M；
步骤三、循环次数内，计算点云球心位置坐标x c，可使用所有坐标的中值或平均值来进行粗略衡量其球心位置,或者通过更为精细的网格划分来寻找球心坐标；
步骤四、根据损失函数，计算当前每个点云的梯度g；
步骤五、计算当前点云中每个点相对于球心位置的变化率δ；
步骤六、计算当前点云中每个点的显著性得分s＝-w*δ；
步骤七、将所有点得分从高到低进行排序，删除排序前[pN/T]个点，即M遮罩中这些点设置为0。
本方法提出一种基于显著性特征的模拟残缺点云的遮罩生成方法，通过对点云中每个点的打分来衡量其对整个点云形状的影响程度，进而剔除一些固定比例的点集，形成所需要的可模拟残缺点云的遮罩。该方法为点云补全修复及重构任务提供了前序多样化残缺点云数据，可生成仿真实场景中点云出现各种比例的缺失情况，弥补了现有随机采样无法根据点的重要性程度来生成遮罩的缺陷。
本发明相对于现有技术具有如下的优点及效果：多样性：能够生成多区域缺失的点云情况，较为简单随机下采样方式，更为多样化，而且更加产生更加灵活多变复杂的遮罩。
应当理解，发明内容部分中所描述的内容并非旨在限定本公开的实施例的关键或重要特征，亦非用于限制本公开的范围。本公开的其他特征将通过以下描述变得容易理解。
附图说明
图1为本发明的流程图；
图2为本发明的效果图。
具体实施方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合附图，通过具体实施例对本发明实施例中的技术方案进行清楚、完整的描述。显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本方面保护的范围。
步骤一S1、获得当前点云点个数，损失函数衡量方式，以及丢弃率及循环次数；
在原始残缺点云表达中，仅使用随机采样方式，并不能很好的表达现实场景中出现的残缺情况。因而，本发明通过对每个点基于显著性特征的估计来通过显著性得分进行衡量，进而在输入部分输入损失函数来进一步好的控制其剔除点数位置。
步骤二S2、在循环次数外，则跳出循环，输出点云遮罩M；
该步骤用于控制循环是否就结束而设置，如未结束继续S3-S7，否则，则输出最终点云遮罩结果。遮罩由0与1组成，1表示该点保留，0则表示该点剔除；
步骤三S3、循环次数内，计算点云球心位置坐标x c，可使用所有坐标的中值或平均值来进行粗略衡量其球心位置，或者通过更为精细的网格划分来寻找球心坐标。
由于我们的假设来源于观察针对点云形状而言，其相对边缘形状而言，越中心的点对于形状没有很大的贡献，因而我们剔除点采用“剥洋葱式“法则，故需要知道其核心点位于什么位置，即这里的点云的球心位置坐标，可使用球坐标系来进行衡量，或直角坐标系中的中值亦或者平均值来进行粗略衡量，也可以通过更为精细的网格划分来寻找球心坐标。
步骤四S4、根据损失函数，计算当前每个点云的梯度变化g；
该步骤为了衡量每个点云所受到损失函数的影响变化率。这里的损失函数可以是点云的分类交叉熵损失函数、也可以是分割异或其他基于点云特征计算出来的损失函数；其依赖于任务需求所设置即可。
步骤五S5、计算当前点云中每个点相对于球心位置的变化率δ
步骤六S6、计算当前点云中每个点的显著性得分s＝-w*δ
该步为下一步计算显著性得分提供了前序支撑，我们假定每个点所得到的一个显著性分数的负值与损失函数梯度成正比。其中w权值依赖于用户设置，简单可设置为全1，w权值范围是[0,1],越大表明其参考其梯度密度显著性越多，越小则表明其受梯度影响较小。这部分有利于区域取值一致，保留局部性，也可以通过学习图结构中邻接矩阵获取或学习得到。
步骤七S7、将所有点得分从高到低进行排序，删除排序前[pN/T]个点，即M遮罩中这些点设置为0。
由步骤一S1可知这里每个循环中需要剔除的点，优先将对形状影响较大的点剔除，一方面增加了输出遮罩的可识别难度性，另一方面提供了缺失点云的多样性表达。
实施例：
本实施例公开了一种基于显著性特征的模拟残缺点云的遮罩生成方法，具体包括下列步骤：
步骤S1、获得当前点云点个数，损失函数衡量方式，以及丢弃率及循环次数；
假定点云为X，当前点云个数为N，损失函数为L(x,y),丢弃率为p，循环次数为T，
其中损失函数可以是点云分类的损失函数，例如通过PointNet[1]，PointNet++[2]，DGCNN[3]等点云网络训练获得的点云分类模型中共的损失函数。
步骤S2、在循环次数外，则跳出循环，输出点云遮罩M；
这里M为包含0或1的标签，0表示该点云已被剔除，1表示保留；也可以是剔除后的点云集合；根据任务的不同来进行输出存储；
步骤S3、循环次数内，计算点云球心位置坐标x c，可使用所有坐标的中值或平均值来进行粗略衡量其球心位置。
值得注意的是，直接在直角坐标系计算梯度是存在问题的，因为在直角坐标 系下点不存在视角/角度不变性。为了解决这个问题，我们选择在球坐标系进行计算。在球坐标系下，一个点的坐标表示为(r,ψ,φ)，其中r是半径。如果一个点沿着r的方向偏移δ，将增加损失函数变化及梯度
Figure PCTCN2020100817-appb-000002
这个变化可以作为每个点相对于识别结果的贡献度。为了计算梯度
Figure PCTCN2020100817-appb-000003
可以使用点云的几何中位点xc来进行衡量，其中，(x i1，x i2，x i3)代表点x i的三维坐标：
x cj＝median({x ij|x i∈X})(j＝1，2，3)，       (2)
步骤S4、根据损失函数，计算当前每个点云的梯度g，可通过梯度变化来衡量点云中每个点对整个点云形状的贡献程度；
Figure PCTCN2020100817-appb-000004
其中，
Figure PCTCN2020100817-appb-000005
步骤S5、计算当前点云中每个点相对于球心位置的变化率δ，也即每个点云相对于核心位置的距离远近衡量，该值物理意义表示为，其相对于球心对点云形状产生影响的程度而言的当前点的重要程度比例。在这里认为的假设为，越边缘区域的点，其对相撞影响更大，而中间球心点的有无，对形状的完整性影响不大。可以理解成“剥洋葱”式层级变化程度。越外层距离中心变化率越大。
δ＝gr i,            (4)
步骤S6、计算当前点云中每个点的显著性得分s＝-w*δ
显著性特征得分表现了每个点对于整个形状的影响程度，因而其是相对于球心位置变化量与权值乘积的衡量。w可以简单设为1，或通过自定义经验值亦可，取值范围一般为[0,1]，越大表明其参考其梯度密度显著性越多，越小则表明其受梯度影响较小，这部分有利于区域取值一致，保留局部性，也可以通过学习图结构中邻接矩阵获取或学习得到；
s i＝-wδ,           (5)
这里表示说可能有用户需求来针对每个点有其他权重的参考部分。
步骤S7、将所有点得分从高到低进行排序，删除排序前[pN/T]个点，即M遮 罩中这些点设置为0。
可通过排序算法，例如快速排序，堆排序等对所有点的分数进行排序，提出排序前[pN/T]个点，即可得到提出后的点或者遮罩。
图2为本发明的效果图，遮罩：即图2中浅色部分，颜色较深为原始图像，为了模拟现实残缺点云的情况需要生成遮罩来获取粉色图中的图像；遮罩可以说不同残缺类型的。不同残缺面积以及形状，都会对后续点云重构等处理产生不同影响。
图2种颜色较深的部分为缺失区域B，剩余部分为最终mask部分，百分比为缺失区域：
第一行四张图为：随机种子采样，随机种子缺失部分是连续的一个区域，因为其在一个种子区域一定半径内面积的点，因而可以看到蓝色部分是一块区域，而非不同区域；
第二行四张图：利用本发明的方法，缺失区域可以是好几块，一定程度上增加了修复的难度。
综上所述，本实施例公开了一种基于显著性特征的模拟残缺点云的遮罩生成方法。针对现有大多数模拟方法都仅仅是基于随机采样，不能够很好的表现其对任务难易的贡献量，不能够满足现实点云场景中缺失的情况，且现有专门针对残缺点云的数据集较少，无法提供大量真实案例进行进一步深入学习与探索。因而本发明所提出的一种基于显著性特征的模拟残缺点云的遮罩生成方法，通过引入显著性特征的打分模式，通过为每个点云中点打分衡量其对整个点云形状的贡献度，来按照比例进行剔除或保留，并且可以模拟不同缺失比例下残缺情况。为后续补全重建任务提供了数据来源以及多样化的模拟遮罩。
上述实施例为本发明较佳的实施方式，但本发明的实施方式并不受上述实施例的限制，其他的任何未背离本发明的精神实质与原理下的所作的改变、修饰、替代、组合、简化等均应为等效的置换方式，都包含在本发明的保护范围之内。
参考附录：
[1]Qi,C.R.,Su,H.,Mo,K.,&Guibas,L.J.“Pointnet:Deep learning on point sets for 3d classification and segmentation”.In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2017,pp.652-660.
[2]Charles R Qi,Li Yi,et al.Pointnet++:Deep hierarchical feature learning on point sets in a metric space.arXiv preprint arXiv:1706.02413,2017.
[3]Yue Wang,Yongbin Sun,etal.Dynamic graph cnn for learning on point clouds.ACM Transactions on Graphics,38(5):1–12,October 2019.
[4]Zheng,Tianhang and Chen,Changyou and Yuan,Junsong and Li,Bo and Ren,Kui.PointCloud Saliency Maps.The IEEE International Conference on Computer Vision(ICCV),October,2019

Claims (7)

  1. 一种基于显著性特征的模拟残缺点云的遮罩生成方法，其特征在于，所述的包括以下步骤：
    步骤一、获得当前点云点个数，损失函数衡量方式，以及丢弃率及循环次数；
    步骤二、在循环次数外，则跳出循环，输出点云遮罩M；
    步骤三、循环次数内，计算点云球心位置坐标x c，可使用所有坐标的中值或平均值来进行粗略衡量其球心位置，或者通过更为精细的网格划分来寻找球心坐标；
    步骤四、根据损失函数，计算当前每个点云的梯度g；
    步骤五、计算当前点云中每个点相对于球心位置的变化率δ；
    步骤六、计算当前点云中每个点的显著性得分s＝-w*δ；
    步骤七、将所有点得分从高到低进行排序，删除排序前[pN/T]个点，即M遮罩中这些点设置为0。
  2. 根据权利要求1所述的一种基于显著性特征的模拟残缺点云的遮罩生成方法，其特征在于，所述的步骤一中，确定一种可衡量的损失函数,损失函数可以是点云分类网络模型函数、点云分割网络模型函数、点云生成任务模型函数等相关损失函数。
  3. 根据权利要求1所述的一种基于显著性特征的模拟残缺点云的遮罩生成方法，其特征在于，所述的步骤二中，其为判断是否结束的依据，如循环结束则输出点云遮罩M；所述的M为包含0或1的标签，0表示该点云已被剔除，1表示保留；也可以是剔除后的点云集合；根据任务的不同来进行输出存储。
  4. 根据权利要求1所述的一种基于显著性特征的模拟残缺点云的遮罩生成方法，其特征在于，所述的步骤三中，计算点云球心位置坐标x c，可使用所有坐标的中值或平均值来进行粗略衡量其球心位置，或精确根据网格化方式计算球心位置；在球坐标系下，一个点的坐标表示为(r,ψ,φ)，其中r是半径。如果一个点沿着r的方向偏移δ，将增加损失函数变化及梯度
    Figure PCTCN2020100817-appb-100001
    使用点云的几何中位点xc来进行衡量，其中，(x i1，x i2，x i3)代表点x i的三维坐标：
    x cj＝median({x ij|x i∈X})(j＝1，2，3)，   (2)
  5. 根据权利要求1所述的一种基于显著性特征的模拟残缺点云的遮罩生成方法，其特征在于，所述的步骤四中计算每个点云的梯度g；可通过梯度变化来衡量点云中每个点对整个点云形状的贡献程度；
    Figure PCTCN2020100817-appb-100002
    其中，
    Figure PCTCN2020100817-appb-100003
  6. 根据权利要求1所述的一种基于显著性特征的模拟残缺点云的遮罩生成方法，其特征在于，所述的步骤五中，计算每个点相对于球心位置的变化率δ；其中，越外层距离中心变化率越大；
    δ＝gr i,   (4)。
  7. 根据权利要求1所述的一种基于显著性特征的模拟残缺点云的遮罩生成方法，其特征在于，所述的步骤六中，计算每个点的显著性得分；w可以简单设为全1，或通过自定义经验值亦可
    s i＝-wδ,    (5)
    其取值范围一般为[0,1]，越大表明其参考其梯度密度显著性越多，越小则表明其受梯度影响较小。
PCT/CN2020/100817 2020-07-01 2020-07-08 基于显著性特征的模拟残缺点云的遮罩生成方法 Ceased WO2022000522A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010620484.5 2020-07-01
CN202010620484.5A CN111783798B (zh) 2020-07-01 2020-07-01 基于显著性特征的模拟残缺点云的遮罩生成方法

Publications (1)

Publication Number Publication Date
WO2022000522A1 true WO2022000522A1 (zh) 2022-01-06

Family

ID=72759934

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/100817 Ceased WO2022000522A1 (zh) 2020-07-01 2020-07-08 基于显著性特征的模拟残缺点云的遮罩生成方法

Country Status (2)

Country Link
CN (1) CN111783798B (zh)
WO (1) WO2022000522A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114565738A (zh) * 2022-03-01 2022-05-31 北京工业大学 一种基于局部几何一致性与特征一致性的点云补全方法
CN114638751A (zh) * 2022-02-24 2022-06-17 南京邮电大学 基于生成对抗网络的端到端点云补全方法
CN115984339A (zh) * 2023-02-15 2023-04-18 西安交通大学 基于几何特征精炼与对抗生成网络的双管道点云补全方法
CN116957991A (zh) * 2023-09-19 2023-10-27 北京渲光科技有限公司 三维模型补全方法以及三维模型补全模型的生成方法
CN117150676A (zh) * 2023-09-06 2023-12-01 华中科技大学 船舶轴系法兰的点云数据集制作方法、系统、设备及介质
CN117876817A (zh) * 2023-12-25 2024-04-12 北京化工大学 一种对抗样本生成方法
CN118859185A (zh) * 2024-09-25 2024-10-29 北京中成康富科技股份有限公司 基于毫米波生物雷达的室内人员主动定位感知方法及系统
CN120495574A (zh) * 2025-04-30 2025-08-15 国网黑龙江省电力有限公司超高压公司 基于vr虚拟现实技术的变电站孪生模型构建方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113205104A (zh) * 2021-04-23 2021-08-03 广西大学 一种基于深度学习的点云补全方法
CN114092469B (zh) * 2021-12-02 2022-08-26 四川大学 一种叶片的修补区域的确定方法、装置及可读存储介质
CN120064318B (zh) * 2025-04-23 2025-09-09 浙江大学 一种考虑多因素的机械零件实物扫描点云缺损模拟方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150003723A1 (en) * 2013-06-27 2015-01-01 Chevron U.S.A. Inc. System and method of detecting objects in scene point cloud
CN109345523A (zh) * 2018-09-21 2019-02-15 中国科学院苏州生物医学工程技术研究所 表面缺陷检测和三维建模方法
CN109685793A (zh) * 2018-12-25 2019-04-26 安徽科大智能物流系统有限公司 一种基于三维点云数据的管身缺陷检测方法及系统
CN111008964A (zh) * 2019-11-27 2020-04-14 易启科技(吉林省)有限公司 一种部件表面缺陷检测方法

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104063898B (zh) * 2014-06-30 2017-05-03 厦门大学 一种三维点云自动补全方法
CN106651752B (zh) * 2016-09-27 2020-01-21 深圳市速腾聚创科技有限公司 三维点云数据配准方法及拼接方法
EP3457357B1 (en) * 2017-09-13 2021-07-07 Tata Consultancy Services Limited Methods and systems for surface fitting based change detection in 3d point-cloud

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150003723A1 (en) * 2013-06-27 2015-01-01 Chevron U.S.A. Inc. System and method of detecting objects in scene point cloud
CN109345523A (zh) * 2018-09-21 2019-02-15 中国科学院苏州生物医学工程技术研究所 表面缺陷检测和三维建模方法
CN109685793A (zh) * 2018-12-25 2019-04-26 安徽科大智能物流系统有限公司 一种基于三维点云数据的管身缺陷检测方法及系统
CN111008964A (zh) * 2019-11-27 2020-04-14 易启科技(吉林省)有限公司 一种部件表面缺陷检测方法

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114638751A (zh) * 2022-02-24 2022-06-17 南京邮电大学 基于生成对抗网络的端到端点云补全方法
CN114565738A (zh) * 2022-03-01 2022-05-31 北京工业大学 一种基于局部几何一致性与特征一致性的点云补全方法
CN115984339A (zh) * 2023-02-15 2023-04-18 西安交通大学 基于几何特征精炼与对抗生成网络的双管道点云补全方法
CN117150676A (zh) * 2023-09-06 2023-12-01 华中科技大学 船舶轴系法兰的点云数据集制作方法、系统、设备及介质
CN116957991A (zh) * 2023-09-19 2023-10-27 北京渲光科技有限公司 三维模型补全方法以及三维模型补全模型的生成方法
CN116957991B (zh) * 2023-09-19 2023-12-15 北京渲光科技有限公司 三维模型补全方法
CN117876817A (zh) * 2023-12-25 2024-04-12 北京化工大学 一种对抗样本生成方法
CN118859185A (zh) * 2024-09-25 2024-10-29 北京中成康富科技股份有限公司 基于毫米波生物雷达的室内人员主动定位感知方法及系统
CN120495574A (zh) * 2025-04-30 2025-08-15 国网黑龙江省电力有限公司超高压公司 基于vr虚拟现实技术的变电站孪生模型构建方法

Also Published As

Publication number Publication date
CN111783798A (zh) 2020-10-16
CN111783798B (zh) 2023-09-15

Similar Documents

Publication Publication Date Title
WO2022000522A1 (zh) 基于显著性特征的模拟残缺点云的遮罩生成方法
Gao et al. Deep leaf‐bootstrapping generative adversarial network for structural image data augmentation
CN109741268B (zh) 一种针对壁画的破损图像补全方法
CN110222628A (zh) 一种基于生成式对抗网络的人脸修复方法
CN107341506A (zh) 一种基于多方面深度学习表达的图像情感分类方法
CN110298321A (zh) 基于深度学习图像分类的道路阻断信息提取
WO2023134142A1 (zh) 一种多尺度点云分类方法及系统
CN119164955A (zh) 精密塑胶模具检测方法、装置、设备及存储介质
Zhiheng et al. PyramNet: Point cloud pyramid attention network and graph embedding module for classification and segmentation
CN106408011A (zh) 基于深度学习的激光扫描三维点云树木自动分类方法
CN103077555B (zh) 一种三维模型构成的自动标注方法
Bird et al. From simulation to reality: CNN transfer learning for scene classification
CN115619963B (zh) 一种基于内容感知的城市建筑物实体建模方法
CN108764250A (zh) 一种运用卷积神经网络提取本质图像的方法
CN109635726A (zh) 一种基于对称式深度网络结合多尺度池化的滑坡识别方法
Yan et al. A novel data augmentation method for detection of specific aircraft in remote sensing RGB images
CN108961358A (zh) 一种获得样本图片的方法、装置及电子设备
CN114997268B (zh) 基于巴氏距离度量损失函数旋转目标精细检测方法和系统
CN116935106A (zh) 飞行员飞行模拟训练数据可用性智能识别方法及装置
CN118864713A (zh) 一种多视图语义分割信息引导的三维纹理重建方法及系统
CN115690574A (zh) 一种基于自监督学习的遥感图像船舶检测方法
CN117496065B (zh) 一种面向数字化工厂的三维实时快速显示方法及系统
Huahong et al. A new type method of adhesive handwritten digit recognition based on improved faster rcnn
CN111353412A (zh) 端到端的3D-CapsNet火焰检测方法及装置
Feng et al. Port container number detection based on improved EAST algorithm

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20942926

Country of ref document: EP

Kind code of ref document: A1