CN111783838A - A point cloud feature space representation method for laser SLAM - Google Patents

A point cloud feature space representation method for laser SLAM Download PDF

Info

Publication number
CN111783838A
CN111783838A CN202010504793.6A CN202010504793A CN111783838A CN 111783838 A CN111783838 A CN 111783838A CN 202010504793 A CN202010504793 A CN 202010504793A CN 111783838 A CN111783838 A CN 111783838A
Authority
CN
China
Prior art keywords
point cloud
feature
network
point
global description
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010504793.6A
Other languages
Chinese (zh)
Inventor
莫凌飞
索传哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southeast University
Original Assignee
Southeast University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southeast University filed Critical Southeast University
Priority to CN202010504793.6A priority Critical patent/CN111783838A/en
Publication of CN111783838A publication Critical patent/CN111783838A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

The invention discloses a point cloud feature space characterization method for laser SLAM, which is used for establishing bidirectional mapping between a point cloud space and a point cloud feature space based on a deep learning network, wherein the feature space realizes key problems of closed-loop detection, relocation and the like in the laser SLAM and map compression storage and transmission. The method comprises the steps of realizing global description feature extraction and compression reconstruction of large scene point clouds through a neural network of a self-encoder structure, designing a characteristic space of the point clouds formed by extracting global description features through the encoder network, giving similarity measurement of scenes according to the distance in the characteristic space, judging whether two or more scene structures are similar, and realizing closed-loop detection and relocation of laser SLAM; reconstructing the point cloud through a designed decoder network, reconstructing an original point cloud from global description features extracted from an encoder network, and realizing compressed storage and low-bandwidth transmission of a point cloud map; the constructed encoder network does not need to be trained in advance according to the point cloud map, and has strong generalization capability.

Description

一种用于激光SLAM的点云特征空间表征方法A point cloud feature space representation method for laser SLAM

技术领域technical field

本发明涉及激光点云建图与自主导航技术领域,尤其是一种用于激光SLAM的点云特征空间表征方法。The invention relates to the technical field of laser point cloud mapping and autonomous navigation, in particular to a point cloud feature space representation method for laser SLAM.

背景技术Background technique

如何让移动机器人更好的理解和感知周围环境以及达到灵活可靠的高层次自主导航,而深度学习所推动的人工智能技术为移动机器人环境感知理解和自主导航带来了高速发展,特别是激光雷达的应用,使机器人直接感知三维环境,但也为点云数据处理带来了挑战。同步定位与地图构建(Simultaneous Localization And Mapping,SLAM)是移动机器人实现自主导航定位的基础和关键技术之一,旨在当机器人进入一个未知环境是,建立局部地图并同时确定机器人在地图中的位置。SLAM主要包括前端里程计、后端位姿图优化、闭环检测与地图构建四部分,而闭环检测是其中关键组成部分之一,解决的是确定移动机器人是否回到过先前访问过的环境的问题,准确的检测出闭环场景对SLAM系统纠正长时间的累积误差,构建全局一致的位姿估计和地图是至关重要的,又由于全局位姿信息的状态估计的漂移是不可避免的,机器人中可靠的闭环检测就十分重要,特别是针对室外大场景,大范围位置漂移以及环境识别是SLAM系统的一大挑战。How to make mobile robots better understand and perceive the surrounding environment and achieve flexible and reliable high-level autonomous navigation, and artificial intelligence technology promoted by deep learning has brought rapid development for mobile robots to understand the environment and autonomous navigation, especially lidar The application of this method enables the robot to directly perceive the 3D environment, but it also brings challenges to the point cloud data processing. Simultaneous Localization And Mapping (SLAM) is one of the basic and key technologies for mobile robots to achieve autonomous navigation and positioning. It aims to establish a local map and determine the robot's position in the map when the robot enters an unknown environment . SLAM mainly includes four parts: front-end odometer, back-end pose graph optimization, closed-loop detection and map construction, and closed-loop detection is one of the key components, which solves the problem of determining whether the mobile robot has returned to the previously visited environment. , Accurately detecting the closed-loop scene is crucial for the SLAM system to correct the long-term cumulative error and build a globally consistent pose estimation and map, and because the drift of the state estimation of the global pose information is unavoidable, in the robot Reliable closed-loop detection is very important, especially for large outdoor scenes. Large-scale position drift and environmental recognition are a major challenge for SLAM systems.

其中如何使机器人识别所处环境是一个重要的问题,闭环检测本质上也是一个场景识别问题,通过传感器数据,例如相机、激光雷达等,进行环境识别与重定位。场景识别一直是模式识别领域的热点、难点问题,而相机需要克服光照、视角等因素对场景识别的不利影响,提取其中有用信息。激光雷达相比相机可以克服光照、季节、天气等带来的影响,但激光点云场景数据中物体种类繁多,包含大量动态场景,即使同一场景在不同时间也会在外界的多种影响因素下存在较大的差别,使得场景识别问题十分复杂,而场景识别的关键则在于如何找出一种能够完全表示同一类场景的结构信息和语义信息的全局描述特征,特别是针对大范围激光点云的特征表征方法。Among them, how to make the robot recognize the environment is an important problem. The closed-loop detection is also a scene recognition problem in nature. Through sensor data, such as cameras, lidars, etc., the environment is recognized and relocated. Scene recognition has always been a hot and difficult issue in the field of pattern recognition, and cameras need to overcome the adverse effects of lighting, perspective and other factors on scene recognition and extract useful information. Compared with cameras, lidar can overcome the influence of light, season, weather, etc., but there are many kinds of objects in the laser point cloud scene data, including a large number of dynamic scenes, even the same scene will be affected by various external factors at different times. There is a big difference, which makes the problem of scene recognition very complicated, and the key of scene recognition is how to find a global description feature that can fully represent the structural information and semantic information of the same type of scene, especially for large-scale laser point clouds. feature characterization method.

传统的基于点云的环境识别算法词袋模型(BoW)、FV(Fisher Vector)、VLAD等通常依赖于全局、离线和高分辨率地图,需要提前根据地图训练好码本以实现高精度定位,而基于深度学习的Segmatch提出基于3D分割匹配的场景识别方法,通过分割后的场景块进行局部匹配,通过几何一致性检验保证局部匹配和场景识别的可靠性,但对于动态场景应用,其对场景快的静态假设并不能够满足。PointnetVLAD提出了新的聚合方法,将PointNet与NetVLAD结合,前者用于特征学习,后者用于特征聚合,能够学习得到全局描述特征,同样基于点云深度学习的PCAN网络,在PointNetVLAD的基础上考虑了不同局部特征的贡献程度,引入了自注意力机制,通过网络学习特征聚合时不同局部特征的贡献权重,但以上方法同样没有考虑局部几何特征和语义上下文特征,点云邻域关系和特征空间分布,在大场景点云场景识别特征编码中效果欠佳。Traditional point cloud-based environment recognition algorithms such as bag-of-words (BoW), FV (Fisher Vector), and VLAD usually rely on global, offline, and high-resolution maps, and it is necessary to train a codebook based on the map in advance to achieve high-precision positioning. Segmatch based on deep learning proposes a scene recognition method based on 3D segmentation and matching, which performs local matching through segmented scene blocks, and ensures the reliability of local matching and scene recognition through geometric consistency check. The fast static assumption cannot be satisfied. PointnetVLAD proposes a new aggregation method, which combines PointNet and NetVLAD. The former is used for feature learning and the latter is used for feature aggregation, which can learn to obtain global description features. The PCAN network based on point cloud deep learning is also considered on the basis of PointNetVLAD. The contribution degree of different local features is introduced, and the self-attention mechanism is introduced to learn the contribution weights of different local features during feature aggregation through the network, but the above methods also do not consider local geometric features and semantic context features, point cloud neighborhood relationship and feature space. distribution, and the effect is not good in the encoding of large scene point cloud scene recognition features.

发明内容SUMMARY OF THE INVENTION

本发明所要解决的技术问题在于,提供一种用于激光SLAM的点云特征空间表征方法,能够在复杂的大场景激光点云中提取表示场景结构信息和语义信息的点云全局描述特征并从该特征恢复重建原始点云,用于激光SLAM的闭环检测、重定位以及压缩存储与传输。The technical problem to be solved by the present invention is to provide a point cloud feature space representation method for laser SLAM, which can extract the point cloud global description features representing the scene structure information and semantic information from the complex large scene laser point cloud, and extract the point cloud global description features from the complex large scene laser point cloud. The feature restores and reconstructs the original point cloud for closed-loop detection, relocation, and compressed storage and transmission of laser SLAM.

为解决上述技术问题,本发明提供一种用于激光SLAM的点云特征空间表征方法,包括如下步骤:In order to solve the above technical problems, the present invention provides a point cloud feature space characterization method for laser SLAM, which includes the following steps:

s1、对点云数据集中点云样本进行滤波、降采样预处理,并根据点云场景的相似性构建训练点云样本对,包括正样本ppos与负样本pnegs1. Perform filtering and downsampling preprocessing on the point cloud samples in the point cloud data set, and construct a training point cloud sample pair according to the similarity of the point cloud scene, including a positive sample p pos and a negative sample p neg ;

s2、构建编码器网络,输入步骤s1的训练点云样本对,通过点云自适应邻域特征提取模块、动态图网络聚合模块、全局描述特征生成模块得到输入点云P的全局描述特征f(P);s2. Build an encoder network, input the training point cloud sample pair in step s1, and obtain the global description feature f of the input point cloud P through the point cloud adaptive neighborhood feature extraction module, the dynamic graph network aggregation module, and the global description feature generation module. P);

s3、构建解码器网络,输入步骤s2中生成的全局描述特征,通过层次化生成策略,由生成关键点轮廓到生成精细化稠密化重建点云

Figure BDA0002526140900000021
s3. Build a decoder network, input the global description features generated in step s2, and use a hierarchical generation strategy to generate key point contours to generate refined and dense reconstructed point clouds.
Figure BDA0002526140900000021

s4、取s1中的训练数据集的样本对,对s2的编码器网络进行训练,得到可以度量点云场景相似性的编码器网络模型和参数;s4. Take the sample pair of the training data set in s1, train the encoder network of s2, and obtain the encoder network model and parameters that can measure the similarity of the point cloud scene;

s5、取s1中的训练数据集的样本对和通过s2编码器网络生成的对应全局描述特征,对s3的解码器网络进行训练,得到可以从全局描述特征恢复重建原始点云的解码器网络模型和参数;s5. Take the sample pairs of the training data set in s1 and the corresponding global description features generated by the s2 encoder network, train the decoder network of s3, and obtain a decoder network model that can restore and reconstruct the original point cloud from the global description features and parameters;

s6、在实际SLAM作业中,取s4中的编码器网络模型和参数,对点云关键帧提取全局描述特征,并与历史关键帧的全局描述特征根据欧式距离进行NN匹配,符合匹配条件的作为闭环候选帧或重定位候选位置;s6. In the actual SLAM operation, take the encoder network model and parameters in s4, extract the global description feature for the point cloud key frame, and perform NN matching with the global description feature of the historical key frame according to the Euclidean distance, and the matching condition is taken as Closed-loop candidate frame or relocation candidate position;

s7、在实际SLAM作业中,对s6提取的点云关键帧全局描述特征进行存储或传输,通过s5中的解码器网络模型和参数可以将全局描述特征恢复重建原始SLAM点云地图;s7. In the actual SLAM operation, store or transmit the global description features of the point cloud keyframes extracted by s6. The global description features can be restored and reconstructed to the original SLAM point cloud map through the decoder network model and parameters in s5;

s8、在实际SLAM作业中,取s6中提取的全局描述特征,并与多个协同建图机器人的历史关键帧的全局描述特征根据欧氏距离进行NN匹配,通过所述的点云特征空间表征方法在点云特征空间进行多机器人激光SLAM协同建图任务。s8. In the actual SLAM operation, take the global description features extracted in s6, and perform NN matching with the global description features of the historical key frames of multiple collaborative mapping robots according to the Euclidean distance, through the point cloud feature space representation. The method performs multi-robot laser SLAM collaborative mapping task in point cloud feature space.

优选的,步骤s1中,构建训练样本对的相似性由样本在地图中的位置坐标辅助判断决定,位置间隔在10m以内视为结构相似的正样本,距离10米到50米的视为负样本选取范围,随机选取负样本,距离50米以外的随机选取样本视为极不相似的极负样本。Preferably, in step s1, the similarity of the constructed training sample pair is determined by the auxiliary judgment of the position coordinates of the samples in the map. The position interval within 10m is regarded as a positive sample with similar structure, and the distance between 10m and 50m is regarded as a negative sample Select the range, randomly select negative samples, and randomly select samples beyond 50 meters as extremely dissimilar negative samples.

优选的,步骤s2中,编码器网络的点云自适应邻域特征提取模块以范围[kmin,kmax]构建点云中每个点pi的kNN邻域

Figure BDA0002526140900000031
利用邻域点云的协方差矩阵Σ的特征值λ1≥λ2≥λ3≥0计算点云邻域的线性
Figure BDA0002526140900000032
平面性
Figure BDA0002526140900000033
和散射性
Figure BDA0002526140900000034
特征,并通过香农信息熵度量方程Ek自适应的确定最优邻域大小
Figure BDA0002526140900000036
通过多层感知器特征提取网络提取邻域特征作为点的特征;Preferably, in step s2, the point cloud adaptive neighborhood feature extraction module of the encoder network constructs the kNN neighborhood of each point p i in the point cloud with the range [k min ,k max ]
Figure BDA0002526140900000031
Using the eigenvalue λ 1 ≥λ 2 ≥λ 3 ≥0 of the covariance matrix Σ of the neighborhood point cloud to calculate the linearity of the point cloud neighborhood
Figure BDA0002526140900000032
flatness
Figure BDA0002526140900000033
and scattering
Figure BDA0002526140900000034
feature, and adaptively determine the optimal neighborhood size through the Shannon information entropy metric equation E k
Figure BDA0002526140900000036
Extract neighborhood features as point features through a multi-layer perceptron feature extraction network;

其中:in:

Figure BDA0002526140900000035
V为特征向量矩阵,Ek=-LλlnLλ-PλlnPλ-SλlnSλ
Figure BDA0002526140900000035
V is an eigenvector matrix, E k =-L λ lnL λ -P λ lnP λ -S λ lnS λ .

优选的,步骤s2中,动态图网络聚合模块由两部分聚合组成:Preferably, in step s2, the dynamic graph network aggregation module is composed of two parts:

(a)特征空间动态图网络聚合:在步骤(3)中的特征空间的所有点进行固定k的kNN构建动态图,并通过图网络进行特征聚合,更新点的特征;图网络的特征聚合过程为:fi'=ρ(φΘ(ε(fi,fj))=ρ(φΘ(fi‖(fj-fi))),φΘ为多层感知器MLP,ρ为最大池化;(a) Dynamic graph network aggregation in feature space: In step (3), a kNN with fixed k is used to construct a dynamic graph, and feature aggregation is performed through the graph network to update the features of the points; the feature aggregation process of the graph network is: f i '=ρ(φ Θ (ε(fi , f j ) )=ρ(φ Θ (fi ‖(f j -f i ) )), φ Θ is the multilayer perceptron MLP, and ρ is max pooling;

(b)物理空间动态图网络聚合:在点云物理空间的所有点进行固定k的kNN构建动态图,利用上述(a)更新后的点的特征通过图网络进行特征更新,更新点的特征;中心点pi构建的动态图G=(V,E)有节点V和边E组成,点pi的特征为fi,节点V为邻域点的特征fj,边E为中心点与邻域点特征的关系ε(fi,fj),其中:ε(fi,fj)=fi‖(fj-fi),‖为特征按通道拼接。(b) Network aggregation of dynamic graphs in physical space: construct a dynamic graph with kNN of fixed k at all points in the physical space of the point cloud, and use the features of the updated points in the above (a) to update the features through the graph network, and update the features of the points; The dynamic graph G=(V, E) constructed by the center point pi is composed of node V and edge E. The feature of point pi is f i , the node V is the feature f j of the neighborhood point, and the edge E is the center point and the neighbor point. The relationship between domain point features ε(f i , f j ), where: ε(fi , f j )=f i ‖(f j -f i ) , ‖ is the feature splicing by channel.

优选的,步骤s2中,全局描述特征生成模块通过NetVLAD Layer生成全局描述特征。Preferably, in step s2, the global description feature generation module generates the global description feature through the NetVLAD Layer.

优选的,步骤s3中,层次化生成策略包括两部分:Preferably, in step s3, the hierarchical generation strategy includes two parts:

(a)关键点轮廓生成网络:通过三层全连接网络和Reshape生成粗糙的个点的关键点轮廓;(a) Keypoint contour generation network: Generate rough keypoint contours of points through a three-layer fully connected network and Reshape;

(b)精细化稠密点云生成网络:在上述(a)生成的关键点中每个点拼接η×η个点的2D平面网格

Figure BDA0002526140900000041
并按通道拼接所述步骤s2中生成的全局描述特征,通过MLP生成稠密重建点云;(b) Refinement of dense point cloud generation network: a 2D planar grid of η × η points is stitched at each point in the key points generated in (a) above
Figure BDA0002526140900000041
And splicing the global description features generated in step s2 according to the channel, and generating dense reconstructed point clouds through MLP;

其中:in:

Figure BDA0002526140900000042
Figure BDA0002526140900000042

η为平面网格边上点的数量,ξ为平面网格点的尺寸。 η is the number of points on the edge of the plane grid, and ξ is the size of the plane grid points.

优选的,步骤s4中,使用度量学习方法训练编码器网络,利用四元组损失函数

Figure BDA0002526140900000044
半监督训练使得输入样本与正样本ppos之间全局描述特征的欧式距离δpos更近,与负样本pneg之间的全局描述特征的欧氏距离δneg更远,极负样本
Figure BDA0002526140900000045
对用于防止负样本对之间的全局描述特征距离错误的拉近,极负样本与负样本的欧式距离为
Figure BDA0002526140900000046
其中:
Figure BDA0002526140900000047
α,β为超参数。Preferably, in step s4, use the metric learning method to train the encoder network, and use the quadruple loss function
Figure BDA0002526140900000044
Semi-supervised training makes the Euclidean distance δ pos of the global description feature between the input sample and the positive sample p pos closer, and the Euclidean distance δ neg of the global description feature between the negative sample p neg is farther, and the extremely negative sample
Figure BDA0002526140900000045
To prevent the error of the global description feature distance between pairs of negative samples, the Euclidean distance between extremely negative samples and negative samples is
Figure BDA0002526140900000046
in:
Figure BDA0002526140900000047
α and β are hyperparameters.

优选的,步骤s5中,有监督的方式训练解码器网络,利用损失函数

Figure BDA0002526140900000048
约束重建点云与真值点云的欧氏距离,其中
Figure BDA0002526140900000049
应用在步骤s2的(a)生成的关键点轮廓,用于约束点云的物理空间分布,
Figure BDA00025261409000000410
应用于稠密重建点云;Preferably, in step s5, the decoder network is trained in a supervised manner, using the loss function
Figure BDA0002526140900000048
Constrains the Euclidean distance between the reconstructed point cloud and the ground truth point cloud, where
Figure BDA0002526140900000049
The keypoint contours generated in (a) of step s2 are applied to constrain the physical spatial distribution of the point cloud,
Figure BDA00025261409000000410
applied to densely reconstructed point clouds;

其中:in:

Figure BDA00025261409000000411
ψ(p)为
Figure BDA00025261409000000412
中p的对应点,
Figure BDA00025261409000000411
ψ(p) is
Figure BDA00025261409000000412
The corresponding point of p in ,

Figure BDA00025261409000000413
Figure BDA00025261409000000413

Figure BDA0002526140900000051
Figure BDA0002526140900000051

优选的,步骤s6和s8中,对点云关键帧的全局描述特征进行NN匹配是通过连续N个当前点云帧与连续N个历史点云帧的序列匹配进行判定,如果连续序列匹配关键帧间的全局描述特征的欧式距离小于设定阈值且与其他历史关键帧特征距离相比梯度变化大于设定阈值,则可视为闭环检测候选序列关键帧或重定位候选位置;如果由多个机器人提供当前连续点云帧和历史关键帧,则为多机器人协同建图;通过所述的点云特征空间表征方法在点云的特征空间进行激光SLAM闭环任务,重定位任务和多机器人协同任务。Preferably, in steps s6 and s8, the NN matching of the global description feature of the point cloud key frame is determined by matching the sequence of N consecutive current point cloud frames with N consecutive historical point cloud frames. If the consecutive sequence matches the key frame If the Euclidean distance between the global description features is less than the set threshold and the gradient change compared with other historical keyframe feature distances is greater than the set threshold, it can be regarded as a closed-loop detection candidate sequence keyframe or relocation candidate position; Provide the current continuous point cloud frame and historical key frame, then it is multi-robot collaborative mapping; laser SLAM closed-loop task, relocation task and multi-robot collaborative task are carried out in the feature space of point cloud through the point cloud feature space representation method.

优选的,步骤s7中,利用s4所述编码器网络对激光点云地图中的关键帧提取全局描述特征作为压缩编码进行存储或通信传输,再利用s5所述解码器网络对存储特征或传输接收特征进行还原,重建原始激光点云地图,所述的点云特征空间表征方法可应用于嵌入式系统或低带宽情况。Preferably, in step s7, the encoder network described in s4 is used to extract the global description features of the key frames in the laser point cloud map as compression codes for storage or communication transmission, and then the decoder network described in s5 is used to store or transmit and receive the features. The features are restored, and the original laser point cloud map is reconstructed. The described point cloud feature space representation method can be applied to embedded systems or low bandwidth situations.

本发明的有益效果为:(1)设计了编码器网络对大场景激光点云数据进行了几何结构特征与语义上下文特征的信息提取,充分挖掘了点云场景信息,生成全局描述特征构成点云的特征空间并以此特征空间中的距离给出场景的相似性度量,用于判断两个或多个场景结构是否相似,在点云特征空间即可实现激光SLAM的闭环检测与重定位;(2)通过所设计的解码器网络对点云进行重建,从编码器网络提取的全局描述特征重建原始点云,实现点云地图的压缩存储与低带宽传输;(3)所构建的编码器网路不需要提前根据点云地图进行训练,具有强大的泛化能力。The beneficial effects of the present invention are as follows: (1) An encoder network is designed to extract the information of geometric structure features and semantic context features for the laser point cloud data of large scenes, fully excavate the scene information of the point cloud, and generate the global description feature to form the point cloud and the distance in this feature space gives the similarity measure of the scene, which is used to judge whether two or more scene structures are similar, and the closed-loop detection and relocation of laser SLAM can be realized in the point cloud feature space; ( 2) Reconstruct the point cloud through the designed decoder network, reconstruct the original point cloud from the global description features extracted from the encoder network, and realize the compressed storage and low-bandwidth transmission of the point cloud map; (3) The constructed encoder network Road does not need to be trained according to the point cloud map in advance, and has a strong generalization ability.

附图说明Description of drawings

图1为本发明的方法流程示意图。FIG. 1 is a schematic flow chart of the method of the present invention.

图2为本发明的编码器网络模型结构示意图。FIG. 2 is a schematic structural diagram of an encoder network model of the present invention.

图3为本发明的解码器网络模型结构示意图。FIG. 3 is a schematic structural diagram of a decoder network model of the present invention.

具体实施方式Detailed ways

如图1所示,一种用于激光SLAM的点云特征空间表征方法,包括如下步骤:As shown in Figure 1, a point cloud feature space representation method for laser SLAM includes the following steps:

步骤1:对点云数据集中点云样本进行滤波、降采样等预处理,并根据点云场景的相似性构建训练点云样本对,包括正样本ppos与负样本pnegStep 1: Perform preprocessing such as filtering and downsampling on the point cloud samples in the point cloud data set, and construct training point cloud sample pairs according to the similarity of the point cloud scene, including the positive sample p pos and the negative sample p neg .

构建训练样本对的相似性由样本在地图中的位置坐标辅助判断决定,位置间隔在10m以内视为结构相似的正样本,距离10米到50米的视为负样本选取范围,随机选取负样本,距离50米以外的随机选取样本视为极不相似的极负样本。The similarity of the constructed training sample pair is determined by the auxiliary judgment of the position coordinates of the samples in the map. The position interval within 10m is regarded as a positive sample with similar structure, and the distance between 10m and 50m is regarded as a negative sample selection range, and a negative sample is randomly selected. , and randomly selected samples at a distance of 50 meters are regarded as extremely dissimilar negative samples.

具体为:选用Oxford RobotCar数据集,使用直通滤波器将点云范围设置为[x:±30m,y:±20m,z:0-10m],并使用随机体素滤波器将点云降采样到4096个点,并归一化至[-1~1]。Specifically: select the Oxford RobotCar dataset, use a pass-through filter to set the point cloud range to [x:±30m, y:±20m, z:0-10m], and use a random voxel filter to downsample the point cloud to 4096 points and normalized to [-1~1].

步骤2:如图2所示的网络结构设计来构建编码器网络,输入步骤1的训练点云样本对,通过点云自适应邻域特征提取模块,动态图网络聚合模块,全局描述特征生成模块得到输入点云P的全局描述特征f(P)。Step 2: The network structure shown in Figure 2 is designed to construct the encoder network, input the training point cloud sample pair of step 1, and use the point cloud adaptive neighborhood feature extraction module, dynamic graph network aggregation module, and global description feature generation module Obtain the global description feature f(P) of the input point cloud P.

编码器网络的点云自适应邻域特征提取模块以范围[kmin,kmax]构建点云中每个点pi的kNN邻域

Figure BDA0002526140900000061
利用邻域点云的协方差矩阵Σ的特征值λ1≥λ2≥λ3≥0计算点云邻域的线性
Figure BDA0002526140900000062
平面性
Figure BDA0002526140900000063
和散射性
Figure BDA0002526140900000064
特征,并通过香农信息熵度量方程Ek自适应的确定最优邻域大小
Figure BDA0002526140900000065
通过MLP(多层感知器)特征提取网络提取邻域特征作为点的特征。The point cloud adaptive neighborhood feature extraction module of the encoder network builds a kNN neighborhood of each point pi in the point cloud in the range [ k min ,k max ]
Figure BDA0002526140900000061
Using the eigenvalue λ 1 ≥λ 2 ≥λ 3 ≥0 of the covariance matrix Σ of the neighborhood point cloud to calculate the linearity of the point cloud neighborhood
Figure BDA0002526140900000062
flatness
Figure BDA0002526140900000063
and scattering
Figure BDA0002526140900000064
feature, and adaptively determine the optimal neighborhood size through the Shannon information entropy metric equation E k
Figure BDA0002526140900000065
Neighborhood features are extracted as point features through an MLP (Multilayer Perceptron) feature extraction network.

其中:in:

Figure BDA0002526140900000066
V为特征向量矩阵
Figure BDA0002526140900000066
V is the eigenvector matrix

Ek=-LλlnLλ-PλlnPλ-SλlnSλ E k = -L λ lnL λ -P λ lnP λ -S λ lnS λ

具体为:设置点云自适应邻域特征范围[20-100],步长为10,计算确定最优邻域大小。Specifically: set the point cloud adaptive neighborhood feature range [20-100], the step size is 10, and calculate to determine the optimal neighborhood size.

动态图网络聚合模块由两部分聚合组成:The dynamic graph network aggregation module consists of two parts:

(a)特征空间动态图网络聚合:在权利要求3所述的特征空间的所有点进行固定k的kNN构建动态图,并通过图网络进行特征聚合,更新点的特征;(a) Dynamic graph network aggregation in feature space: kNN with fixed k is constructed at all points in the feature space of claim 3 to construct a dynamic graph, and feature aggregation is performed through the graph network to update the features of the points;

(b)物理空间动态图网络聚合:在点云物理空间的所有点进行固定k的kNN构建动态图,利用上述(a)更新后的点的特征通过图网络进行特征更新,更新点的特征。中心点pi构建的动态图G=(V,E)有节点V和边E组成,点pi的特征为fi,节点V为邻域点的特征fj,边E为中心点与邻域点特征的关系ε(fi,fj)。(b) Physical space dynamic graph network aggregation: A kNN with a fixed k is used to construct a dynamic graph at all points in the physical space of the point cloud, and the features of the updated points (a) above are used to update the features through the graph network to update the features of the points. The dynamic graph G=(V, E) constructed by the center point pi is composed of node V and edge E. The feature of point pi is f i , the node V is the feature f j of the neighborhood point, and the edge E is the center point and the neighbor point. The relation ε(f i ,f j ) of the domain point features.

其中:ε(fi,fj)=fi‖(fj-fi),‖为特征按通道拼接;Among them: ε(f i , f j )=fi ‖(f j -f i ) , ‖ is the feature splicing by channel;

图网络的特征聚合过程为:The feature aggregation process of the graph network is:

fi'=ρ(φΘ(ε(fi,fj))=ρ(φΘ(fi‖(fj-fi))),φΘ为MLP(多层感知器),ρ为最大池化。f i '=ρ(φ Θ (ε(fi , f j ) )=ρ(φ Θ (fi ‖(f j -f i ) )), φ Θ is MLP (multi-layer perceptron), and ρ is max pooling.

具体为:动态图构建使用固定k=20的kNN,φΘ为两层(64,64)的MLPSpecifically: kNN with fixed k=20 is used for dynamic graph construction, and φ Θ is an MLP with two layers (64,64).

优选地,全局描述特征生成模块通过NetVLAD Layer生成全局描述特征。Preferably, the global description feature generation module generates the global description feature through the NetVLAD Layer.

具体为:NetVLAD Layer选取中心特征维度D=256,共选取K=64个中心特征。Specifically: NetVLAD Layer selects the central feature dimension D=256, and selects K=64 central features in total.

全局描述特征生成模块通过NetVLAD Layer生成全局描述特征。The global description feature generation module generates global description features through the NetVLAD Layer.

步骤3:如图3所示的网络结构设计来构建解码器网络,输入步骤2中生成的全局描述特征,通过层次化生成策略,由生成关键点轮廓到生成精细化稠密化重建点云

Figure BDA0002526140900000075
Step 3: The network structure design shown in Figure 3 is used to construct the decoder network, input the global description features generated in step 2, and through the hierarchical generation strategy, from generating key point contours to generating refined and dense reconstructed point clouds
Figure BDA0002526140900000075

层次化生成策略包括两部分:The hierarchical generation strategy consists of two parts:

(a)关键点轮廓生成网络:通过三层全连接网络和Reshape生成粗糙的个点的关键点轮廓;(a) Keypoint contour generation network: Generate rough keypoint contours of points through a three-layer fully connected network and Reshape;

(b)精细化稠密点云生成网络:在上述(a)生成的关键点中每个点拼接η×η个点的2D平面网格

Figure BDA0002526140900000071
并按通道拼接所述步骤2中生成的全局描述特征,通过MLP(多层感知器)生成稠密重建点云。(b) Refinement of dense point cloud generation network: a 2D planar grid of η × η points is stitched at each point in the key points generated in (a) above
Figure BDA0002526140900000071
And the global description features generated in step 2 are spliced by channel, and the dense reconstructed point cloud is generated by MLP (multi-layer perceptron).

其中:in:

Figure BDA0002526140900000072
Figure BDA0002526140900000072

Figure BDA0002526140900000073
η为平面网格边上点的数量,ξ为平面网格点的尺寸。
Figure BDA0002526140900000073
η is the number of points on the edge of the plane grid, and ξ is the size of the plane grid points.

具体为:关键点轮廓生成网络生成包含4096个关键点的粗糙轮廓,并且拼接2×2个点的平面网络,尺寸ξ=0.05。Specifically: the key point contour generation network generates a rough contour containing 4096 key points, and splices a 2×2 point plane network with a size of ξ=0.05.

步骤4:取步骤1中的训练数据集的样本对,对步骤2的编码器网络进行训练,得到可以度量点云场景相似性的编码器网络模型和参数。Step 4: Take the sample pair of the training data set in Step 1, train the encoder network in Step 2, and obtain the encoder network model and parameters that can measure the similarity of the point cloud scene.

使用度量学习方法训练编码器网络,利用四元组损失函数

Figure BDA0002526140900000074
半监督训练使得输入样本与正样本ppos之间全局描述特征的欧式距离δpos更近,与负样本pneg之间的全局描述特征的欧氏距离δneg更远,极负样本
Figure BDA0002526140900000081
对用于防止负样本对之间的全局描述特征距离错误的拉近,极负样本与负样本的欧式距离为
Figure BDA0002526140900000082
Train an encoder network using metric learning methods, utilizing a quadruple loss function
Figure BDA0002526140900000074
Semi-supervised training makes the Euclidean distance δ pos of the global description feature between the input sample and the positive sample p pos closer, and the Euclidean distance δ neg of the global description feature between the negative sample p neg is farther, and the extremely negative sample
Figure BDA0002526140900000081
To prevent the error of the global description feature distance between pairs of negative samples, the Euclidean distance between extremely negative samples and negative samples is
Figure BDA0002526140900000082

其中:in:

Figure BDA0002526140900000083
α,β为超参数。
Figure BDA0002526140900000083
α and β are hyperparameters.

具体为:设置超参数α=0.5,β=0.2,训练时选取2个正样本,18个负样本和1一个随机选取的极负样本。Specifically: set the hyperparameters α=0.5, β=0.2, select 2 positive samples, 18 negative samples and 1 randomly selected extremely negative sample during training.

步骤5:取步骤1中的训练数据集的样本对和通过步骤2编码器网络生成的对应全局描述特征,对步骤3的解码器网络进行训练,得到可以从全局描述特征恢复重建原始点云的解码器网络模型和参数。Step 5: Take the sample pair of the training data set in step 1 and the corresponding global description feature generated by the encoder network in step 2, train the decoder network in step 3, and obtain the original point cloud that can be recovered and reconstructed from the global description feature. Decoder network model and parameters.

步骤5中,有监督的方式训练解码器网络,利用损失函数

Figure BDA0002526140900000084
约束重建点云与真值点云的欧氏距离,其中
Figure BDA0002526140900000085
应用在生成的关键点轮廓,用于约束点云的物理空间分布,
Figure BDA0002526140900000086
应用于稠密重建点云。In step 5, the decoder network is trained in a supervised manner, using the loss function
Figure BDA0002526140900000084
Constrains the Euclidean distance between the reconstructed point cloud and the ground truth point cloud, where
Figure BDA0002526140900000085
Applied to the generated keypoint contours to constrain the physical spatial distribution of the point cloud,
Figure BDA0002526140900000086
Applied to densely reconstructed point clouds.

其中:in:

Figure BDA0002526140900000087
ψ(p)为
Figure BDA0002526140900000088
中p的对应点
Figure BDA0002526140900000087
ψ(p) is
Figure BDA0002526140900000088
The corresponding point of p in

Figure BDA0002526140900000089
Figure BDA0002526140900000089

Figure BDA00025261409000000810
Figure BDA00025261409000000810

步骤6:在实际SLAM作业中,取步骤4中的编码器网络模型和参数,对点云关键帧提取全局描述特征,并与历史关键帧的全局描述特征根据欧式距离进行NN匹配,符合匹配条件的作为闭环候选帧或重定位候选位置。Step 6: In the actual SLAM job, take the encoder network model and parameters in step 4, extract the global description feature for the point cloud key frame, and perform NN matching with the global description feature of the historical key frame according to the Euclidean distance, which meets the matching conditions is used as a closed-loop candidate frame or a relocation candidate position.

步骤6中,对点云关键帧的全局描述特征进行NN匹配是通过连续N个当前点云帧与连续N个历史点云帧的序列匹配进行判定,如果连续序列匹配关键帧间的全局描述特征的欧式距离小于设定阈值且与其他历史关键帧特征距离相比梯度变化大于设定阈值,则可视为闭环检测候选序列关键帧或重定位候选位置。通过所述的点云特征空间表征方法在点云的特征空间进行激光SLAM闭环任务和重定位任务。In step 6, the NN matching of the global description features of the point cloud key frames is determined by the sequence matching of N consecutive current point cloud frames and consecutive N consecutive historical point cloud frames, if the consecutive sequences match the global description features between key frames. The Euclidean distance of is less than the set threshold and the gradient change compared with other historical key frame feature distances is greater than the set threshold, it can be regarded as a closed loop detection candidate sequence key frame or relocation candidate position. The laser SLAM closed-loop task and relocation task are performed in the feature space of the point cloud by the described point cloud feature space characterization method.

步骤7:在实际SLAM作业中,对步骤6提取的点云关键帧全局描述特征进行存储或传输,通过步骤5中的解码器网络模型和参数可以将全局描述特征恢复重建原始SLAM点云地图。Step 7: In the actual SLAM operation, store or transmit the global description features of the point cloud keyframes extracted in Step 6. The global description features can be restored and reconstructed to the original SLAM point cloud map through the decoder network model and parameters in Step 5.

利用步骤4所述编码器网络对激光点云地图中的关键帧提取全局描述特征作为压缩编码进行存储或通信传输,再利用步骤5所述解码器网络对存储特征或传输接收特征进行还原,重建原始激光点云地图,所述的点云特征空间表征方法可应用于嵌入式系统或低带宽情况。Use the encoder network described in step 4 to extract the global description features from the key frames in the laser point cloud map as compression codes for storage or communication transmission, and then use the decoder network described in step 5 to restore the storage features or transmission and reception features, reconstruct Raw laser point cloud map, the described point cloud feature space characterization method can be applied to embedded systems or low bandwidth situations.

步骤8:在实际SLAM作业中,取s6中提取的全局描述特征,并与多个协同建图机器人的历史关键帧的全局描述特征根据欧氏距离进行NN匹配,通过所述的点云特征空间表征方法在点云特征空间进行多机器人激光SLAM协同建图任务。Step 8: In the actual SLAM job, take the global description features extracted in s6, and perform NN matching with the global description features of the historical key frames of multiple collaborative mapping robots according to the Euclidean distance, through the point cloud feature space described The characterization method performs a multi-robot laser SLAM collaborative mapping task in the point cloud feature space.

步骤8中,如果由多个机器人提供当前连续点云帧和历史关键帧,则为多机器人协同建图,对所有的点云历史关键帧的全局描述特征进行NN匹配,则可实现多机器人协同建图。通过所述的点云特征空间表征方法在点云的特征空间进行激光SLAM多机器人协同任务。In step 8, if multiple robots provide the current continuous point cloud frame and historical key frames, the multi-robot collaborative mapping is performed, and NN matching is performed on the global description features of all point cloud historical key frames, so that multi-robot collaboration can be realized. Build a map. The laser SLAM multi-robot collaborative task is carried out in the feature space of the point cloud through the described point cloud feature space characterization method.

本发明的有益效果为:(1)设计了编码器网络对大场景激光点云数据进行了几何结构特征与语义上下文特征的信息提取,该点云特征空间表征方法通过特征检索的方式实现场景识别,在Oxford RobotCar数据集场景识别任务中平均召回率达到93.35%;(2)通过所设计的解码器网络对点云进行重建,从编码器网络提取的全局描述特征重建原始点云,实现点云地图的压缩存储与低带宽传输;(3)所构建的编码器网路不需要提前根据点云地图进行训练,具有强大的泛化能力,在KITTI等多种数据集中可以无需训练直接应用。The beneficial effects of the invention are as follows: (1) An encoder network is designed to extract information of geometric structure features and semantic context features for laser point cloud data in large scenes, and the point cloud feature space representation method realizes scene recognition by means of feature retrieval. , in the scene recognition task of the Oxford RobotCar dataset, the average recall rate reaches 93.35%; (2) The point cloud is reconstructed through the designed decoder network, and the original point cloud is reconstructed from the global description features extracted by the encoder network to realize the point cloud. Map compression storage and low-bandwidth transmission; (3) The constructed encoder network does not need to be trained according to the point cloud map in advance, and has a strong generalization ability, which can be directly applied in various data sets such as KITTI without training.

Claims (10)

1.一种用于激光SLAM的点云特征空间表征方法,其特征在于,包括如下步骤:1. a point cloud feature space characterization method for laser SLAM, is characterized in that, comprises the steps: s1、对点云数据集中点云样本进行滤波、降采样预处理,并根据点云场景的相似性构建训练点云样本对,包括正样本ppos与负样本pnegs1. Perform filtering and downsampling preprocessing on the point cloud samples in the point cloud data set, and construct a training point cloud sample pair according to the similarity of the point cloud scene, including a positive sample p pos and a negative sample p neg ; s2、构建编码器网络,输入步骤s1的训练点云样本对,通过点云自适应邻域特征提取模块、动态图网络聚合模块、全局描述特征生成模块得到输入点云P的全局描述特征f(P);s2. Build an encoder network, input the training point cloud sample pair in step s1, and obtain the global description feature f of the input point cloud P through the point cloud adaptive neighborhood feature extraction module, the dynamic graph network aggregation module, and the global description feature generation module. P); s3、构建解码器网络,输入步骤s2中生成的全局描述特征,通过层次化生成策略,由生成关键点轮廓到生成精细化稠密化重建点云
Figure FDA0002526140890000011
s3. Build a decoder network, input the global description features generated in step s2, and use a hierarchical generation strategy to generate key point contours to generate refined and dense reconstructed point clouds.
Figure FDA0002526140890000011
s4、取s1中的训练数据集的样本对,对s2的编码器网络进行训练,得到可以度量点云场景相似性的编码器网络模型和参数;s4. Take the sample pair of the training data set in s1, train the encoder network of s2, and obtain the encoder network model and parameters that can measure the similarity of the point cloud scene; s5、取s1中的训练数据集的样本对和通过s2编码器网络生成的对应全局描述特征,对s3的解码器网络进行训练,得到可以从全局描述特征恢复重建原始点云的解码器网络模型和参数;s5. Take the sample pairs of the training data set in s1 and the corresponding global description features generated by the s2 encoder network, train the decoder network of s3, and obtain a decoder network model that can restore and reconstruct the original point cloud from the global description features and parameters; s6、在实际SLAM作业中,取s4中的编码器网络模型和参数,对点云关键帧提取全局描述特征,并与历史关键帧的全局描述特征根据欧式距离进行NN匹配,符合匹配条件的作为闭环候选帧或重定位候选位置;s6. In the actual SLAM operation, take the encoder network model and parameters in s4, extract the global description feature for the point cloud key frame, and perform NN matching with the global description feature of the historical key frame according to the Euclidean distance, and the matching condition is taken as Closed-loop candidate frame or relocation candidate position; s7、在实际SLAM作业中,对s6提取的点云关键帧全局描述特征进行存储或传输,通过s5中的解码器网络模型和参数可以将全局描述特征恢复重建原始SLAM点云地图;s7. In the actual SLAM operation, store or transmit the global description features of the point cloud keyframes extracted by s6. The global description features can be restored and reconstructed to the original SLAM point cloud map through the decoder network model and parameters in s5; s8、在实际SLAM作业中,取s6中提取的全局描述特征,并与多个协同建图机器人的历史关键帧的全局描述特征根据欧氏距离进行NN匹配,通过所述的点云特征空间表征方法在点云特征空间进行多机器人激光SLAM协同建图任务。s8. In the actual SLAM operation, take the global description features extracted in s6, and perform NN matching with the global description features of the historical key frames of multiple collaborative mapping robots according to the Euclidean distance, through the point cloud feature space representation. The method performs multi-robot laser SLAM collaborative mapping task in point cloud feature space.
2.如权利要求1所述的用于激光SLAM的点云特征空间表征方法,其特征在于,步骤s1中,构建训练样本对的相似性由样本在地图中的位置坐标辅助判断决定,位置间隔在10m以内视为结构相似的正样本,距离10米到50米的视为负样本选取范围,随机选取负样本,距离50米以外的随机选取样本视为极不相似的极负样本。2. The point cloud feature space representation method for laser SLAM as claimed in claim 1, characterized in that, in step s1, the similarity of constructing the training sample pair is determined by the auxiliary judgment of the position coordinates of the sample in the map, and the position interval Within 10m, it is regarded as a positive sample with similar structure, and the distance from 10m to 50m is regarded as the negative sample selection range, and the negative sample is randomly selected. 3.如权利要求1所述的用于激光SLAM的点云特征空间表征方法,其特征在于,步骤s2中,编码器网络的点云自适应邻域特征提取模块以范围[kmin,kmax]构建点云中每个点pi的kNN邻域Ni,利用邻域点云的协方差矩阵Σ的特征值λ1≥λ2≥λ3≥0计算点云邻域的线性
Figure FDA0002526140890000021
平面性
Figure FDA0002526140890000022
和散射性
Figure FDA0002526140890000023
特征,并通过香农信息熵度量方程Ek自适应的确定最优邻域大小
Figure FDA0002526140890000024
通过多层感知器特征提取网络提取邻域特征作为点的特征;
3. The point cloud feature space characterization method for laser SLAM as claimed in claim 1, wherein in step s2, the point cloud adaptive neighborhood feature extraction module of the encoder network uses the range [k min , km max ] ] Construct the kNN neighborhood N i of each point p i in the point cloud, and use the eigenvalue λ 1 ≥λ 2 ≥λ 3 ≥0 of the covariance matrix Σ of the neighborhood point cloud to calculate the linearity of the point cloud neighborhood
Figure FDA0002526140890000021
flatness
Figure FDA0002526140890000022
and scattering
Figure FDA0002526140890000023
feature, and adaptively determine the optimal neighborhood size through the Shannon information entropy metric equation E k
Figure FDA0002526140890000024
Extract neighborhood features as point features through a multi-layer perceptron feature extraction network;
其中:in:
Figure FDA0002526140890000025
V为特征向量矩阵,Ek=-Lλ ln Lλ-Pλ ln Pλ-Sλ lnSλ
Figure FDA0002526140890000025
V is an eigenvector matrix, E k =-L λ ln L λ -P λ ln P λ -S λ lnS λ .
4.如权利要求1所述的用于激光SLAM的点云特征空间表征方法,其特征在于,步骤s2中,动态图网络聚合模块由两部分聚合组成:4. The point cloud feature space characterization method for laser SLAM as claimed in claim 1, wherein in step s2, the dynamic graph network aggregation module is composed of two parts: (a)特征空间动态图网络聚合:在步骤(3)中的特征空间的所有点进行固定k的kNN构建动态图,并通过图网络进行特征聚合,更新点的特征;图网络的特征聚合过程为:fi'=ρ(φΘ(ε(fi,fj))=ρ(φΘ(fi‖(fj-fi))),φΘ为多层感知器MLP,ρ为最大池化;(a) Dynamic graph network aggregation in feature space: In step (3), a kNN with fixed k is used to construct a dynamic graph, and feature aggregation is performed through the graph network to update the features of the points; the feature aggregation process of the graph network is: f i '=ρ(φ Θ (ε(fi , f j ) )=ρ(φ Θ (fi ‖(f j -f i ) )), φ Θ is the multilayer perceptron MLP, and ρ is max pooling; (b)物理空间动态图网络聚合:在点云物理空间的所有点进行固定k的kNN构建动态图,利用上述(a)更新后的点的特征通过图网络进行特征更新,更新点的特征;中心点pi构建的动态图G=(V,E)有节点V和边E组成,点pi的特征为fi,节点V为邻域点的特征fj,边E为中心点与邻域点特征的关系ε(fi,fj),其中:ε(fi,fj)=fi‖(fj-fi),‖为特征按通道拼接。(b) Network aggregation of dynamic graphs in physical space: construct a dynamic graph with kNN of fixed k at all points in the physical space of the point cloud, and use the features of the updated points in the above (a) to update the features through the graph network, and update the features of the points; The dynamic graph G=(V, E) constructed by the center point pi is composed of node V and edge E. The feature of point pi is f i , the node V is the feature f j of the neighborhood point, and the edge E is the center point and the neighbor point. The relationship between domain point features ε(f i , f j ), where: ε(fi , f j )=f i ‖(f j -f i ) , ‖ is the feature splicing by channel. 5.如权利要求1所述的用于激光SLAM的点云特征空间表征方法,其特征在于,步骤s2中,全局描述特征生成模块通过NetVLAD Layer生成全局描述特征。5. The point cloud feature space representation method for laser SLAM as claimed in claim 1, wherein in step s2, the global description feature generation module generates global description features through NetVLAD Layer. 6.如权利要求1所述的用于激光SLAM的点云特征空间表征方法,其特征在于,步骤s3中,层次化生成策略包括两部分:6. The point cloud feature space characterization method for laser SLAM as claimed in claim 1, wherein in step s3, the hierarchical generation strategy comprises two parts: (a)关键点轮廓生成网络:通过三层全连接网络和Reshape生成粗糙的个点的关键点轮廓;(a) Keypoint contour generation network: Generate rough keypoint contours of points through a three-layer fully connected network and Reshape; (b)精细化稠密点云生成网络:在上述(a)生成的关键点中每个点拼接η×η个点的2D平面网格
Figure FDA0002526140890000031
并按通道拼接所述步骤s2中生成的全局描述特征,通过MLP生成稠密重建点云;
(b) Refinement of dense point cloud generation network: a 2D planar grid of η × η points is stitched at each point in the key points generated in (a) above
Figure FDA0002526140890000031
And splicing the global description features generated in step s2 according to the channel, and generating dense reconstructed point clouds through MLP;
其中:in:
Figure FDA0002526140890000032
Figure FDA0002526140890000032
Figure FDA0002526140890000033
η为平面网格边上点的数量,ξ为平面网格点的尺寸。
Figure FDA0002526140890000033
η is the number of points on the edge of the plane grid, and ξ is the size of the plane grid points.
7.如权利要求1所述的用于激光SLAM的点云特征空间表征方法,其特征在于,步骤s4中,使用度量学习方法训练编码器网络,利用四元组损失函数
Figure FDA0002526140890000034
半监督训练使得输入样本与正样本ppos之间全局描述特征的欧式距离δpos更近,与负样本pneg之间的全局描述特征的欧氏距离δneg更远,极负样本
Figure FDA00025261408900000314
对用于防止负样本对之间的全局描述特征距离错误的拉近,极负样本与负样本的欧式距离为
Figure FDA00025261408900000313
其中:
Figure FDA0002526140890000035
α,β为超参数。
7. The point cloud feature space representation method for laser SLAM as claimed in claim 1, wherein in step s4, the encoder network is trained using a metric learning method, and a quadruple loss function is used.
Figure FDA0002526140890000034
Semi-supervised training makes the Euclidean distance δ pos of the global description feature between the input sample and the positive sample p pos closer, and the Euclidean distance δ neg of the global description feature between the negative sample p neg is farther, and the extremely negative sample
Figure FDA00025261408900000314
To prevent the error of the global description feature distance between pairs of negative samples, the Euclidean distance between extremely negative samples and negative samples is
Figure FDA00025261408900000313
in:
Figure FDA0002526140890000035
α and β are hyperparameters.
8.如权利要求1所述的用于激光SLAM的点云特征空间表征方法,其特征在于,步骤s5中,有监督的方式训练解码器网络,利用损失函数
Figure FDA0002526140890000036
约束重建点云与真值点云的欧氏距离,其中
Figure FDA0002526140890000037
应用在步骤s2的(a)生成的关键点轮廓,用于约束点云的物理空间分布,
Figure FDA0002526140890000038
应用于稠密重建点云;
8. The point cloud feature space representation method for laser SLAM as claimed in claim 1, characterized in that, in step s5, the decoder network is trained in a supervised manner, using a loss function
Figure FDA0002526140890000036
Constrains the Euclidean distance between the reconstructed point cloud and the ground truth point cloud, where
Figure FDA0002526140890000037
The keypoint contours generated in (a) of step s2 are applied to constrain the physical spatial distribution of the point cloud,
Figure FDA0002526140890000038
applied to densely reconstructed point clouds;
其中:in:
Figure FDA0002526140890000039
ψ(p)为
Figure FDA00025261408900000310
中p的对应点,
Figure FDA0002526140890000039
ψ(p) is
Figure FDA00025261408900000310
The corresponding point of p in ,
Figure FDA00025261408900000311
Figure FDA00025261408900000311
Figure FDA00025261408900000312
Figure FDA00025261408900000312
9.如权利要求1所述的用于激光SLAM的点云特征空间表征方法,其特征在于,步骤s6和s8中,对点云关键帧的全局描述特征进行NN匹配是通过连续N个当前点云帧与连续N个历史点云帧的序列匹配进行判定,如果连续序列匹配关键帧间的全局描述特征的欧式距离小于设定阈值且与其他历史关键帧特征距离相比梯度变化大于设定阈值,则可视为闭环检测候选序列关键帧或重定位候选位置;如果由多个机器人提供当前连续点云帧和历史关键帧,则为多机器人协同建图;通过所述的点云特征空间表征方法在点云的特征空间进行激光SLAM闭环任务,重定位任务和多机器人协同任务。9. The point cloud feature space characterization method for laser SLAM as claimed in claim 1, wherein in steps s6 and s8, performing NN matching on the global description feature of the point cloud key frame is performed by consecutive N current points It is judged that the cloud frame matches the sequence of N consecutive historical point cloud frames. If the Euclidean distance of the global description feature between the consecutive sequence matching key frames is less than the set threshold and the gradient change compared with the feature distances of other historical key frames is greater than the set threshold , it can be regarded as a closed-loop detection candidate sequence key frame or a repositioning candidate position; if multiple robots provide the current continuous point cloud frame and historical key frame, it is a multi-robot collaborative mapping; through the point cloud feature space representation The method performs laser SLAM closed-loop tasks, relocation tasks and multi-robot collaborative tasks in the feature space of point clouds. 10.如权利要求1所述的用于激光SLAM的点云特征空间表征方法,其特征在于,步骤s7中,利用s4所述编码器网络对激光点云地图中的关键帧提取全局描述特征作为压缩编码进行存储或通信传输,再利用s5所述解码器网络对存储特征或传输接收特征进行还原,重建原始激光点云地图,所述的点云特征空间表征方法可应用于嵌入式系统或低带宽情况。10. The point cloud feature space characterization method for laser SLAM as claimed in claim 1, wherein in step s7, the encoder network described in s4 is used to extract the global description feature for the key frame in the laser point cloud map as Compression coding is used for storage or communication transmission, and then the decoder network described in s5 is used to restore the storage features or transmission and reception features to reconstruct the original laser point cloud map. The point cloud feature space representation method can be applied to embedded systems or low bandwidth situation.
CN202010504793.6A 2020-06-05 2020-06-05 A point cloud feature space representation method for laser SLAM Pending CN111783838A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010504793.6A CN111783838A (en) 2020-06-05 2020-06-05 A point cloud feature space representation method for laser SLAM

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010504793.6A CN111783838A (en) 2020-06-05 2020-06-05 A point cloud feature space representation method for laser SLAM

Publications (1)

Publication Number Publication Date
CN111783838A true CN111783838A (en) 2020-10-16

Family

ID=72754632

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010504793.6A Pending CN111783838A (en) 2020-06-05 2020-06-05 A point cloud feature space representation method for laser SLAM

Country Status (1)

Country Link
CN (1) CN111783838A (en)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112581515A (en) * 2020-11-13 2021-03-30 上海交通大学 Outdoor scene point cloud registration method based on graph neural network
CN113139996A (en) * 2021-05-06 2021-07-20 南京大学 Point cloud registration method and system based on three-dimensional point cloud geometric feature learning
CN113219493A (en) * 2021-04-26 2021-08-06 中山大学 End-to-end point cloud data compression method based on three-dimensional laser radar sensor
CN113343765A (en) * 2021-05-11 2021-09-03 武汉大学 Scene retrieval method and system based on point cloud rigid registration
CN113486963A (en) * 2021-07-12 2021-10-08 厦门大学 Density self-adaptive point cloud end-to-end sampling method
CN113538675A (en) * 2021-06-30 2021-10-22 同济人工智能研究院(苏州)有限公司 Neural network for calculating attention weight for laser point cloud and training method
CN113674416A (en) * 2021-08-26 2021-11-19 中国电子科技集团公司信息科学研究院 Three-dimensional map construction method and device, electronic equipment and storage medium
CN113808268A (en) * 2021-09-17 2021-12-17 浙江大学 A low-bandwidth globally consistent multi-machine dense mapping method
CN113810736A (en) * 2021-08-26 2021-12-17 北京邮电大学 AI-driven real-time point cloud video transmission method and system
CN113807184A (en) * 2021-08-17 2021-12-17 北京百度网讯科技有限公司 Obstacle detection method and device, electronic equipment and automatic driving vehicle
CN114612660A (en) * 2022-03-01 2022-06-10 浙江工业大学 Three-dimensional modeling method based on multi-feature fusion point cloud segmentation
CN114998699A (en) * 2022-06-09 2022-09-02 北京大学深圳研究生院 Distortion recovery method, device and electronic device for compressed point cloud
CN115578426A (en) * 2022-10-25 2023-01-06 哈尔滨工业大学 Indoor service robot repositioning method based on dense feature matching
CN115950414A (en) * 2023-01-29 2023-04-11 北京工商大学 Adaptive multi-fusion SLAM method for different sensor data
CN116222577A (en) * 2023-04-27 2023-06-06 苏州浪潮智能科技有限公司 Closed loop detection method, training method, system, electronic equipment and storage medium
CN117557904A (en) * 2023-11-14 2024-02-13 国网电力空间技术有限公司 Multi-model-fused power transmission channel construction machinery identification method

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102592281A (en) * 2012-01-16 2012-07-18 北方工业大学 Image matching method
CN110243370A (en) * 2019-05-16 2019-09-17 西安理工大学 A 3D Semantic Map Construction Method for Indoor Environment Based on Deep Learning

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102592281A (en) * 2012-01-16 2012-07-18 北方工业大学 Image matching method
CN110243370A (en) * 2019-05-16 2019-09-17 西安理工大学 A 3D Semantic Map Construction Method for Indoor Environment Based on Deep Learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHUANZHE SUO等: "LPD-AE: Latent Space Representation of Large-Scale 3D Point Cloud", 《IEEE ACCESS》 *

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112581515B (en) * 2020-11-13 2022-12-13 上海交通大学 Point cloud registration method for outdoor scenes based on graph neural network
CN112581515A (en) * 2020-11-13 2021-03-30 上海交通大学 Outdoor scene point cloud registration method based on graph neural network
CN113219493A (en) * 2021-04-26 2021-08-06 中山大学 End-to-end point cloud data compression method based on three-dimensional laser radar sensor
CN113219493B (en) * 2021-04-26 2023-08-25 中山大学 An end-to-end point cloud data compression method based on 3D lidar sensor
CN113139996A (en) * 2021-05-06 2021-07-20 南京大学 Point cloud registration method and system based on three-dimensional point cloud geometric feature learning
CN113139996B (en) * 2021-05-06 2024-02-06 南京大学 Point cloud registration method and system based on three-dimensional point cloud geometric feature learning
CN113343765B (en) * 2021-05-11 2022-07-22 武汉大学 Scene retrieval method and system based on point cloud rigid registration
CN113343765A (en) * 2021-05-11 2021-09-03 武汉大学 Scene retrieval method and system based on point cloud rigid registration
CN113538675A (en) * 2021-06-30 2021-10-22 同济人工智能研究院(苏州)有限公司 Neural network for calculating attention weight for laser point cloud and training method
CN113486963A (en) * 2021-07-12 2021-10-08 厦门大学 Density self-adaptive point cloud end-to-end sampling method
CN113486963B (en) * 2021-07-12 2023-07-07 厦门大学 A Density-Adaptive Point Cloud-to-End Sampling Method
CN113807184A (en) * 2021-08-17 2021-12-17 北京百度网讯科技有限公司 Obstacle detection method and device, electronic equipment and automatic driving vehicle
CN113810736A (en) * 2021-08-26 2021-12-17 北京邮电大学 AI-driven real-time point cloud video transmission method and system
CN113674416B (en) * 2021-08-26 2024-04-26 中国电子科技集团公司信息科学研究院 Three-dimensional map construction method and device, electronic equipment and storage medium
CN113674416A (en) * 2021-08-26 2021-11-19 中国电子科技集团公司信息科学研究院 Three-dimensional map construction method and device, electronic equipment and storage medium
CN113808268A (en) * 2021-09-17 2021-12-17 浙江大学 A low-bandwidth globally consistent multi-machine dense mapping method
CN113808268B (en) * 2021-09-17 2023-09-26 浙江大学 A low-bandwidth globally consistent multi-machine dense mapping method
CN114612660A (en) * 2022-03-01 2022-06-10 浙江工业大学 Three-dimensional modeling method based on multi-feature fusion point cloud segmentation
CN114612660B (en) * 2022-03-01 2025-06-13 浙江工业大学 A 3D modeling method based on multi-feature fusion point cloud segmentation
CN114998699A (en) * 2022-06-09 2022-09-02 北京大学深圳研究生院 Distortion recovery method, device and electronic device for compressed point cloud
CN115578426B (en) * 2022-10-25 2023-08-18 哈尔滨工业大学 Indoor service robot repositioning method based on dense feature matching
CN115578426A (en) * 2022-10-25 2023-01-06 哈尔滨工业大学 Indoor service robot repositioning method based on dense feature matching
CN115950414A (en) * 2023-01-29 2023-04-11 北京工商大学 Adaptive multi-fusion SLAM method for different sensor data
CN116222577A (en) * 2023-04-27 2023-06-06 苏州浪潮智能科技有限公司 Closed loop detection method, training method, system, electronic equipment and storage medium
CN117557904A (en) * 2023-11-14 2024-02-13 国网电力空间技术有限公司 Multi-model-fused power transmission channel construction machinery identification method

Similar Documents

Publication Publication Date Title
CN111783838A (en) A point cloud feature space representation method for laser SLAM
Vidanapathirana et al. Locus: Lidar-based place recognition using spatiotemporal higher-order pooling
CN109597087B (en) A 3D object detection method based on point cloud data
CN112529015B (en) Three-dimensional point cloud processing method, device and equipment based on geometric unwrapping
CN113139996B (en) Point cloud registration method and system based on three-dimensional point cloud geometric feature learning
Duan et al. Deep learning for visual SLAM in transportation robotics: A review
Murali et al. Utilizing semantic visual landmarks for precise vehicle navigation
CN106910202B (en) Image segmentation method and system for ground object of remote sensing image
CN113536920A (en) A semi-supervised 3D point cloud object detection method
Du et al. ResDLPS-Net: Joint residual-dense optimization for large-scale point cloud semantic segmentation
CN111739051A (en) A Multi-Sequence MRI Image Segmentation Method Based on Residual Network
CN112528808B (en) Method and device for identifying obstacles on celestial body surface
CN111797836A (en) A deep learning-based obstacle segmentation method for extraterrestrial celestial rovers
KR102615412B1 (en) Apparatus and method for performing visual localization
CN116772820A (en) Local refinement mapping system and method based on SLAM and semantic segmentation
Contreras et al. O-poco: Online point cloud compression mapping for visual odometry and slam
CN109785409B (en) Image-text data fusion method and system based on attention mechanism
CN115311538A (en) A method for agent target search based on scene prior
CN103810747A (en) Three-dimensional point cloud object shape similarity comparing method based on two-dimensional mainstream shape
CN116222577A (en) Closed loop detection method, training method, system, electronic equipment and storage medium
CN114463614B (en) Salient Object Detection Using Hierarchical Saliency Modeling with Generative Parameters
Wang et al. Deep learning-based semantic segmentation and surface reconstruction for point clouds of offshore oil production equipment
Jia et al. DispNet based stereo matching for planetary scene depth estimation using remote sensing images
CN114663600A (en) Point cloud reconstruction method and system based on self-encoder
CN107529647A (en) A kind of cloud atlas cloud amount computational methods based on the unsupervised sparse learning network of multilayer

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20201016