CN111783838A - A point cloud feature space representation method for laser SLAM - Google Patents
A point cloud feature space representation method for laser SLAM Download PDFInfo
- Publication number
- CN111783838A CN111783838A CN202010504793.6A CN202010504793A CN111783838A CN 111783838 A CN111783838 A CN 111783838A CN 202010504793 A CN202010504793 A CN 202010504793A CN 111783838 A CN111783838 A CN 111783838A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- feature
- network
- point
- global description
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000012512 characterization method Methods 0.000 claims abstract description 13
- 238000013507 mapping Methods 0.000 claims abstract description 12
- 238000001514 detection method Methods 0.000 claims abstract description 11
- 230000005540 biological transmission Effects 0.000 claims abstract description 10
- 238000000605 extraction Methods 0.000 claims abstract description 10
- 238000003860 storage Methods 0.000 claims abstract description 10
- 230000006835 compression Effects 0.000 claims abstract description 6
- 238000007906 compression Methods 0.000 claims abstract description 6
- 230000002776 aggregation Effects 0.000 claims description 21
- 238000004220 aggregation Methods 0.000 claims description 21
- 238000012549 training Methods 0.000 claims description 20
- 230000003044 adaptive effect Effects 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000009826 distribution Methods 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 3
- 238000004891 communication Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 abstract description 4
- 238000013528 artificial neural network Methods 0.000 abstract 1
- 230000002457 bidirectional effect Effects 0.000 abstract 1
- 238000005259 measurement Methods 0.000 abstract 1
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 101000595182 Homo sapiens Podocan Proteins 0.000 description 1
- 102100036036 Podocan Human genes 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 238000011524 similarity measure Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
Description
技术领域technical field
本发明涉及激光点云建图与自主导航技术领域,尤其是一种用于激光SLAM的点云特征空间表征方法。The invention relates to the technical field of laser point cloud mapping and autonomous navigation, in particular to a point cloud feature space representation method for laser SLAM.
背景技术Background technique
如何让移动机器人更好的理解和感知周围环境以及达到灵活可靠的高层次自主导航,而深度学习所推动的人工智能技术为移动机器人环境感知理解和自主导航带来了高速发展,特别是激光雷达的应用,使机器人直接感知三维环境,但也为点云数据处理带来了挑战。同步定位与地图构建(Simultaneous Localization And Mapping,SLAM)是移动机器人实现自主导航定位的基础和关键技术之一,旨在当机器人进入一个未知环境是,建立局部地图并同时确定机器人在地图中的位置。SLAM主要包括前端里程计、后端位姿图优化、闭环检测与地图构建四部分,而闭环检测是其中关键组成部分之一,解决的是确定移动机器人是否回到过先前访问过的环境的问题,准确的检测出闭环场景对SLAM系统纠正长时间的累积误差,构建全局一致的位姿估计和地图是至关重要的,又由于全局位姿信息的状态估计的漂移是不可避免的,机器人中可靠的闭环检测就十分重要,特别是针对室外大场景,大范围位置漂移以及环境识别是SLAM系统的一大挑战。How to make mobile robots better understand and perceive the surrounding environment and achieve flexible and reliable high-level autonomous navigation, and artificial intelligence technology promoted by deep learning has brought rapid development for mobile robots to understand the environment and autonomous navigation, especially lidar The application of this method enables the robot to directly perceive the 3D environment, but it also brings challenges to the point cloud data processing. Simultaneous Localization And Mapping (SLAM) is one of the basic and key technologies for mobile robots to achieve autonomous navigation and positioning. It aims to establish a local map and determine the robot's position in the map when the robot enters an unknown environment . SLAM mainly includes four parts: front-end odometer, back-end pose graph optimization, closed-loop detection and map construction, and closed-loop detection is one of the key components, which solves the problem of determining whether the mobile robot has returned to the previously visited environment. , Accurately detecting the closed-loop scene is crucial for the SLAM system to correct the long-term cumulative error and build a globally consistent pose estimation and map, and because the drift of the state estimation of the global pose information is unavoidable, in the robot Reliable closed-loop detection is very important, especially for large outdoor scenes. Large-scale position drift and environmental recognition are a major challenge for SLAM systems.
其中如何使机器人识别所处环境是一个重要的问题,闭环检测本质上也是一个场景识别问题,通过传感器数据,例如相机、激光雷达等,进行环境识别与重定位。场景识别一直是模式识别领域的热点、难点问题,而相机需要克服光照、视角等因素对场景识别的不利影响,提取其中有用信息。激光雷达相比相机可以克服光照、季节、天气等带来的影响,但激光点云场景数据中物体种类繁多,包含大量动态场景,即使同一场景在不同时间也会在外界的多种影响因素下存在较大的差别,使得场景识别问题十分复杂,而场景识别的关键则在于如何找出一种能够完全表示同一类场景的结构信息和语义信息的全局描述特征,特别是针对大范围激光点云的特征表征方法。Among them, how to make the robot recognize the environment is an important problem. The closed-loop detection is also a scene recognition problem in nature. Through sensor data, such as cameras, lidars, etc., the environment is recognized and relocated. Scene recognition has always been a hot and difficult issue in the field of pattern recognition, and cameras need to overcome the adverse effects of lighting, perspective and other factors on scene recognition and extract useful information. Compared with cameras, lidar can overcome the influence of light, season, weather, etc., but there are many kinds of objects in the laser point cloud scene data, including a large number of dynamic scenes, even the same scene will be affected by various external factors at different times. There is a big difference, which makes the problem of scene recognition very complicated, and the key of scene recognition is how to find a global description feature that can fully represent the structural information and semantic information of the same type of scene, especially for large-scale laser point clouds. feature characterization method.
传统的基于点云的环境识别算法词袋模型(BoW)、FV(Fisher Vector)、VLAD等通常依赖于全局、离线和高分辨率地图,需要提前根据地图训练好码本以实现高精度定位,而基于深度学习的Segmatch提出基于3D分割匹配的场景识别方法,通过分割后的场景块进行局部匹配,通过几何一致性检验保证局部匹配和场景识别的可靠性,但对于动态场景应用,其对场景快的静态假设并不能够满足。PointnetVLAD提出了新的聚合方法,将PointNet与NetVLAD结合,前者用于特征学习,后者用于特征聚合,能够学习得到全局描述特征,同样基于点云深度学习的PCAN网络,在PointNetVLAD的基础上考虑了不同局部特征的贡献程度,引入了自注意力机制,通过网络学习特征聚合时不同局部特征的贡献权重,但以上方法同样没有考虑局部几何特征和语义上下文特征,点云邻域关系和特征空间分布,在大场景点云场景识别特征编码中效果欠佳。Traditional point cloud-based environment recognition algorithms such as bag-of-words (BoW), FV (Fisher Vector), and VLAD usually rely on global, offline, and high-resolution maps, and it is necessary to train a codebook based on the map in advance to achieve high-precision positioning. Segmatch based on deep learning proposes a scene recognition method based on 3D segmentation and matching, which performs local matching through segmented scene blocks, and ensures the reliability of local matching and scene recognition through geometric consistency check. The fast static assumption cannot be satisfied. PointnetVLAD proposes a new aggregation method, which combines PointNet and NetVLAD. The former is used for feature learning and the latter is used for feature aggregation, which can learn to obtain global description features. The PCAN network based on point cloud deep learning is also considered on the basis of PointNetVLAD. The contribution degree of different local features is introduced, and the self-attention mechanism is introduced to learn the contribution weights of different local features during feature aggregation through the network, but the above methods also do not consider local geometric features and semantic context features, point cloud neighborhood relationship and feature space. distribution, and the effect is not good in the encoding of large scene point cloud scene recognition features.
发明内容SUMMARY OF THE INVENTION
本发明所要解决的技术问题在于,提供一种用于激光SLAM的点云特征空间表征方法,能够在复杂的大场景激光点云中提取表示场景结构信息和语义信息的点云全局描述特征并从该特征恢复重建原始点云,用于激光SLAM的闭环检测、重定位以及压缩存储与传输。The technical problem to be solved by the present invention is to provide a point cloud feature space representation method for laser SLAM, which can extract the point cloud global description features representing the scene structure information and semantic information from the complex large scene laser point cloud, and extract the point cloud global description features from the complex large scene laser point cloud. The feature restores and reconstructs the original point cloud for closed-loop detection, relocation, and compressed storage and transmission of laser SLAM.
为解决上述技术问题,本发明提供一种用于激光SLAM的点云特征空间表征方法,包括如下步骤:In order to solve the above technical problems, the present invention provides a point cloud feature space characterization method for laser SLAM, which includes the following steps:
s1、对点云数据集中点云样本进行滤波、降采样预处理,并根据点云场景的相似性构建训练点云样本对,包括正样本ppos与负样本pneg;s1. Perform filtering and downsampling preprocessing on the point cloud samples in the point cloud data set, and construct a training point cloud sample pair according to the similarity of the point cloud scene, including a positive sample p pos and a negative sample p neg ;
s2、构建编码器网络,输入步骤s1的训练点云样本对,通过点云自适应邻域特征提取模块、动态图网络聚合模块、全局描述特征生成模块得到输入点云P的全局描述特征f(P);s2. Build an encoder network, input the training point cloud sample pair in step s1, and obtain the global description feature f of the input point cloud P through the point cloud adaptive neighborhood feature extraction module, the dynamic graph network aggregation module, and the global description feature generation module. P);
s3、构建解码器网络,输入步骤s2中生成的全局描述特征,通过层次化生成策略,由生成关键点轮廓到生成精细化稠密化重建点云 s3. Build a decoder network, input the global description features generated in step s2, and use a hierarchical generation strategy to generate key point contours to generate refined and dense reconstructed point clouds.
s4、取s1中的训练数据集的样本对,对s2的编码器网络进行训练,得到可以度量点云场景相似性的编码器网络模型和参数;s4. Take the sample pair of the training data set in s1, train the encoder network of s2, and obtain the encoder network model and parameters that can measure the similarity of the point cloud scene;
s5、取s1中的训练数据集的样本对和通过s2编码器网络生成的对应全局描述特征,对s3的解码器网络进行训练,得到可以从全局描述特征恢复重建原始点云的解码器网络模型和参数;s5. Take the sample pairs of the training data set in s1 and the corresponding global description features generated by the s2 encoder network, train the decoder network of s3, and obtain a decoder network model that can restore and reconstruct the original point cloud from the global description features and parameters;
s6、在实际SLAM作业中,取s4中的编码器网络模型和参数,对点云关键帧提取全局描述特征,并与历史关键帧的全局描述特征根据欧式距离进行NN匹配,符合匹配条件的作为闭环候选帧或重定位候选位置;s6. In the actual SLAM operation, take the encoder network model and parameters in s4, extract the global description feature for the point cloud key frame, and perform NN matching with the global description feature of the historical key frame according to the Euclidean distance, and the matching condition is taken as Closed-loop candidate frame or relocation candidate position;
s7、在实际SLAM作业中,对s6提取的点云关键帧全局描述特征进行存储或传输,通过s5中的解码器网络模型和参数可以将全局描述特征恢复重建原始SLAM点云地图;s7. In the actual SLAM operation, store or transmit the global description features of the point cloud keyframes extracted by s6. The global description features can be restored and reconstructed to the original SLAM point cloud map through the decoder network model and parameters in s5;
s8、在实际SLAM作业中,取s6中提取的全局描述特征,并与多个协同建图机器人的历史关键帧的全局描述特征根据欧氏距离进行NN匹配,通过所述的点云特征空间表征方法在点云特征空间进行多机器人激光SLAM协同建图任务。s8. In the actual SLAM operation, take the global description features extracted in s6, and perform NN matching with the global description features of the historical key frames of multiple collaborative mapping robots according to the Euclidean distance, through the point cloud feature space representation. The method performs multi-robot laser SLAM collaborative mapping task in point cloud feature space.
优选的,步骤s1中,构建训练样本对的相似性由样本在地图中的位置坐标辅助判断决定,位置间隔在10m以内视为结构相似的正样本,距离10米到50米的视为负样本选取范围,随机选取负样本,距离50米以外的随机选取样本视为极不相似的极负样本。Preferably, in step s1, the similarity of the constructed training sample pair is determined by the auxiliary judgment of the position coordinates of the samples in the map. The position interval within 10m is regarded as a positive sample with similar structure, and the distance between 10m and 50m is regarded as a negative sample Select the range, randomly select negative samples, and randomly select samples beyond 50 meters as extremely dissimilar negative samples.
优选的,步骤s2中,编码器网络的点云自适应邻域特征提取模块以范围[kmin,kmax]构建点云中每个点pi的kNN邻域利用邻域点云的协方差矩阵Σ的特征值λ1≥λ2≥λ3≥0计算点云邻域的线性平面性和散射性特征,并通过香农信息熵度量方程Ek自适应的确定最优邻域大小通过多层感知器特征提取网络提取邻域特征作为点的特征;Preferably, in step s2, the point cloud adaptive neighborhood feature extraction module of the encoder network constructs the kNN neighborhood of each point p i in the point cloud with the range [k min ,k max ] Using the eigenvalue λ 1 ≥λ 2 ≥λ 3 ≥0 of the covariance matrix Σ of the neighborhood point cloud to calculate the linearity of the point cloud neighborhood flatness and scattering feature, and adaptively determine the optimal neighborhood size through the Shannon information entropy metric equation E k Extract neighborhood features as point features through a multi-layer perceptron feature extraction network;
其中:in:
V为特征向量矩阵,Ek=-LλlnLλ-PλlnPλ-SλlnSλ。 V is an eigenvector matrix, E k =-L λ lnL λ -P λ lnP λ -S λ lnS λ .
优选的,步骤s2中,动态图网络聚合模块由两部分聚合组成:Preferably, in step s2, the dynamic graph network aggregation module is composed of two parts:
(a)特征空间动态图网络聚合:在步骤(3)中的特征空间的所有点进行固定k的kNN构建动态图,并通过图网络进行特征聚合,更新点的特征;图网络的特征聚合过程为:fi'=ρ(φΘ(ε(fi,fj))=ρ(φΘ(fi‖(fj-fi))),φΘ为多层感知器MLP,ρ为最大池化;(a) Dynamic graph network aggregation in feature space: In step (3), a kNN with fixed k is used to construct a dynamic graph, and feature aggregation is performed through the graph network to update the features of the points; the feature aggregation process of the graph network is: f i '=ρ(φ Θ (ε(fi , f j ) )=ρ(φ Θ (fi ‖(f j -f i ) )), φ Θ is the multilayer perceptron MLP, and ρ is max pooling;
(b)物理空间动态图网络聚合:在点云物理空间的所有点进行固定k的kNN构建动态图,利用上述(a)更新后的点的特征通过图网络进行特征更新,更新点的特征;中心点pi构建的动态图G=(V,E)有节点V和边E组成,点pi的特征为fi,节点V为邻域点的特征fj,边E为中心点与邻域点特征的关系ε(fi,fj),其中:ε(fi,fj)=fi‖(fj-fi),‖为特征按通道拼接。(b) Network aggregation of dynamic graphs in physical space: construct a dynamic graph with kNN of fixed k at all points in the physical space of the point cloud, and use the features of the updated points in the above (a) to update the features through the graph network, and update the features of the points; The dynamic graph G=(V, E) constructed by the center point pi is composed of node V and edge E. The feature of point pi is f i , the node V is the feature f j of the neighborhood point, and the edge E is the center point and the neighbor point. The relationship between domain point features ε(f i , f j ), where: ε(fi , f j )=f i ‖(f j -f i ) , ‖ is the feature splicing by channel.
优选的,步骤s2中,全局描述特征生成模块通过NetVLAD Layer生成全局描述特征。Preferably, in step s2, the global description feature generation module generates the global description feature through the NetVLAD Layer.
优选的,步骤s3中,层次化生成策略包括两部分:Preferably, in step s3, the hierarchical generation strategy includes two parts:
(a)关键点轮廓生成网络:通过三层全连接网络和Reshape生成粗糙的个点的关键点轮廓;(a) Keypoint contour generation network: Generate rough keypoint contours of points through a three-layer fully connected network and Reshape;
(b)精细化稠密点云生成网络:在上述(a)生成的关键点中每个点拼接η×η个点的2D平面网格并按通道拼接所述步骤s2中生成的全局描述特征,通过MLP生成稠密重建点云;(b) Refinement of dense point cloud generation network: a 2D planar grid of η × η points is stitched at each point in the key points generated in (a) above And splicing the global description features generated in step s2 according to the channel, and generating dense reconstructed point clouds through MLP;
其中:in:
η为平面网格边上点的数量,ξ为平面网格点的尺寸。 η is the number of points on the edge of the plane grid, and ξ is the size of the plane grid points.
优选的,步骤s4中,使用度量学习方法训练编码器网络,利用四元组损失函数半监督训练使得输入样本与正样本ppos之间全局描述特征的欧式距离δpos更近,与负样本pneg之间的全局描述特征的欧氏距离δneg更远,极负样本对用于防止负样本对之间的全局描述特征距离错误的拉近,极负样本与负样本的欧式距离为其中:α,β为超参数。Preferably, in step s4, use the metric learning method to train the encoder network, and use the quadruple loss function Semi-supervised training makes the Euclidean distance δ pos of the global description feature between the input sample and the positive sample p pos closer, and the Euclidean distance δ neg of the global description feature between the negative sample p neg is farther, and the extremely negative sample To prevent the error of the global description feature distance between pairs of negative samples, the Euclidean distance between extremely negative samples and negative samples is in: α and β are hyperparameters.
优选的,步骤s5中,有监督的方式训练解码器网络,利用损失函数约束重建点云与真值点云的欧氏距离,其中应用在步骤s2的(a)生成的关键点轮廓,用于约束点云的物理空间分布,应用于稠密重建点云;Preferably, in step s5, the decoder network is trained in a supervised manner, using the loss function Constrains the Euclidean distance between the reconstructed point cloud and the ground truth point cloud, where The keypoint contours generated in (a) of step s2 are applied to constrain the physical spatial distribution of the point cloud, applied to densely reconstructed point clouds;
其中:in:
ψ(p)为中p的对应点, ψ(p) is The corresponding point of p in ,
优选的,步骤s6和s8中,对点云关键帧的全局描述特征进行NN匹配是通过连续N个当前点云帧与连续N个历史点云帧的序列匹配进行判定,如果连续序列匹配关键帧间的全局描述特征的欧式距离小于设定阈值且与其他历史关键帧特征距离相比梯度变化大于设定阈值,则可视为闭环检测候选序列关键帧或重定位候选位置;如果由多个机器人提供当前连续点云帧和历史关键帧,则为多机器人协同建图;通过所述的点云特征空间表征方法在点云的特征空间进行激光SLAM闭环任务,重定位任务和多机器人协同任务。Preferably, in steps s6 and s8, the NN matching of the global description feature of the point cloud key frame is determined by matching the sequence of N consecutive current point cloud frames with N consecutive historical point cloud frames. If the consecutive sequence matches the key frame If the Euclidean distance between the global description features is less than the set threshold and the gradient change compared with other historical keyframe feature distances is greater than the set threshold, it can be regarded as a closed-loop detection candidate sequence keyframe or relocation candidate position; Provide the current continuous point cloud frame and historical key frame, then it is multi-robot collaborative mapping; laser SLAM closed-loop task, relocation task and multi-robot collaborative task are carried out in the feature space of point cloud through the point cloud feature space representation method.
优选的,步骤s7中,利用s4所述编码器网络对激光点云地图中的关键帧提取全局描述特征作为压缩编码进行存储或通信传输,再利用s5所述解码器网络对存储特征或传输接收特征进行还原,重建原始激光点云地图,所述的点云特征空间表征方法可应用于嵌入式系统或低带宽情况。Preferably, in step s7, the encoder network described in s4 is used to extract the global description features of the key frames in the laser point cloud map as compression codes for storage or communication transmission, and then the decoder network described in s5 is used to store or transmit and receive the features. The features are restored, and the original laser point cloud map is reconstructed. The described point cloud feature space representation method can be applied to embedded systems or low bandwidth situations.
本发明的有益效果为:(1)设计了编码器网络对大场景激光点云数据进行了几何结构特征与语义上下文特征的信息提取,充分挖掘了点云场景信息,生成全局描述特征构成点云的特征空间并以此特征空间中的距离给出场景的相似性度量,用于判断两个或多个场景结构是否相似,在点云特征空间即可实现激光SLAM的闭环检测与重定位;(2)通过所设计的解码器网络对点云进行重建,从编码器网络提取的全局描述特征重建原始点云,实现点云地图的压缩存储与低带宽传输;(3)所构建的编码器网路不需要提前根据点云地图进行训练,具有强大的泛化能力。The beneficial effects of the present invention are as follows: (1) An encoder network is designed to extract the information of geometric structure features and semantic context features for the laser point cloud data of large scenes, fully excavate the scene information of the point cloud, and generate the global description feature to form the point cloud and the distance in this feature space gives the similarity measure of the scene, which is used to judge whether two or more scene structures are similar, and the closed-loop detection and relocation of laser SLAM can be realized in the point cloud feature space; ( 2) Reconstruct the point cloud through the designed decoder network, reconstruct the original point cloud from the global description features extracted from the encoder network, and realize the compressed storage and low-bandwidth transmission of the point cloud map; (3) The constructed encoder network Road does not need to be trained according to the point cloud map in advance, and has a strong generalization ability.
附图说明Description of drawings
图1为本发明的方法流程示意图。FIG. 1 is a schematic flow chart of the method of the present invention.
图2为本发明的编码器网络模型结构示意图。FIG. 2 is a schematic structural diagram of an encoder network model of the present invention.
图3为本发明的解码器网络模型结构示意图。FIG. 3 is a schematic structural diagram of a decoder network model of the present invention.
具体实施方式Detailed ways
如图1所示,一种用于激光SLAM的点云特征空间表征方法,包括如下步骤:As shown in Figure 1, a point cloud feature space representation method for laser SLAM includes the following steps:
步骤1:对点云数据集中点云样本进行滤波、降采样等预处理,并根据点云场景的相似性构建训练点云样本对,包括正样本ppos与负样本pneg。Step 1: Perform preprocessing such as filtering and downsampling on the point cloud samples in the point cloud data set, and construct training point cloud sample pairs according to the similarity of the point cloud scene, including the positive sample p pos and the negative sample p neg .
构建训练样本对的相似性由样本在地图中的位置坐标辅助判断决定,位置间隔在10m以内视为结构相似的正样本,距离10米到50米的视为负样本选取范围,随机选取负样本,距离50米以外的随机选取样本视为极不相似的极负样本。The similarity of the constructed training sample pair is determined by the auxiliary judgment of the position coordinates of the samples in the map. The position interval within 10m is regarded as a positive sample with similar structure, and the distance between 10m and 50m is regarded as a negative sample selection range, and a negative sample is randomly selected. , and randomly selected samples at a distance of 50 meters are regarded as extremely dissimilar negative samples.
具体为:选用Oxford RobotCar数据集,使用直通滤波器将点云范围设置为[x:±30m,y:±20m,z:0-10m],并使用随机体素滤波器将点云降采样到4096个点,并归一化至[-1~1]。Specifically: select the Oxford RobotCar dataset, use a pass-through filter to set the point cloud range to [x:±30m, y:±20m, z:0-10m], and use a random voxel filter to downsample the point cloud to 4096 points and normalized to [-1~1].
步骤2:如图2所示的网络结构设计来构建编码器网络,输入步骤1的训练点云样本对,通过点云自适应邻域特征提取模块,动态图网络聚合模块,全局描述特征生成模块得到输入点云P的全局描述特征f(P)。Step 2: The network structure shown in Figure 2 is designed to construct the encoder network, input the training point cloud sample pair of
编码器网络的点云自适应邻域特征提取模块以范围[kmin,kmax]构建点云中每个点pi的kNN邻域利用邻域点云的协方差矩阵Σ的特征值λ1≥λ2≥λ3≥0计算点云邻域的线性平面性和散射性特征,并通过香农信息熵度量方程Ek自适应的确定最优邻域大小通过MLP(多层感知器)特征提取网络提取邻域特征作为点的特征。The point cloud adaptive neighborhood feature extraction module of the encoder network builds a kNN neighborhood of each point pi in the point cloud in the range [ k min ,k max ] Using the eigenvalue λ 1 ≥λ 2 ≥λ 3 ≥0 of the covariance matrix Σ of the neighborhood point cloud to calculate the linearity of the point cloud neighborhood flatness and scattering feature, and adaptively determine the optimal neighborhood size through the Shannon information entropy metric equation E k Neighborhood features are extracted as point features through an MLP (Multilayer Perceptron) feature extraction network.
其中:in:
V为特征向量矩阵 V is the eigenvector matrix
Ek=-LλlnLλ-PλlnPλ-SλlnSλ E k = -L λ lnL λ -P λ lnP λ -S λ lnS λ
具体为:设置点云自适应邻域特征范围[20-100],步长为10,计算确定最优邻域大小。Specifically: set the point cloud adaptive neighborhood feature range [20-100], the step size is 10, and calculate to determine the optimal neighborhood size.
动态图网络聚合模块由两部分聚合组成:The dynamic graph network aggregation module consists of two parts:
(a)特征空间动态图网络聚合:在权利要求3所述的特征空间的所有点进行固定k的kNN构建动态图,并通过图网络进行特征聚合,更新点的特征;(a) Dynamic graph network aggregation in feature space: kNN with fixed k is constructed at all points in the feature space of
(b)物理空间动态图网络聚合:在点云物理空间的所有点进行固定k的kNN构建动态图,利用上述(a)更新后的点的特征通过图网络进行特征更新,更新点的特征。中心点pi构建的动态图G=(V,E)有节点V和边E组成,点pi的特征为fi,节点V为邻域点的特征fj,边E为中心点与邻域点特征的关系ε(fi,fj)。(b) Physical space dynamic graph network aggregation: A kNN with a fixed k is used to construct a dynamic graph at all points in the physical space of the point cloud, and the features of the updated points (a) above are used to update the features through the graph network to update the features of the points. The dynamic graph G=(V, E) constructed by the center point pi is composed of node V and edge E. The feature of point pi is f i , the node V is the feature f j of the neighborhood point, and the edge E is the center point and the neighbor point. The relation ε(f i ,f j ) of the domain point features.
其中:ε(fi,fj)=fi‖(fj-fi),‖为特征按通道拼接;Among them: ε(f i , f j )=fi ‖(f j -f i ) , ‖ is the feature splicing by channel;
图网络的特征聚合过程为:The feature aggregation process of the graph network is:
fi'=ρ(φΘ(ε(fi,fj))=ρ(φΘ(fi‖(fj-fi))),φΘ为MLP(多层感知器),ρ为最大池化。f i '=ρ(φ Θ (ε(fi , f j ) )=ρ(φ Θ (fi ‖(f j -f i ) )), φ Θ is MLP (multi-layer perceptron), and ρ is max pooling.
具体为:动态图构建使用固定k=20的kNN,φΘ为两层(64,64)的MLPSpecifically: kNN with fixed k=20 is used for dynamic graph construction, and φ Θ is an MLP with two layers (64,64).
优选地,全局描述特征生成模块通过NetVLAD Layer生成全局描述特征。Preferably, the global description feature generation module generates the global description feature through the NetVLAD Layer.
具体为:NetVLAD Layer选取中心特征维度D=256,共选取K=64个中心特征。Specifically: NetVLAD Layer selects the central feature dimension D=256, and selects K=64 central features in total.
全局描述特征生成模块通过NetVLAD Layer生成全局描述特征。The global description feature generation module generates global description features through the NetVLAD Layer.
步骤3:如图3所示的网络结构设计来构建解码器网络,输入步骤2中生成的全局描述特征,通过层次化生成策略,由生成关键点轮廓到生成精细化稠密化重建点云 Step 3: The network structure design shown in Figure 3 is used to construct the decoder network, input the global description features generated in step 2, and through the hierarchical generation strategy, from generating key point contours to generating refined and dense reconstructed point clouds
层次化生成策略包括两部分:The hierarchical generation strategy consists of two parts:
(a)关键点轮廓生成网络:通过三层全连接网络和Reshape生成粗糙的个点的关键点轮廓;(a) Keypoint contour generation network: Generate rough keypoint contours of points through a three-layer fully connected network and Reshape;
(b)精细化稠密点云生成网络:在上述(a)生成的关键点中每个点拼接η×η个点的2D平面网格并按通道拼接所述步骤2中生成的全局描述特征,通过MLP(多层感知器)生成稠密重建点云。(b) Refinement of dense point cloud generation network: a 2D planar grid of η × η points is stitched at each point in the key points generated in (a) above And the global description features generated in step 2 are spliced by channel, and the dense reconstructed point cloud is generated by MLP (multi-layer perceptron).
其中:in:
η为平面网格边上点的数量,ξ为平面网格点的尺寸。 η is the number of points on the edge of the plane grid, and ξ is the size of the plane grid points.
具体为:关键点轮廓生成网络生成包含4096个关键点的粗糙轮廓,并且拼接2×2个点的平面网络,尺寸ξ=0.05。Specifically: the key point contour generation network generates a rough contour containing 4096 key points, and splices a 2×2 point plane network with a size of ξ=0.05.
步骤4:取步骤1中的训练数据集的样本对,对步骤2的编码器网络进行训练,得到可以度量点云场景相似性的编码器网络模型和参数。Step 4: Take the sample pair of the training data set in
使用度量学习方法训练编码器网络,利用四元组损失函数半监督训练使得输入样本与正样本ppos之间全局描述特征的欧式距离δpos更近,与负样本pneg之间的全局描述特征的欧氏距离δneg更远,极负样本对用于防止负样本对之间的全局描述特征距离错误的拉近,极负样本与负样本的欧式距离为 Train an encoder network using metric learning methods, utilizing a quadruple loss function Semi-supervised training makes the Euclidean distance δ pos of the global description feature between the input sample and the positive sample p pos closer, and the Euclidean distance δ neg of the global description feature between the negative sample p neg is farther, and the extremely negative sample To prevent the error of the global description feature distance between pairs of negative samples, the Euclidean distance between extremely negative samples and negative samples is
其中:in:
α,β为超参数。 α and β are hyperparameters.
具体为:设置超参数α=0.5,β=0.2,训练时选取2个正样本,18个负样本和1一个随机选取的极负样本。Specifically: set the hyperparameters α=0.5, β=0.2, select 2 positive samples, 18 negative samples and 1 randomly selected extremely negative sample during training.
步骤5:取步骤1中的训练数据集的样本对和通过步骤2编码器网络生成的对应全局描述特征,对步骤3的解码器网络进行训练,得到可以从全局描述特征恢复重建原始点云的解码器网络模型和参数。Step 5: Take the sample pair of the training data set in
步骤5中,有监督的方式训练解码器网络,利用损失函数约束重建点云与真值点云的欧氏距离,其中应用在生成的关键点轮廓,用于约束点云的物理空间分布,应用于稠密重建点云。In step 5, the decoder network is trained in a supervised manner, using the loss function Constrains the Euclidean distance between the reconstructed point cloud and the ground truth point cloud, where Applied to the generated keypoint contours to constrain the physical spatial distribution of the point cloud, Applied to densely reconstructed point clouds.
其中:in:
ψ(p)为中p的对应点 ψ(p) is The corresponding point of p in
步骤6:在实际SLAM作业中,取步骤4中的编码器网络模型和参数,对点云关键帧提取全局描述特征,并与历史关键帧的全局描述特征根据欧式距离进行NN匹配,符合匹配条件的作为闭环候选帧或重定位候选位置。Step 6: In the actual SLAM job, take the encoder network model and parameters in step 4, extract the global description feature for the point cloud key frame, and perform NN matching with the global description feature of the historical key frame according to the Euclidean distance, which meets the matching conditions is used as a closed-loop candidate frame or a relocation candidate position.
步骤6中,对点云关键帧的全局描述特征进行NN匹配是通过连续N个当前点云帧与连续N个历史点云帧的序列匹配进行判定,如果连续序列匹配关键帧间的全局描述特征的欧式距离小于设定阈值且与其他历史关键帧特征距离相比梯度变化大于设定阈值,则可视为闭环检测候选序列关键帧或重定位候选位置。通过所述的点云特征空间表征方法在点云的特征空间进行激光SLAM闭环任务和重定位任务。In step 6, the NN matching of the global description features of the point cloud key frames is determined by the sequence matching of N consecutive current point cloud frames and consecutive N consecutive historical point cloud frames, if the consecutive sequences match the global description features between key frames. The Euclidean distance of is less than the set threshold and the gradient change compared with other historical key frame feature distances is greater than the set threshold, it can be regarded as a closed loop detection candidate sequence key frame or relocation candidate position. The laser SLAM closed-loop task and relocation task are performed in the feature space of the point cloud by the described point cloud feature space characterization method.
步骤7:在实际SLAM作业中,对步骤6提取的点云关键帧全局描述特征进行存储或传输,通过步骤5中的解码器网络模型和参数可以将全局描述特征恢复重建原始SLAM点云地图。Step 7: In the actual SLAM operation, store or transmit the global description features of the point cloud keyframes extracted in Step 6. The global description features can be restored and reconstructed to the original SLAM point cloud map through the decoder network model and parameters in Step 5.
利用步骤4所述编码器网络对激光点云地图中的关键帧提取全局描述特征作为压缩编码进行存储或通信传输,再利用步骤5所述解码器网络对存储特征或传输接收特征进行还原,重建原始激光点云地图,所述的点云特征空间表征方法可应用于嵌入式系统或低带宽情况。Use the encoder network described in step 4 to extract the global description features from the key frames in the laser point cloud map as compression codes for storage or communication transmission, and then use the decoder network described in step 5 to restore the storage features or transmission and reception features, reconstruct Raw laser point cloud map, the described point cloud feature space characterization method can be applied to embedded systems or low bandwidth situations.
步骤8:在实际SLAM作业中,取s6中提取的全局描述特征,并与多个协同建图机器人的历史关键帧的全局描述特征根据欧氏距离进行NN匹配,通过所述的点云特征空间表征方法在点云特征空间进行多机器人激光SLAM协同建图任务。Step 8: In the actual SLAM job, take the global description features extracted in s6, and perform NN matching with the global description features of the historical key frames of multiple collaborative mapping robots according to the Euclidean distance, through the point cloud feature space described The characterization method performs a multi-robot laser SLAM collaborative mapping task in the point cloud feature space.
步骤8中,如果由多个机器人提供当前连续点云帧和历史关键帧,则为多机器人协同建图,对所有的点云历史关键帧的全局描述特征进行NN匹配,则可实现多机器人协同建图。通过所述的点云特征空间表征方法在点云的特征空间进行激光SLAM多机器人协同任务。In step 8, if multiple robots provide the current continuous point cloud frame and historical key frames, the multi-robot collaborative mapping is performed, and NN matching is performed on the global description features of all point cloud historical key frames, so that multi-robot collaboration can be realized. Build a map. The laser SLAM multi-robot collaborative task is carried out in the feature space of the point cloud through the described point cloud feature space characterization method.
本发明的有益效果为:(1)设计了编码器网络对大场景激光点云数据进行了几何结构特征与语义上下文特征的信息提取,该点云特征空间表征方法通过特征检索的方式实现场景识别,在Oxford RobotCar数据集场景识别任务中平均召回率达到93.35%;(2)通过所设计的解码器网络对点云进行重建,从编码器网络提取的全局描述特征重建原始点云,实现点云地图的压缩存储与低带宽传输;(3)所构建的编码器网路不需要提前根据点云地图进行训练,具有强大的泛化能力,在KITTI等多种数据集中可以无需训练直接应用。The beneficial effects of the invention are as follows: (1) An encoder network is designed to extract information of geometric structure features and semantic context features for laser point cloud data in large scenes, and the point cloud feature space representation method realizes scene recognition by means of feature retrieval. , in the scene recognition task of the Oxford RobotCar dataset, the average recall rate reaches 93.35%; (2) The point cloud is reconstructed through the designed decoder network, and the original point cloud is reconstructed from the global description features extracted by the encoder network to realize the point cloud. Map compression storage and low-bandwidth transmission; (3) The constructed encoder network does not need to be trained according to the point cloud map in advance, and has a strong generalization ability, which can be directly applied in various data sets such as KITTI without training.
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010504793.6A CN111783838A (en) | 2020-06-05 | 2020-06-05 | A point cloud feature space representation method for laser SLAM |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010504793.6A CN111783838A (en) | 2020-06-05 | 2020-06-05 | A point cloud feature space representation method for laser SLAM |
Publications (1)
Publication Number | Publication Date |
---|---|
CN111783838A true CN111783838A (en) | 2020-10-16 |
Family
ID=72754632
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010504793.6A Pending CN111783838A (en) | 2020-06-05 | 2020-06-05 | A point cloud feature space representation method for laser SLAM |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111783838A (en) |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112581515A (en) * | 2020-11-13 | 2021-03-30 | 上海交通大学 | Outdoor scene point cloud registration method based on graph neural network |
CN113139996A (en) * | 2021-05-06 | 2021-07-20 | 南京大学 | Point cloud registration method and system based on three-dimensional point cloud geometric feature learning |
CN113219493A (en) * | 2021-04-26 | 2021-08-06 | 中山大学 | End-to-end point cloud data compression method based on three-dimensional laser radar sensor |
CN113343765A (en) * | 2021-05-11 | 2021-09-03 | 武汉大学 | Scene retrieval method and system based on point cloud rigid registration |
CN113486963A (en) * | 2021-07-12 | 2021-10-08 | 厦门大学 | Density self-adaptive point cloud end-to-end sampling method |
CN113538675A (en) * | 2021-06-30 | 2021-10-22 | 同济人工智能研究院(苏州)有限公司 | Neural network for calculating attention weight for laser point cloud and training method |
CN113674416A (en) * | 2021-08-26 | 2021-11-19 | 中国电子科技集团公司信息科学研究院 | Three-dimensional map construction method and device, electronic equipment and storage medium |
CN113808268A (en) * | 2021-09-17 | 2021-12-17 | 浙江大学 | A low-bandwidth globally consistent multi-machine dense mapping method |
CN113810736A (en) * | 2021-08-26 | 2021-12-17 | 北京邮电大学 | AI-driven real-time point cloud video transmission method and system |
CN113807184A (en) * | 2021-08-17 | 2021-12-17 | 北京百度网讯科技有限公司 | Obstacle detection method and device, electronic equipment and automatic driving vehicle |
CN114612660A (en) * | 2022-03-01 | 2022-06-10 | 浙江工业大学 | Three-dimensional modeling method based on multi-feature fusion point cloud segmentation |
CN114998699A (en) * | 2022-06-09 | 2022-09-02 | 北京大学深圳研究生院 | Distortion recovery method, device and electronic device for compressed point cloud |
CN115578426A (en) * | 2022-10-25 | 2023-01-06 | 哈尔滨工业大学 | Indoor service robot repositioning method based on dense feature matching |
CN115950414A (en) * | 2023-01-29 | 2023-04-11 | 北京工商大学 | Adaptive multi-fusion SLAM method for different sensor data |
CN116222577A (en) * | 2023-04-27 | 2023-06-06 | 苏州浪潮智能科技有限公司 | Closed loop detection method, training method, system, electronic equipment and storage medium |
CN117557904A (en) * | 2023-11-14 | 2024-02-13 | 国网电力空间技术有限公司 | Multi-model-fused power transmission channel construction machinery identification method |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102592281A (en) * | 2012-01-16 | 2012-07-18 | 北方工业大学 | Image matching method |
CN110243370A (en) * | 2019-05-16 | 2019-09-17 | 西安理工大学 | A 3D Semantic Map Construction Method for Indoor Environment Based on Deep Learning |
-
2020
- 2020-06-05 CN CN202010504793.6A patent/CN111783838A/en active Pending
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102592281A (en) * | 2012-01-16 | 2012-07-18 | 北方工业大学 | Image matching method |
CN110243370A (en) * | 2019-05-16 | 2019-09-17 | 西安理工大学 | A 3D Semantic Map Construction Method for Indoor Environment Based on Deep Learning |
Non-Patent Citations (1)
Title |
---|
CHUANZHE SUO等: "LPD-AE: Latent Space Representation of Large-Scale 3D Point Cloud", 《IEEE ACCESS》 * |
Cited By (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112581515B (en) * | 2020-11-13 | 2022-12-13 | 上海交通大学 | Point cloud registration method for outdoor scenes based on graph neural network |
CN112581515A (en) * | 2020-11-13 | 2021-03-30 | 上海交通大学 | Outdoor scene point cloud registration method based on graph neural network |
CN113219493A (en) * | 2021-04-26 | 2021-08-06 | 中山大学 | End-to-end point cloud data compression method based on three-dimensional laser radar sensor |
CN113219493B (en) * | 2021-04-26 | 2023-08-25 | 中山大学 | An end-to-end point cloud data compression method based on 3D lidar sensor |
CN113139996A (en) * | 2021-05-06 | 2021-07-20 | 南京大学 | Point cloud registration method and system based on three-dimensional point cloud geometric feature learning |
CN113139996B (en) * | 2021-05-06 | 2024-02-06 | 南京大学 | Point cloud registration method and system based on three-dimensional point cloud geometric feature learning |
CN113343765B (en) * | 2021-05-11 | 2022-07-22 | 武汉大学 | Scene retrieval method and system based on point cloud rigid registration |
CN113343765A (en) * | 2021-05-11 | 2021-09-03 | 武汉大学 | Scene retrieval method and system based on point cloud rigid registration |
CN113538675A (en) * | 2021-06-30 | 2021-10-22 | 同济人工智能研究院(苏州)有限公司 | Neural network for calculating attention weight for laser point cloud and training method |
CN113486963A (en) * | 2021-07-12 | 2021-10-08 | 厦门大学 | Density self-adaptive point cloud end-to-end sampling method |
CN113486963B (en) * | 2021-07-12 | 2023-07-07 | 厦门大学 | A Density-Adaptive Point Cloud-to-End Sampling Method |
CN113807184A (en) * | 2021-08-17 | 2021-12-17 | 北京百度网讯科技有限公司 | Obstacle detection method and device, electronic equipment and automatic driving vehicle |
CN113810736A (en) * | 2021-08-26 | 2021-12-17 | 北京邮电大学 | AI-driven real-time point cloud video transmission method and system |
CN113674416B (en) * | 2021-08-26 | 2024-04-26 | 中国电子科技集团公司信息科学研究院 | Three-dimensional map construction method and device, electronic equipment and storage medium |
CN113674416A (en) * | 2021-08-26 | 2021-11-19 | 中国电子科技集团公司信息科学研究院 | Three-dimensional map construction method and device, electronic equipment and storage medium |
CN113808268A (en) * | 2021-09-17 | 2021-12-17 | 浙江大学 | A low-bandwidth globally consistent multi-machine dense mapping method |
CN113808268B (en) * | 2021-09-17 | 2023-09-26 | 浙江大学 | A low-bandwidth globally consistent multi-machine dense mapping method |
CN114612660A (en) * | 2022-03-01 | 2022-06-10 | 浙江工业大学 | Three-dimensional modeling method based on multi-feature fusion point cloud segmentation |
CN114612660B (en) * | 2022-03-01 | 2025-06-13 | 浙江工业大学 | A 3D modeling method based on multi-feature fusion point cloud segmentation |
CN114998699A (en) * | 2022-06-09 | 2022-09-02 | 北京大学深圳研究生院 | Distortion recovery method, device and electronic device for compressed point cloud |
CN115578426B (en) * | 2022-10-25 | 2023-08-18 | 哈尔滨工业大学 | Indoor service robot repositioning method based on dense feature matching |
CN115578426A (en) * | 2022-10-25 | 2023-01-06 | 哈尔滨工业大学 | Indoor service robot repositioning method based on dense feature matching |
CN115950414A (en) * | 2023-01-29 | 2023-04-11 | 北京工商大学 | Adaptive multi-fusion SLAM method for different sensor data |
CN116222577A (en) * | 2023-04-27 | 2023-06-06 | 苏州浪潮智能科技有限公司 | Closed loop detection method, training method, system, electronic equipment and storage medium |
CN117557904A (en) * | 2023-11-14 | 2024-02-13 | 国网电力空间技术有限公司 | Multi-model-fused power transmission channel construction machinery identification method |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111783838A (en) | A point cloud feature space representation method for laser SLAM | |
Vidanapathirana et al. | Locus: Lidar-based place recognition using spatiotemporal higher-order pooling | |
CN109597087B (en) | A 3D object detection method based on point cloud data | |
CN112529015B (en) | Three-dimensional point cloud processing method, device and equipment based on geometric unwrapping | |
CN113139996B (en) | Point cloud registration method and system based on three-dimensional point cloud geometric feature learning | |
Duan et al. | Deep learning for visual SLAM in transportation robotics: A review | |
Murali et al. | Utilizing semantic visual landmarks for precise vehicle navigation | |
CN106910202B (en) | Image segmentation method and system for ground object of remote sensing image | |
CN113536920A (en) | A semi-supervised 3D point cloud object detection method | |
Du et al. | ResDLPS-Net: Joint residual-dense optimization for large-scale point cloud semantic segmentation | |
CN111739051A (en) | A Multi-Sequence MRI Image Segmentation Method Based on Residual Network | |
CN112528808B (en) | Method and device for identifying obstacles on celestial body surface | |
CN111797836A (en) | A deep learning-based obstacle segmentation method for extraterrestrial celestial rovers | |
KR102615412B1 (en) | Apparatus and method for performing visual localization | |
CN116772820A (en) | Local refinement mapping system and method based on SLAM and semantic segmentation | |
Contreras et al. | O-poco: Online point cloud compression mapping for visual odometry and slam | |
CN109785409B (en) | Image-text data fusion method and system based on attention mechanism | |
CN115311538A (en) | A method for agent target search based on scene prior | |
CN103810747A (en) | Three-dimensional point cloud object shape similarity comparing method based on two-dimensional mainstream shape | |
CN116222577A (en) | Closed loop detection method, training method, system, electronic equipment and storage medium | |
CN114463614B (en) | Salient Object Detection Using Hierarchical Saliency Modeling with Generative Parameters | |
Wang et al. | Deep learning-based semantic segmentation and surface reconstruction for point clouds of offshore oil production equipment | |
Jia et al. | DispNet based stereo matching for planetary scene depth estimation using remote sensing images | |
CN114663600A (en) | Point cloud reconstruction method and system based on self-encoder | |
CN107529647A (en) | A kind of cloud atlas cloud amount computational methods based on the unsupervised sparse learning network of multilayer |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201016 |