WO2024120269A1 - Position recognition method for fusing point cloud map, motion model and local feature - Google Patents

Position recognition method for fusing point cloud map, motion model and local feature Download PDF

Info

Publication number
WO2024120269A1
WO2024120269A1 PCT/CN2023/134918 CN2023134918W WO2024120269A1 WO 2024120269 A1 WO2024120269 A1 WO 2024120269A1 CN 2023134918 W CN2023134918 W CN 2023134918W WO 2024120269 A1 WO2024120269 A1 WO 2024120269A1
Authority
WO
WIPO (PCT)
Prior art keywords
map
descriptor
point cloud
motion model
point
Prior art date
Application number
PCT/CN2023/134918
Other languages
French (fr)
Chinese (zh)
Inventor
张永军
史鹏程
李加元
Original Assignee
武汉大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 武汉大学 filed Critical 武汉大学
Publication of WO2024120269A1 publication Critical patent/WO2024120269A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/29Geographical information databases
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/246Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10028Range image; Depth image; 3D point clouds
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02TCLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00Road transport of goods or passengers
    • Y02T10/10Internal combustion engine [ICE] based vehicles
    • Y02T10/40Engine management systems

Definitions

  • the present invention belongs to the field of surveying, mapping, remote sensing and unmanned driving technology, and specifically relates to a location recognition method that integrates point cloud maps, motion models and local features.
  • SLAM Simultaneous Localization and Mapping
  • Position recognition is a technology that helps robots determine whether to revisit historical locations, and it has a strong correlation with the SLAM system.
  • the classic SLAM framework includes: front-end odometer, back-end optimization, loop detection and back-end optimization, among which the tasks of position recognition and loop detection modules are the same. When the loop is effectively identified, the system passes the matching constraints to the back-end optimization module to weaken the trajectory drift. Quickly and accurately identifying revisited locations and constructing loop constraints are very helpful in improving positioning accuracy and maintaining system stability.
  • visual location recognition schemes usually use word models (BOW, Bag of Words) to encode image data into dictionaries, and design various feature descriptors combined with dictionaries for retrieval and matching.
  • BOW Bag of Words
  • the matching error increases significantly and the location recognition performance drops sharply.
  • Location recognition schemes based on three-dimensional point cloud data pay more attention to spatial geometric information and can overcome the influence of factors such as lighting changes and small field of view.
  • common point cloud-based location recognition schemes include: local descriptor-based, global descriptor-based, semantic information-based, deep learning-based, and artificial landmark-based methods.
  • Local descriptors perform feature statistics on the neighborhood space of key points according to specific rules, but are limited by the poor repeatability of feature points and are not effective in high-speed autonomous driving scenarios.
  • Global descriptors directly encode the entire point cloud as an environmental description, overcoming the problem of feature instability, but ignoring the relationship between features.
  • Semantic features elevate data association from the traditional pixel level to the object level, which can enhance the robot's understanding of the surrounding environment.
  • this method has low data processing efficiency and requires complex operations to infer the relationship between targets.
  • Deep learning methods strengthen feature learning by training neural networks to generate deep expressions of image features. Although it improves the efficiency and accuracy of location recognition, These features are complex and difficult to understand, and model training requires tedious data cleaning.
  • the model training results depend to a certain extent on the diversity of samples, and the differences in scenes may affect the recognition effect.
  • Artificial landmarks are usually deployed in advance in specific scenes. Although the cost is low, they are easily ineffective in dynamic scenes due to obstruction of vision.
  • the present invention proposes a novel location recognition method integrating point cloud maps, motion models and local features.
  • the proposed method only needs to use multi-line laser radar as the only measurement sensor, effectively overcomes the influence of illumination changes, and is more stable than the visual solution.
  • the invention innovatively incorporates the motion model to effectively solve the problem of perceptual confusion in location recognition, and can still effectively detect the loop position when the environment is highly similar.
  • Combining the point cloud map with the local features significantly reduces the search space, and can still achieve superior operating efficiency under limited computing resources, improve the fault tolerance of the system, and ensure that other functional modules are carried out in an orderly manner.
  • the method can overcome the problem of target occlusion or the presence of dynamic targets, and improve the safety performance of the system in dynamic cities.
  • the method can still maintain good recognition accuracy and efficiency when the trajectory continues to increase and the data continues to accumulate.
  • As a novel location recognition solution it can provide recognition results quickly and accurately, and improve the positioning accuracy of the system.
  • the present invention proposes a position recognition method that integrates point cloud map, motion model and local features.
  • data is collected in the unmanned driving scene in advance, and the relative posture is calculated and the point cloud map is spliced through high-precision combined navigation or matching algorithm, and reference points are generated inside the map as virtual landmarks.
  • a bird's-eye view feature description set with different orientations is constructed in the map to simulate the position and orientation of the vehicle, and the map-related data is saved offline.
  • the vehicle trajectory is initialized, and the vehicle position is estimated by global search and local search in the map description set for the first frame and the second frame of data, respectively.
  • the Kalman filter algorithm is used to estimate the current vehicle position, and further determine whether there is a loop.
  • the present invention designs a location recognition method integrating point cloud map, motion model and local features, which mainly includes the following steps:
  • Step 1 For the multi-frame lidar point cloud recorded offline, the relative pose is calculated based on high-precision integrated navigation, the scene prior point cloud map is stitched, and reference points are generated inside the map as virtual landmarks.
  • Step 2 traverse the map reference points, combine with the point cloud map, construct the feature description of the bird's-eye view under different directions, generate a map description set, and save the map-related data offline.
  • Step 3 For the point cloud of the first frame of the trajectory, a global brute force search is used in the map description set to calculate the best matching map descriptor, and the vehicle position is estimated based on the map reference point.
  • Step 4 For the point cloud of the second frame of the trajectory, with the help of the starting position estimation, a local search is used in the map description set to calculate the best matching map descriptor, and the vehicle position is estimated based on the map reference point.
  • Step 5 For real-time point cloud data, descriptor similarity is used as a reliability measure. With the help of the position estimation of the first two frames, combined with the vehicle motion model and local search, the Kalman filter is used to estimate the current vehicle position.
  • step 1 includes the following sub-steps:
  • Step 1.1 Use an autonomous driving vehicle equipped with a multi-line laser radar to collect data in advance inside the scene.
  • Use combined navigation such as the global positioning system (GPS) and the inertial navigation system (INS) to calculate high-precision pose and stitch point cloud maps. If there is no combined navigation device, a laser odometer matching algorithm can be used instead.
  • GPS global positioning system
  • INS inertial navigation system
  • Step 1.2 use two height thresholds z 1 and z 2 to filter out some point clouds in the map based on the sensor height, downsample the map using voxels with a side length of 1 m , and project it into the XOY plane.
  • the coordinate system of the laser radar is in the direction of travel with the X axis pointing to the left, and the Z axis pointing vertically upward. If it is installed in other directions, adjust the threshold accordingly.
  • Step 1.3 calculate the two-dimensional coordinate extreme values (x min , y min , x max , y max ) of the projected map, generate reference points Mr in the map as virtual landmarks according to equidistant sampling, and use point cloud processing software such as CloudCompare to crop some unreasonable reference points.
  • the reference point coordinates are calculated as follows:
  • i and j are the row and column numbers of the sampling points
  • dx and dy are the sampling distances
  • (x, y) are the coordinates of the sampling points.
  • step 2 includes the following sub-steps:
  • ⁇ ' is the angle resolution
  • i is the orientation angle number
  • N + represents a positive integer
  • Step 2.2 for the map reference point p k (x k ,y k ) ⁇ M r , where k is the map point index.
  • Convert the descriptor into a vector f with n r ⁇ n c rows and 1 column, and concatenate all descriptor vectors into a map feature matrix F [f 1 ,f 2 ,...,f k ].
  • the specific calculation method of the feature descriptor is as follows:
  • j is the neighborhood point index
  • r and c are the descriptor row and column indices
  • d max is the maximum distance
  • d' is the distance resolution
  • ⁇ ' is the angle resolution
  • Step 2.3 save the map-related data as an offline binary file, which includes: (1) map reference point cloud; (2) map reference point KD tree; (3) map feature matrix.
  • the map feature matrix file records the global index of each occupied element, as follows:
  • id m , id ⁇ and id f are the map reference point, orientation angle and index of the descriptor vector element respectively.
  • s is the data recorded offline
  • n ⁇ is the number of angles
  • n f n r ⁇
  • n c is the number of descriptor elements.
  • mod and rem represent modulus and remainder operations respectively.
  • step 3 includes the following sub-steps:
  • Step 3.1 after starting the real-time system, only one offline map file needs to be loaded.
  • For the first frame of the lidar point cloud perform height filtering, where the height retention range should be as consistent as possible with the map value range.
  • Step 3.2 using the map descriptor as a reference, calculate the hit rate of the real-time descriptor in the map descriptor as the evaluation function.
  • the evaluation function L is as follows:
  • n(1,0) and n(1,1) represent the number of elements in fs and fm that are (1,0) and (0,1), respectively.
  • Step 3.3 based on the best matching vector
  • the index and map reference point Mr are used to estimate the current vehicle position.
  • step 4 includes the following sub-steps:
  • Step 4.1 For the second frame of the laser radar point cloud in the trajectory, use the position estimation of the first frame to search for the nearest neighbor point cloud P in the map reference point KD tree, which contains N k points.
  • j is the map reference point index and i is the map feature description set index.
  • step 4.2 in the local descriptor set, a brute force search is used to calculate the map descriptor that best matches the real-time descriptor, and the position and orientation of the current vehicle are calculated based on the descriptor index.
  • step 5 includes the following sub-steps:
  • Step 5.1 For the real-time LiDAR point cloud after trajectory initialization, use the position estimation of the first two frames and the uniform motion model of the vehicle to predict the vehicle's motion position.
  • the vehicle's motion model is as follows:
  • ⁇ and ⁇ represent the posterior and prior variables respectively.
  • x t+1 represents the position variable at time t+1
  • F( ⁇ ) represents the motion model of the vehicle
  • A is the Jacobian coefficient matrix of error propagation in the motion model
  • V represents the variance matrix
  • ⁇ x is the position estimation error
  • the variance of motion prediction is the variance of motion prediction.
  • Step 5.2 using the position estimate of the previous frame, a local search is performed in the map reference point to find the best matching descriptor in the neighborhood descriptor set, and the map reference point with similar neighborhood is calculated as the observation model:
  • H( ⁇ ) is the observation model, and the present invention uses local search as the observation model.
  • ⁇ s is the observation error.
  • the evaluation function result in formula (5) calculated using the real-time descriptor fs and the map descriptor fm ; in each calculation, the variance ⁇ s of the position estimate is calculated using the evaluation function result and the sampling distances dx , dy .
  • Step 5.3 after determining the motion model and observation model, use Kalman filtering to estimate the current vehicle position.
  • the calculation method is as follows:
  • K is the Kalman gain
  • B is the Jacobian matrix of the error propagation equation of the observation model.
  • step 5 also includes comparing the historical trajectory to determine whether there is a loop. If the distance difference with the historical position is less than a set threshold, it is considered to be a loop, otherwise it is not a loop, thereby achieving position recognition.
  • step 2.1 the map rotation angle resolution is set to 3 degrees, that is, ⁇ is 3.
  • the feature description size of the annular bird's-eye view in step 2.2 is set to 40 rows and 60 columns.
  • step 4.1 the number of nearest neighbors searched in the map reference point is 20.
  • the present invention solves a key problem in the field of autonomous driving and artificial intelligence, namely, position recognition.
  • the present invention proposes a novel position recognition method that integrates point cloud maps, motion models, and local features, calculates the vehicle position within the global map, and effectively solves the problems of perceptual confusion and high recognition error rate in position recognition or loop detection.
  • This invention is different from the general Kalman filter data fusion algorithm. It uses the local search feature descriptor as the observation model, which creates a new idea for the Kalman filter algorithm. At the same time, it only uses the laser radar as a single measurement sensor, which significantly reduces the cost of the unmanned driving system and improves the applicability of the algorithm.
  • the present invention combines the map with local features, significantly reducing the search space and The algorithm can still achieve superior operating efficiency with low computing resources.
  • the motion model is added to the algorithm to effectively overcome the impact of target occlusion or dynamic targets, greatly improving the system's fault tolerance.
  • the position recognition performance will not degrade with the increase of trajectory or running time, which improves the safety performance of the unmanned driving system in dynamic cities.
  • the present invention can effectively realize the position recognition of autonomous driving vehicles in a large-scale, dynamic urban environment.
  • the real-time position calculation rate can reach 200FPS (Frames Per Seond), without relying on other external signals. While reducing the system cost, it significantly improves the efficiency of loop detection in SLAM technology.
  • the invention solves the problem of position recognition in a large-scale scene, and adds the motion model to the position calculation, which can solve the effects of perception confusion, target occlusion, etc.
  • fast position recognition can effectively build matching associations with historical data, improve the overall positioning accuracy of the system, ensure the normal operation of other modules, and effectively improve the fault tolerance of the system.
  • the position recognition performance will not be degraded by the increase of trajectory and running time. Therefore, the method of the present invention is of great significance to the fields of unmanned driving, robots, etc.
  • FIG. 1 is a flow chart of the present invention.
  • FIG. 2 is a schematic diagram of global map reference points in the present invention.
  • FIG. 3 is a schematic diagram of the first frame position estimation of the trajectory in the present invention.
  • FIG. 4 is a schematic diagram of the second frame position estimation of the trajectory in the present invention.
  • FIG. 5 is a schematic diagram of a Kalman filter algorithm taking into account motion model and local search in the present invention.
  • FIG. 6 is a schematic diagram of the calculation result of the revisited position matching in the present invention.
  • a location recognition method integrating a point cloud map, a motion model and local features mainly includes the following steps:
  • Step 1 For the multi-frame lidar point cloud recorded offline, the relative pose is calculated based on high-precision integrated navigation, the scene prior point cloud map is stitched, and reference points are generated inside the map as virtual landmarks.
  • the laser radar point cloud data is collected in advance around the autonomous driving scene, and the vehicle’s posture information is calculated using high-precision integrated navigation.
  • the relative position between adjacent frame point clouds is further calculated.
  • One key frame is selected every 10 frames, and all key frame point clouds are spliced into a point cloud map in the same coordinate system.
  • the coordinate system of the lidar is oriented with the X axis pointing in the forward direction, the Y axis pointing to the left, and the Z axis pointing vertically upward. If it is installed in other directions, just adjust the threshold accordingly.
  • the coordinate extremes of the map projection point cloud are calculated, the sampling distance intervals dx and dy in the x and y directions are set to 1m, the map reference points are generated, and CloudCompare is used to crop some erroneous points.
  • step 1 the specific implementation of step 1 is as follows:
  • Step 1.1 Use an autonomous driving vehicle equipped with a multi-line laser radar to collect data in advance inside the scene.
  • Use combined navigation such as the global positioning system (GPS) and the inertial navigation system (INS) to calculate high-precision pose and stitch point cloud maps. If there is no combined navigation device, a laser odometer matching algorithm can be used instead.
  • GPS global positioning system
  • INS inertial navigation system
  • Step 1.2 use two height thresholds z 1 and z 2 to filter out some point clouds in the map based on the sensor height, downsample the map using voxels with a side length of 1 m , and project it into the XOY plane.
  • the coordinate system of the laser radar is in the direction of travel with the X axis pointing to the left, and the Z axis pointing vertically upward. If it is installed in other directions, adjust the threshold accordingly.
  • Step 1.3 calculate the two-dimensional coordinate extreme values (x min , y min , x max , y max ) of the projected map, generate reference points Mr in the map as virtual landmarks according to equidistant sampling, and use point cloud processing software such as CloudCompare to crop some unreasonable reference points.
  • the reference point coordinates are calculated as follows:
  • i and j are the row and column numbers of the sampling points
  • dx and dy are the sampling distances
  • (x, y) are the coordinates of the sampling points.
  • Step 2 traverse the map reference points, combine with the point cloud map, construct the feature description of the bird's-eye view under different directions, generate a map description set, and save the map-related data offline.
  • a KD tree is constructed in the map reference points, and at each reference point, the map point cloud is rotated with an angle resolution of 3 degrees around the reference point.
  • the green points are map reference points
  • the orange points are point cloud maps.
  • FIG. 2(b) shows a schematic diagram of the feature descriptor of the bird's-eye view.
  • map reference point cloud, map feature matrix and map reference point KD tree are saved as offline files.
  • step 2 is as follows:
  • ⁇ ' is the angle resolution
  • i is the orientation angle number
  • N + represents a positive integer
  • Step 2.2 for the map reference point p k (x k ,y k ) ⁇ M r , p k represents the kth point in the reference point Mr , x k ,y k represent the horizontal and vertical coordinates of p k , where k is the map point index.
  • j is the neighborhood point index
  • r and c are the descriptor row and column indices
  • d max is the maximum distance
  • d' is the distance resolution
  • ⁇ ' is the angle resolution
  • Step 2.3 save the map-related data as an offline binary file, which includes: (1) map reference point cloud; (2) map reference point KD tree; (3) map feature matrix.
  • the map feature matrix file records the global index of each occupied element, as follows:
  • id m , id ⁇ and id f are the map reference point, orientation angle and index of the descriptor vector element respectively.
  • s is the data recorded offline
  • n ⁇ is the number of angles
  • n f n r ⁇
  • n c is the number of descriptor elements. mod and rem They represent modulo and remainder operations respectively.
  • Step 3 is a schematic diagram of the position calculation of the first frame of trajectory data.
  • a global brute force search is used in the map description set to calculate the best matching map descriptor, and the vehicle position is estimated based on the map reference point.
  • the hit rate of the real-time descriptor in the map descriptor is calculated as the evaluation function, and the map descriptor with the best evaluation function is calculated by brute force search in all map descriptors.
  • the vehicle’s position is estimated based on the index of the optimal map descriptor and the map reference point cloud.
  • step 3 is as follows:
  • Step 3.1 after starting the real-time system, only one offline map file needs to be loaded.
  • For the first frame of the lidar point cloud perform height filtering, where the height retention range should be as consistent as possible with the map value range.
  • Step 3.2 using the map descriptor as a reference, calculate the hit rate of the real-time descriptor in the map descriptor as the evaluation function.
  • the evaluation function L is calculated as follows:
  • fs and fm are the real-time and map descriptor vectors, respectively.
  • n(1,0) and n(1,1) represent the number of elements in fs and fm that are (1,0) and (0,1), respectively.
  • Step 3.3 based on the best matching vector
  • the index and map reference point Mr are used to estimate the current vehicle position.
  • Step 4 is a schematic diagram of the position calculation of the second frame data.
  • a local search is used in the map description set to calculate the best matching map.
  • Graph descriptor estimates the vehicle position based on map reference points.
  • the position results of the first frame are used to search for 20 nearest neighbor points in the map reference point cloud, and a local descriptor set is constructed according to the map descriptor index.
  • step 4 is as follows:
  • Step 4.1 for the second frame of the laser radar point cloud in the trajectory, use the position estimation of the first frame to search for the nearest neighbor point cloud P in the map reference point KD tree, which contains N k points.
  • j is the map reference point index and i is the map feature description set index.
  • step 4.2 in the local descriptor set, a brute force search is used to calculate the map descriptor that best matches the real-time descriptor, and the position and orientation of the current vehicle are calculated based on the descriptor index.
  • Step 5 is a schematic diagram of the position calculation of real-time data.
  • descriptor similarity is used as a reliability metric.
  • the Kalman filter is used to estimate the current vehicle position. The historical trajectory is further compared to determine whether there is a loop.
  • the position estimation of the first two frames is used to predict the current vehicle position using a uniform motion model.
  • the position estimation error is set to 1m.
  • a local search similar to the second frame of the trajectory is used to estimate the position of the vehicle.
  • the similarity of feature descriptors is used as a measure of position deviation, and the Kalman filter algorithm is used to estimate the current position, and the historical trajectory is compared to determine whether there is a loop. If the distance difference with the historical position is less than the set threshold, it is considered a loop, otherwise it is not a loop, thereby achieving position recognition.
  • step 5 is as follows:
  • Step 5.1 For the real-time LiDAR point cloud after trajectory initialization, use the position estimation of the first two frames and the uniform motion model of the vehicle to predict the vehicle's motion position.
  • the vehicle's motion model is as follows:
  • ⁇ and ⁇ represent the posterior and prior variables respectively.
  • x t+1 represents the position variable at time t+1
  • F( ⁇ ) represents the motion model of the vehicle
  • A is the Jacobian coefficient matrix of error propagation in the motion model
  • V represents the variance matrix.
  • ⁇ x is the position estimation error, is the variance of motion prediction.
  • Step 5.2 using the position estimate of the previous frame, a local search is performed in the map reference point to find the best matching descriptor in the neighborhood descriptor set, and the map reference point with similar neighborhood is calculated as the observation model:
  • H( ⁇ ) is the observation model, and the present invention uses local search as the observation model.
  • ⁇ s is the observation error.
  • the evaluation function result in formula (5) calculated using the real-time descriptor fs and the map descriptor fm ; in each calculation, the variance ⁇ s of the position estimate is calculated using the evaluation function result and the sampling distances dx , dy ;
  • Step 5.3 after determining the motion model and observation model, use Kalman filtering to estimate the current vehicle position.
  • the calculation method is as follows:
  • K is the Kalman gain
  • B is the Jacobian matrix of the error propagation equation of the observation model.
  • Step 5.4 compare the historical trajectory of the vehicle to determine whether there is a loop.
  • Figure 6 is a schematic diagram of the revisit position matching effect in three dynamic environments.
  • the horizontal black line is the trajectory of the vehicle
  • the vertical gray line is the revisit position matching
  • the vertical black line is the wrong matching result.
  • Three typical outdoor dynamic scenes are selected as experimental scenes: city (DCC 02, #1), university (KAIST 02, #2) and rural (KITTI 05, #3).
  • the poses are all calculated by high-precision combined navigation. Only the lidar point cloud is used to solve the position information in the experiment.
  • the three scenes collect 3151 frames (#1), 4190 frames (#2) and 2761 frames (#3) of point clouds respectively.
  • the distance is set to 8 meters for loop matching.
  • Each data sequence contains a certain amount of revisited positions.
  • the position error within 8m is considered to be the correct position calculation result.
  • Table 1 The experimental results are shown in Table 1.
  • the success rate of position calculation can reach more than 99%, and the average error is about 0.5m, which has good position accuracy.
  • the lightweight descriptor creation takes about 6ms, and the real-time position solution speed can reach 200FPS.
  • the maximum F1 score can reach more than 0.99, which has good position recognition accuracy. This method can provide an efficient and accurate position recognition method for autonomous driving and improve the overall stability of the system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Optical Radar Systems And Details Thereof (AREA)
  • Navigation (AREA)

Abstract

Provided in the present invention is a position recognition method for fusing a point cloud map, a motion model and a local feature. The proposed method overcomes the problem of position recognition in a large-range dynamic environment. The present invention can effectively overcome the effects of changes in illumination, etc., while reducing system costs without the need for an artificial mark or an external signal. In the present invention, a vehicle motion model is innovatively added to position recognition, such that the problem of awareness confusion is effectively solved. Only a multi-line LiDAR is used as the sole measurement sensor and a local search result is used as an observation model, such that a new line of thought is provided for a data fusion scheme based on Kalman filtering. A point cloud map is combined with a local feature, such that a search space is significantly reduced, the excellent operation efficiency can still be achieved with limited computing resources, and the orderly progress of other modules of a system is guaranteed. In addition, the position recognition performance in the present invention is not affected by trajectory length or operation time, such that problems such as target shielding can be overcome.

Description

一种融合点云地图、运动模型和局部特征的位置识别方法A location recognition method integrating point cloud map, motion model and local features 技术领域Technical Field
本发明属于测绘遥感和无人驾驶技术领域,具体涉及一种融合点云地图、运动模型和局部特征的位置识别方法。The present invention belongs to the field of surveying, mapping, remote sensing and unmanned driving technology, and specifically relates to a location recognition method that integrates point cloud maps, motion models and local features.
背景技术Background technique
随着人工智能和传感器技术的快速发展,使得自主机器人和自动驾驶广泛应用于众多领域,如物流配送、刑侦侦察、交通出行和野外救援等。同时定位与建图(SLAM,Simultaneous Localization and Mapping)作为自动驾驶的核心技术,近年来已逐渐成为智能系统相关研究的热点问题。位置识别是一项帮助机器人判断是否重新访问历史位置的技术,与SLAM系统具有很强的关联性。经典的SLAM框架包括:前端里程计、后端优化、回环检测和后端优化,其中位置识别与回环检测模块的任务是相同的。在有效识别出回环时,系统将匹配约束传递给后端优化模块,以此削弱轨迹漂移。快速、准确地识别出重访位置,构建回环约束,十分有利于提高定位精度,维护系统稳定性。With the rapid development of artificial intelligence and sensor technology, autonomous robots and autonomous driving are widely used in many fields, such as logistics distribution, criminal investigation, transportation and field rescue. Simultaneous Localization and Mapping (SLAM), as the core technology of autonomous driving, has gradually become a hot topic in the research of intelligent systems in recent years. Position recognition is a technology that helps robots determine whether to revisit historical locations, and it has a strong correlation with the SLAM system. The classic SLAM framework includes: front-end odometer, back-end optimization, loop detection and back-end optimization, among which the tasks of position recognition and loop detection modules are the same. When the loop is effectively identified, the system passes the matching constraints to the back-end optimization module to weaken the trajectory drift. Quickly and accurately identifying revisited locations and constructing loop constraints are very helpful in improving positioning accuracy and maintaining system stability.
目前,视觉位置识别方案通常利用词模型(BOW,Bag of Words),将图像数据编码为词典,设计各种特征描述符与词典相结合进行检索和匹配。但在光照变化剧烈、动态目标较多、季节更替明显的场景下,匹配误差显著增加,位置识别性能急剧下降。基于三维点云数据的位置识别方案更加关注空间几何信息,能够克服光照变化、视场角小等因素的影响。目前常见的基于点云的位置识别方案有:基于局部描述符,基于全局描述符,基于语义信息、基于深度学习和基于人工地标等方法。局部描述符按照特定的规则对关键点邻域空间进行特征统计,但受限于特征点的可重复性较差,在高速自动驾驶场景下效果不佳。全局描述符直接对整个点云进行编码作为环境描述,克服了特征不稳定的问题,但忽略了特征之间的相互关系。语义特征将数据关联从传统的像素级别提升到物体级别,可以强化机器人对周围环境的理解。但是这种方法数据处理效率较低,需要进行复杂的操作来推断目标之间联系。深度学习方法通过训练神经网络来强化特征学习,生成图像特征的深层次表达。虽然提高了位置识别的效率与准确率,但是 这些特征较为复杂难以理解,模型训练需要繁琐的数据清洗工作。此外,模型训练结果一定程度上依赖于样本的多样性,场景的差异性可能会影响识别效果。人工地标通常是在特定场景提前布设,虽然成本较低,但在动态场景下容易因视野遮挡而失效。At present, visual location recognition schemes usually use word models (BOW, Bag of Words) to encode image data into dictionaries, and design various feature descriptors combined with dictionaries for retrieval and matching. However, in scenes with drastic lighting changes, many dynamic targets, and obvious seasonal changes, the matching error increases significantly and the location recognition performance drops sharply. Location recognition schemes based on three-dimensional point cloud data pay more attention to spatial geometric information and can overcome the influence of factors such as lighting changes and small field of view. At present, common point cloud-based location recognition schemes include: local descriptor-based, global descriptor-based, semantic information-based, deep learning-based, and artificial landmark-based methods. Local descriptors perform feature statistics on the neighborhood space of key points according to specific rules, but are limited by the poor repeatability of feature points and are not effective in high-speed autonomous driving scenarios. Global descriptors directly encode the entire point cloud as an environmental description, overcoming the problem of feature instability, but ignoring the relationship between features. Semantic features elevate data association from the traditional pixel level to the object level, which can enhance the robot's understanding of the surrounding environment. However, this method has low data processing efficiency and requires complex operations to infer the relationship between targets. Deep learning methods strengthen feature learning by training neural networks to generate deep expressions of image features. Although it improves the efficiency and accuracy of location recognition, These features are complex and difficult to understand, and model training requires tedious data cleaning. In addition, the model training results depend to a certain extent on the diversity of samples, and the differences in scenes may affect the recognition effect. Artificial landmarks are usually deployed in advance in specific scenes. Although the cost is low, they are easily ineffective in dynamic scenes due to obstruction of vision.
发明内容Summary of the invention
针对现有技术中存在的上述问题和缺陷,本发明提出了一种新颖的融合点云地图、运动模型和局部特征的位置识别方法。所提方法只需使用多线激光雷达作为唯一的量测传感器,有效克服了光照变化的影响,相比视觉方案更加稳定。该发明创新性地融入运动模型,有效解决位置识别中感知混淆的问题,当环境高度相似时仍能有效检测回环位置。将点云地图与局部特征相结合,显著降低了搜索空间,在有限的计算资源下仍能取得优越的运行效率,提高系统的容错率,保证其他功能模块有序进行。该方法能够克服目标遮挡或存在动态目标的问题,提高系统在动态城市下的安全性能。此外,该方法在轨迹持续增加,数据不断累积的情况下,仍能保持良好的识别准确率和效率。作为一种新颖的位置识别方案,能够快速、准确地提供识别结果,提高系统的定位精度。In view of the above problems and defects existing in the prior art, the present invention proposes a novel location recognition method integrating point cloud maps, motion models and local features. The proposed method only needs to use multi-line laser radar as the only measurement sensor, effectively overcomes the influence of illumination changes, and is more stable than the visual solution. The invention innovatively incorporates the motion model to effectively solve the problem of perceptual confusion in location recognition, and can still effectively detect the loop position when the environment is highly similar. Combining the point cloud map with the local features significantly reduces the search space, and can still achieve superior operating efficiency under limited computing resources, improve the fault tolerance of the system, and ensure that other functional modules are carried out in an orderly manner. The method can overcome the problem of target occlusion or the presence of dynamic targets, and improve the safety performance of the system in dynamic cities. In addition, the method can still maintain good recognition accuracy and efficiency when the trajectory continues to increase and the data continues to accumulate. As a novel location recognition solution, it can provide recognition results quickly and accurately, and improve the positioning accuracy of the system.
本发明为解决无人驾驶与人工智能领域的关键技术(位置识别)中存在的感知混淆、运行效率低,识别错误率高以及抗干扰性差等问题,提出了一种融合点云地图、运动模型和局部特征的位置识别方法。首先,预先在无人驾驶场景采集数据,通过高精度的组合导航或匹配算法计算相对位姿并拼接点云地图,在地图内部生成参考点作为虚拟地标。然后,在地图内构建不同朝向的鸟瞰图特征描述集,以此模拟车辆的位置和朝向,并离线保存地图相关数据。其次,对车辆轨迹进行初始化,针对第一帧和第二帧数据,分别在地图描述集中采用全局搜索和局部搜索的方式估算车辆位置。最后,在实时位置识别中,基于车辆运动模型和局部搜索,采用卡尔曼滤波算法估算当前车辆位置,进一步判断是否存在回环。In order to solve the problems of perceptual confusion, low operating efficiency, high recognition error rate and poor anti-interference in the key technology (position recognition) in the field of unmanned driving and artificial intelligence, the present invention proposes a position recognition method that integrates point cloud map, motion model and local features. First, data is collected in the unmanned driving scene in advance, and the relative posture is calculated and the point cloud map is spliced through high-precision combined navigation or matching algorithm, and reference points are generated inside the map as virtual landmarks. Then, a bird's-eye view feature description set with different orientations is constructed in the map to simulate the position and orientation of the vehicle, and the map-related data is saved offline. Secondly, the vehicle trajectory is initialized, and the vehicle position is estimated by global search and local search in the map description set for the first frame and the second frame of data, respectively. Finally, in real-time position recognition, based on the vehicle motion model and local search, the Kalman filter algorithm is used to estimate the current vehicle position, and further determine whether there is a loop.
为实现上述目的,本发明所设计的一种融合点云地图、运动模型和局部特征的位置识别方法,主要包括以下步骤:To achieve the above purpose, the present invention designs a location recognition method integrating point cloud map, motion model and local features, which mainly includes the following steps:
步骤1,针对离线记录的多帧激光雷达点云,基于高精度组合导航计算相对位姿,拼接场景先验点云地图,在地图内部生成参考点作为虚拟地标。 Step 1: For the multi-frame lidar point cloud recorded offline, the relative pose is calculated based on high-precision integrated navigation, the scene prior point cloud map is stitched, and reference points are generated inside the map as virtual landmarks.
步骤2,遍历地图参考点,结合点云地图,构建不同朝向下鸟瞰图特征描述,生成地图描述集,并离线保存地图相关数据。Step 2: traverse the map reference points, combine with the point cloud map, construct the feature description of the bird's-eye view under different directions, generate a map description set, and save the map-related data offline.
步骤3,针对轨迹第一帧点云,在地图描述集中采用全局暴力搜索计算出最佳匹配的地图描述子,基于地图参考点估算车辆位置。Step 3: For the point cloud of the first frame of the trajectory, a global brute force search is used in the map description set to calculate the best matching map descriptor, and the vehicle position is estimated based on the map reference point.
步骤4,针对轨迹第二帧点云,借助起始位置估计,在地图描述集中采用局部搜索计算出最佳匹配的地图描述子,基于地图参考点估算车辆位置。Step 4: For the point cloud of the second frame of the trajectory, with the help of the starting position estimation, a local search is used in the map description set to calculate the best matching map descriptor, and the vehicle position is estimated based on the map reference point.
步骤5,针对实时点云数据,采用描述符相似性作为可靠性度量。借助前两帧位置估计,结合车辆运动模型和局部搜索,采用卡尔曼滤波估算当前车辆位置。Step 5: For real-time point cloud data, descriptor similarity is used as a reliability measure. With the help of the position estimation of the first two frames, combined with the vehicle motion model and local search, the Kalman filter is used to estimate the current vehicle position.
进一步的,步骤1的具体实现包括如下子步骤;Furthermore, the specific implementation of step 1 includes the following sub-steps:
步骤1.1,采用搭载多线激光雷达的自动驾驶车辆预先在场景内部环绕采集数据。利用全球定位系统(GPS,Global Positioning System)和惯性导航系统(INS,Inertial Navigation System)等组合导航,计算高精度位姿,并拼接点云地图。若没有组合导航装置,可利用激光里程计匹配算法代替。Step 1.1: Use an autonomous driving vehicle equipped with a multi-line laser radar to collect data in advance inside the scene. Use combined navigation such as the global positioning system (GPS) and the inertial navigation system (INS) to calculate high-precision pose and stitch point cloud maps. If there is no combined navigation device, a laser odometer matching algorithm can be used instead.
步骤1.2,结合传感器的高度利用两个高度阈值z1和z2滤除地图中部分点云,采用边长为lm的体素对地图进行降采样,将其投影到XOY平面内。其中,激光雷达的坐标系方向为X轴指向前进方向,Y轴向左,Z轴竖直向上。若为其他安装方向,对应调整阈值即可。Step 1.2, use two height thresholds z 1 and z 2 to filter out some point clouds in the map based on the sensor height, downsample the map using voxels with a side length of 1 m , and project it into the XOY plane. The coordinate system of the laser radar is in the direction of travel with the X axis pointing to the left, and the Z axis pointing vertically upward. If it is installed in other directions, adjust the threshold accordingly.
步骤1.3,计算投影地图的二维坐标极值(xmin,ymin,xmax,ymax),按照等距离采样在地图内生成参考点Mr作为虚拟地标,利用点云处理软件,如CloudCompare裁剪部分不合理参考点。参考点坐标计算如下:
Step 1.3, calculate the two-dimensional coordinate extreme values (x min , y min , x max , y max ) of the projected map, generate reference points Mr in the map as virtual landmarks according to equidistant sampling, and use point cloud processing software such as CloudCompare to crop some unreasonable reference points. The reference point coordinates are calculated as follows:
式中i和j为采样点行列号,dx和dy为采样距离,(x,y)为采样点坐标。Where i and j are the row and column numbers of the sampling points, dx and dy are the sampling distances, and (x, y) are the coordinates of the sampling points.
进一步的,步骤2的具体实现包括如下子步骤;Furthermore, the specific implementation of step 2 includes the following sub-steps:
步骤2.1,在地图参考点中构建KD树treec,遍历所有地图参考点,在每一点处:以当前点为中心,依次对地图旋转不同角度θ,其中角度计算公式如下:
θi=i·θ',i∈N+,θ∈(0,2π]        (2)
Step 2.1, construct a KD tree tree c in the map reference point, traverse all map reference points, and at each point: take the current point as the center, rotate the map by different angles θ in turn, where the angle calculation formula is as follows:
θ i =i·θ',i∈N + ,θ∈(0,2π] (2)
式中θ'为角度分辨率,i为朝向角度序号,N+表示正整数。Where θ' is the angle resolution, i is the orientation angle number, and N + represents a positive integer.
步骤2.2,针对地图参考点pk(xk,yk)∈Mr,其中k为地图点索引。旋转角度θi下生成nr行nc列的水平面内(鸟瞰图)的环形特征描述符,模拟车辆在该点朝向为θi时所采集的数据。将描述符转为nr×nc行1列的向量f,并将所有描述符向量拼接为地图特征矩阵F=[f1,f2,...,fk]。特征描述符的具体计算方式如下:
Step 2.2, for the map reference point p k (x k ,y k )∈M r , where k is the map point index. Generate a circular feature descriptor in the horizontal plane (bird's eye view) with n r rows and n c columns under the rotation angle θ i , simulating the data collected when the vehicle is facing θ i at this point. Convert the descriptor into a vector f with n r × n c rows and 1 column, and concatenate all descriptor vectors into a map feature matrix F = [f 1 ,f 2 ,...,f k ]. The specific calculation method of the feature descriptor is as follows:
式中j为邻域点索引,r和c为描述符行列索引,dmax为最大距离,d'为距离分辨率,α'为角度分辨率。Where j is the neighborhood point index, r and c are the descriptor row and column indices, d max is the maximum distance, d' is the distance resolution, and α' is the angle resolution.
步骤2.3,将地图相关数据保存为离线的二进制文件,其中包括:(1)地图参考点云;(2)地图参考点KD树;(3)地图特征矩阵。其中,地图特征矩阵文件中记录了每一个占据元素的全局索引,具体如下:
Step 2.3, save the map-related data as an offline binary file, which includes: (1) map reference point cloud; (2) map reference point KD tree; (3) map feature matrix. The map feature matrix file records the global index of each occupied element, as follows:
式中idm、idθ和idf分别为地图参考点、朝向角度和描述符向量元素的索引。s为离线记录的数据,nθ为角度数量,nf=nr·nc为描述符元素数。mod和rem分别表示取模和取余运算。where id m , id θ and id f are the map reference point, orientation angle and index of the descriptor vector element respectively. s is the data recorded offline, n θ is the number of angles, n f = n r · n c is the number of descriptor elements. mod and rem represent modulus and remainder operations respectively.
进一步的,步骤3的具体实现包括如下子步骤;Furthermore, the specific implementation of step 3 includes the following sub-steps:
步骤3.1,在开启实时运行系统后,只需加载一次离线地图文件。针对第一帧激光雷达点云,进行高度滤波,其中高度保留范围应尽可能与地图取值范围一致。生成nr行nc列鸟瞰图描述符,并调整为nr×nc行1列的特征向量。Step 3.1, after starting the real-time system, only one offline map file needs to be loaded. For the first frame of the lidar point cloud, perform height filtering, where the height retention range should be as consistent as possible with the map value range. Generate an n r row and n c column bird's-eye view descriptor and adjust it to a feature vector of n r × n c rows and 1 column.
步骤3.2,以地图描述符为参考,计算实时描述符在地图描述符的击中率作为评价函数。在所有地图描述符集中,选取最大评价函数对应的地图描述符向量作为最佳候选。评价函数L计算公式如下:
Step 3.2, using the map descriptor as a reference, calculate the hit rate of the real-time descriptor in the map descriptor as the evaluation function. In all map descriptor sets, select the map descriptor vector corresponding to the maximum evaluation function As the best candidate. The calculation formula of the evaluation function L is as follows:
式中fs和fm分别为实时与地图描述符向量。n(1,0)和n(1,1)分别表示fs和fm中为(1,0)和(0,1)的元素数。Where fs and fm are real-time and map descriptor vectors, respectively. n(1,0) and n(1,1) represent the number of elements in fs and fm that are (1,0) and (0,1), respectively.
步骤3.3,根据最佳匹配向量索引和地图参考点Mr,估算当前车辆的位置。位置计算方式如下:
(xv,yv)∈Mr,v=mod(idbest,nθ)      (6)
Step 3.3, based on the best matching vector The index and map reference point Mr are used to estimate the current vehicle position. The position is calculated as follows:
(x v ,y v )∈M r ,v=mod(id best ,n θ ) (6)
式中(xv,yv)为当前车辆的位置估计,v为地图参考点索引,idbest为最佳匹配的地图描述向量索引。Where (x v ,y v ) is the estimated position of the current vehicle, v is the map reference point index, and id best is the index of the best matching map description vector.
进一步的,步骤4的具体实现包括如下子步骤;Furthermore, the specific implementation of step 4 includes the following sub-steps:
步骤4.1针对轨迹中第二帧激光雷达点云,利用第一帧的位置估计,在地图参考点KD树中搜索出最近邻点云P,其包含点数为Nk,根据地图描述符集索引,构建局部描述符集合FL
FL={fi,i=[j·nf·nθ,(j+1)·nf·nθ],pj∈P}      (7)
Step 4.1 For the second frame of the laser radar point cloud in the trajectory, use the position estimation of the first frame to search for the nearest neighbor point cloud P in the map reference point KD tree, which contains N k points. According to the map descriptor set index, construct the local descriptor set FL :
F L ={ fi ,i=[j· nf · ,(j+1)· nf · ],pj∈P} ( 7)
式中j是地图参考点索引,i是地图特征描述集索引。Where j is the map reference point index and i is the map feature description set index.
步骤4.2在局部描述符集中,采用暴力搜索计算出与实时描述符最匹配的地图描述符、根据描述符索引,计算当前车辆的位置和朝向。In step 4.2, in the local descriptor set, a brute force search is used to calculate the map descriptor that best matches the real-time descriptor, and the position and orientation of the current vehicle are calculated based on the descriptor index.
进一步的,步骤5的具体实现包括如下子步骤;Furthermore, the specific implementation of step 5 includes the following sub-steps:
步骤5.1,对于轨迹初始化后的实时激光雷达点云,利用前两帧的位置估计,采用车辆的匀速运动模型,预测出车辆的运动位置。其中车辆的运动模型如下:

Step 5.1: For the real-time LiDAR point cloud after trajectory initialization, use the position estimation of the first two frames and the uniform motion model of the vehicle to predict the vehicle's motion position. The vehicle's motion model is as follows:

式中^分别代表后验和先验变量。xt+1表示在t+1时刻的位置变量,F(·)表示车辆的运动模型,A为运动模型中误差传播的雅克比系数矩阵,V表示方差矩阵,ωx为位置估计误差,为运动预测的方差。 Where ~ and ^ represent the posterior and prior variables respectively. x t+1 represents the position variable at time t+1, F(·) represents the motion model of the vehicle, A is the Jacobian coefficient matrix of error propagation in the motion model, V represents the variance matrix, ω x is the position estimation error, is the variance of motion prediction.
步骤5.2,利用前一帧的位置估计,在地图参考点中采用局部搜索,在邻域描述符集中寻找最佳匹配的描述符,计算出邻域相似的地图参考点,以此作为观测模型:

Step 5.2, using the position estimate of the previous frame, a local search is performed in the map reference point to find the best matching descriptor in the neighborhood descriptor set, and the map reference point with similar neighborhood is calculated as the observation model:

式中H(·)是观测模型,该发明使用局部搜索作为观测模型。是使用局部搜索估计的车辆位置,ωs是观测误差。为利用实时描述符fs和地图描述符fm计算得到的公式(5)中评价函数结果;每一次计算中,使用评价函数结果和采样距离dx、dy计算位置估计的方差σsWherein H(·) is the observation model, and the present invention uses local search as the observation model. is the vehicle position estimated using local search and ωs is the observation error. is the evaluation function result in formula (5) calculated using the real-time descriptor fs and the map descriptor fm ; in each calculation, the variance σs of the position estimate is calculated using the evaluation function result and the sampling distances dx , dy .
步骤5.3,在确定运动模型和观测模型后,采用卡尔曼滤波估算出当前车辆的位置,计算方式如下:


Step 5.3, after determining the motion model and observation model, use Kalman filtering to estimate the current vehicle position. The calculation method is as follows:


式中K为卡尔曼增益,B对为观测模型的误差传播方程的雅克比矩阵。本发明通过上述公式可以快速地计算出车辆位置。Where K is the Kalman gain, and B is the Jacobian matrix of the error propagation equation of the observation model. The present invention can quickly calculate the vehicle position through the above formula.
进一步的,步骤5还包括对比历史轨迹判断是否存在回环,如果与历史位置的距离差小于设定阈值,则认为是回环,否则非回环,以此实现位置识别。Furthermore, step 5 also includes comparing the historical trajectory to determine whether there is a loop. If the distance difference with the historical position is less than a set threshold, it is considered to be a loop, otherwise it is not a loop, thereby achieving position recognition.
进一步的,步骤2.1中地图旋转角度分辨率设置为3度,即θ为3。Furthermore, in step 2.1, the map rotation angle resolution is set to 3 degrees, that is, θ is 3.
进一步的,步骤2.2中环形鸟瞰图特征描述尺寸设置为40行60列。Furthermore, the feature description size of the annular bird's-eye view in step 2.2 is set to 40 rows and 60 columns.
进一步的,步骤4.1中在地图参考点中搜索最近邻个数为20。Furthermore, in step 4.1, the number of nearest neighbors searched in the map reference point is 20.
本发明具有如下积极效果:The present invention has the following positive effects:
1)本发明解决了自动驾驶和人工智能领域的一个关键问题,即位置识别。本发明提出了一种新颖的融合点云地图、运动模型和局部特征的位置识别方法,在全局地图内部计算出车辆位置,有效解决了位置识别或回环检测中感知混淆、识别错误率高的问题。1) The present invention solves a key problem in the field of autonomous driving and artificial intelligence, namely, position recognition. The present invention proposes a novel position recognition method that integrates point cloud maps, motion models, and local features, calculates the vehicle position within the global map, and effectively solves the problems of perceptual confusion and high recognition error rate in position recognition or loop detection.
2)本发明不同于一般的卡尔曼滤波数据融合算法,使用局部搜索特征描述符作为观测模型,为卡尔曼滤波算法开创了一种新的思路。同时,仅使用激光雷达作为单一的量测传感器,显著降低了无人驾驶系统的成本,提高了算法的适用性。2) This invention is different from the general Kalman filter data fusion algorithm. It uses the local search feature descriptor as the observation model, which creates a new idea for the Kalman filter algorithm. At the same time, it only uses the laser radar as a single measurement sensor, which significantly reduces the cost of the unmanned driving system and improves the applicability of the algorithm.
3)本发明将地图与局部特征相结合,显著降低搜索空间,在有限的计 算资源下依旧可以取得优越的运行效率。在算法中加入运动模型,有效克服了目标遮挡或动态目标的影响,大幅度提升系统的容错率。同时,位置识别性能不会随轨迹或运行时间增加而退化,提高了无人驾驶系统在动态城市下的安全性能。3) The present invention combines the map with local features, significantly reducing the search space and The algorithm can still achieve superior operating efficiency with low computing resources. The motion model is added to the algorithm to effectively overcome the impact of target occlusion or dynamic targets, greatly improving the system's fault tolerance. At the same time, the position recognition performance will not degrade with the increase of trajectory or running time, which improves the safety performance of the unmanned driving system in dynamic cities.
采用本发明可有效实现在大范围、动态城市环境下自动驾驶车辆的位置识别,实时位置计算速率可达到200FPS(Frames Per Seond),无需依赖其他外源信号,在降低了系统成本的同时,显著提高了SLAM技术中回环检测的效率。提出将局部特征搜索作为观测模型,不借助其他传感器量测值,为卡尔曼滤波算法提供了一种新的思想。该发明解决了大范围场景下的位置识别问题,将运动模型加入到位置计算中,能够解决感知混淆、目标遮挡等影响。同时,快速的位置识别可以有效地与历史数据构建匹配关联,提高系统整体的定位精度,保证其他模块正常进行,有效提高系统的容错率。此外,位置识别性能不会受到轨迹和运行时间的增加而退化。因此,本发明方法对无人驾驶、机器人等领域具有重要意义。The present invention can effectively realize the position recognition of autonomous driving vehicles in a large-scale, dynamic urban environment. The real-time position calculation rate can reach 200FPS (Frames Per Seond), without relying on other external signals. While reducing the system cost, it significantly improves the efficiency of loop detection in SLAM technology. It is proposed to use local feature search as an observation model without the help of other sensor measurements, which provides a new idea for the Kalman filter algorithm. The invention solves the problem of position recognition in a large-scale scene, and adds the motion model to the position calculation, which can solve the effects of perception confusion, target occlusion, etc. At the same time, fast position recognition can effectively build matching associations with historical data, improve the overall positioning accuracy of the system, ensure the normal operation of other modules, and effectively improve the fault tolerance of the system. In addition, the position recognition performance will not be degraded by the increase of trajectory and running time. Therefore, the method of the present invention is of great significance to the fields of unmanned driving, robots, etc.
附图说明BRIEF DESCRIPTION OF THE DRAWINGS
图1为本发明的流程图。FIG. 1 is a flow chart of the present invention.
图2为本发明中全局地图参考点示意图。FIG. 2 is a schematic diagram of global map reference points in the present invention.
图3为本发明中轨迹第一帧位置估计示意图。FIG. 3 is a schematic diagram of the first frame position estimation of the trajectory in the present invention.
图4为本发明中轨迹第二帧位置估计示意图。FIG. 4 is a schematic diagram of the second frame position estimation of the trajectory in the present invention.
图5为本发明中顾及运动模型和局部搜索的卡尔曼滤波算法示意图。FIG. 5 is a schematic diagram of a Kalman filter algorithm taking into account motion model and local search in the present invention.
图6为本发明中重访位置匹配计算结果的示意图。FIG. 6 is a schematic diagram of the calculation result of the revisited position matching in the present invention.
具体实施方式Detailed ways
下面结合附图对本发明的方案做进一步详细的说明。The scheme of the present invention is further described in detail below in conjunction with the accompanying drawings.
如图1所示,一种融合点云地图、运动模型和局部特征的位置识别方法,主要包括以下步骤:As shown in FIG1 , a location recognition method integrating a point cloud map, a motion model and local features mainly includes the following steps:
步骤1,针对离线记录的多帧激光雷达点云,基于高精度组合导航计算相对位姿,拼接场景先验点云地图,在地图内部生成参考点作为虚拟地标。Step 1: For the multi-frame lidar point cloud recorded offline, the relative pose is calculated based on high-precision integrated navigation, the scene prior point cloud map is stitched, and reference points are generated inside the map as virtual landmarks.
首先,预先在自动驾驶场景环绕采集激光雷达点云数据,利用高精度组合导航计算出车辆的位姿信息,进一步计算出相邻帧点云之间的相对位 姿,每10帧选取1帧关键帧,将所有关键帧点云在相同坐标系下拼接成点云地图。First, the laser radar point cloud data is collected in advance around the autonomous driving scene, and the vehicle’s posture information is calculated using high-precision integrated navigation. The relative position between adjacent frame point clouds is further calculated. One key frame is selected every 10 frames, and all key frame point clouds are spliced into a point cloud map in the same coordinate system.
然后,测量传感器距离地面的高度,设置两个高度阈值z1和z2,保留阈值范围内的点云。采用边长为0.1m的体素格网对地图进行降采样,并投影至水平面内。实验中,激光雷达的坐标系方向为X轴指向前进方向,Y轴向左,Z轴竖直向上。若为其他安装方向,对应调整阈值即可。Then, measure the height of the sensor from the ground, set two height thresholds z 1 and z 2 , and retain the point cloud within the threshold range. Use a voxel grid with a side length of 0.1m to downsample the map and project it into the horizontal plane. In the experiment, the coordinate system of the lidar is oriented with the X axis pointing in the forward direction, the Y axis pointing to the left, and the Z axis pointing vertically upward. If it is installed in other directions, just adjust the threshold accordingly.
最后,计算地图投影点云的坐标极值,设置x和y方向的采样距离间隔dx和dy均为1m,生成地图参考点,并采用CloudCompare裁剪部分错误点。Finally, the coordinate extremes of the map projection point cloud are calculated, the sampling distance intervals dx and dy in the x and y directions are set to 1m, the map reference points are generated, and CloudCompare is used to crop some erroneous points.
本实施例中,步骤1的具体实现方式如下:In this embodiment, the specific implementation of step 1 is as follows:
步骤1.1,采用搭载多线激光雷达的自动驾驶车辆预先在场景内部环绕采集数据。利用全球定位系统(GPS,Global Positioning System)和惯性导航系统(INS,Inertial Navigation System)等组合导航,计算高精度位姿,并拼接点云地图。若没有组合导航装置,可利用激光里程计匹配算法代替。Step 1.1: Use an autonomous driving vehicle equipped with a multi-line laser radar to collect data in advance inside the scene. Use combined navigation such as the global positioning system (GPS) and the inertial navigation system (INS) to calculate high-precision pose and stitch point cloud maps. If there is no combined navigation device, a laser odometer matching algorithm can be used instead.
步骤1.2,结合传感器的高度利用两个高度阈值z1和z2滤除地图中部分点云,采用边长为lm的体素对地图进行降采样,将其投影到XOY平面内。其中,激光雷达的坐标系方向为X轴指向前进方向,Y轴向左,Z轴竖直向上。若为其他安装方向,对应调整阈值即可。Step 1.2, use two height thresholds z 1 and z 2 to filter out some point clouds in the map based on the sensor height, downsample the map using voxels with a side length of 1 m , and project it into the XOY plane. The coordinate system of the laser radar is in the direction of travel with the X axis pointing to the left, and the Z axis pointing vertically upward. If it is installed in other directions, adjust the threshold accordingly.
步骤1.3,计算投影地图的二维坐标极值(xmin,ymin,xmax,ymax),按照等距离采样在地图内生成参考点Mr作为虚拟地标,利用点云处理软件,如CloudCompare裁剪部分不合理参考点。参考点坐标计算如下:
Step 1.3, calculate the two-dimensional coordinate extreme values (x min , y min , x max , y max ) of the projected map, generate reference points Mr in the map as virtual landmarks according to equidistant sampling, and use point cloud processing software such as CloudCompare to crop some unreasonable reference points. The reference point coordinates are calculated as follows:
式中i和j为采样点行列号,dx和dy为采样距离,(x,y)为采样点坐标。Where i and j are the row and column numbers of the sampling points, dx and dy are the sampling distances, and (x, y) are the coordinates of the sampling points.
步骤2,遍历地图参考点,结合点云地图,构建不同朝向下鸟瞰图特征描述,生成地图描述集,并离线保存地图相关数据。Step 2: traverse the map reference points, combine with the point cloud map, construct the feature description of the bird's-eye view under different directions, generate a map description set, and save the map-related data offline.
首先,在地图参点中构建KD树,并在每一个参考点处,以参考点为中心,以3度的角度分辨率,旋转地图点云。图2(a)中绿色点为地图参考点,橙色点为点云地图。First, a KD tree is constructed in the map reference points, and at each reference point, the map point cloud is rotated with an angle resolution of 3 degrees around the reference point. In Figure 2(a), the green points are map reference points, and the orange points are point cloud maps.
然后,在每一个参考点处,每个旋转角度下,生成40行60列的环形 描述符,最大距离dmax为80m,将每一个描述符转为一个2400行1列的向量,并将所有特征向量构建为一个特征矩阵。如图2(b)为鸟瞰图特征描述符示意图。Then, at each reference point and each rotation angle, a ring with 40 rows and 60 columns is generated. Descriptor, the maximum distance d max is 80m, each descriptor is converted into a vector of 2400 rows and 1 column, and all feature vectors are constructed into a feature matrix. Figure 2(b) shows a schematic diagram of the feature descriptor of the bird's-eye view.
最后,将地图参考点云、地图特征矩阵和地图参考点KD树保存为离线文件。Finally, the map reference point cloud, map feature matrix and map reference point KD tree are saved as offline files.
本实施例中,步骤2的具体实现方式如下:In this embodiment, the specific implementation of step 2 is as follows:
步骤2.1,在地图参考点中构建KD树treec,遍历所有地图参考点,在每一点处:以当前点为中心,依次对地图旋转不同角度θ,其中角度计算公式如下:
θi=i·θ',i∈N+,θ∈(0,2π]    (2)
Step 2.1, construct a KD tree tree c in the map reference point, traverse all map reference points, and at each point: take the current point as the center, rotate the map by different angles θ in turn, where the angle calculation formula is as follows:
θ i =i·θ',i∈N + ,θ∈(0,2π] (2)
式中θ'为角度分辨率,i为朝向角度序号,N+表示正整数。Where θ' is the angle resolution, i is the orientation angle number, and N + represents a positive integer.
步骤2.2,针对地图参考点pk(xk,yk)∈Mr,pk表示参考点Mr中的第k个点,xk,yk表示pk的横纵坐标,其中k为地图点索引。旋转角度θi下生成nr行nc列的水平面内(鸟瞰图)的环形特征描述符,模拟车辆在该点朝向为θi时所采集的数据。将描述符转为nr×nc行1列的向量f,并将所有描述符向量拼接为地图特征矩阵F=[f1,f2,....,fk]。特征描述符的具体计算方式如下:
Step 2.2, for the map reference point p k (x k ,y k )∈M r , p k represents the kth point in the reference point Mr , x k ,y k represent the horizontal and vertical coordinates of p k , where k is the map point index. Generate a circular feature descriptor in the horizontal plane (bird's eye view) with n r rows and n c columns under the rotation angle θ i , simulating the data collected when the vehicle is facing θ i at this point. Convert the descriptor into a vector f with n r × n c rows and 1 column, and concatenate all descriptor vectors into a map feature matrix F = [f 1 ,f 2 ,....,f k ]. The specific calculation method of the feature descriptor is as follows:
式中j为邻域点索引,r和c为描述符行列索引,dmax为最大距离,d'为距离分辨率,α'为角度分辨率。Where j is the neighborhood point index, r and c are the descriptor row and column indices, d max is the maximum distance, d' is the distance resolution, and α' is the angle resolution.
步骤2.3,将地图相关数据保存为离线的二进制文件,其中包括:(1)地图参考点云;(2)地图参考点KD树;(3)地图特征矩阵。其中,地图特征矩阵文件中记录了每一个占据元素的全局索引,具体如下
Step 2.3, save the map-related data as an offline binary file, which includes: (1) map reference point cloud; (2) map reference point KD tree; (3) map feature matrix. The map feature matrix file records the global index of each occupied element, as follows:
式中idm、idθ和idf分别为地图参考点、朝向角度和描述符向量元素的索引。s为离线记录的数据,nθ为角度数量,nf=nr·nc为描述符元素数。mod和rem 分别表示取模和取余运算。where id m , id θ and id f are the map reference point, orientation angle and index of the descriptor vector element respectively. s is the data recorded offline, n θ is the number of angles, and n f = n r · n c is the number of descriptor elements. mod and rem They represent modulo and remainder operations respectively.
步骤3,如图3为轨迹第一帧数据的位置计算示意图。针对轨迹第一帧点云,在地图描述集中采用全局暴力搜索计算出最佳匹配的地图描述子,基于地图参考点估算车辆位置。Step 3, as shown in Figure 3, is a schematic diagram of the position calculation of the first frame of trajectory data. For the point cloud of the first frame of the trajectory, a global brute force search is used in the map description set to calculate the best matching map descriptor, and the vehicle position is estimated based on the map reference point.
首先,开启实时系统后,加载离线地图文件,对于第一帧激光雷达点云,生成40行60列的环形鸟瞰图特征描述符,并调整为2400行1列的向量。First, after starting the real-time system, load the offline map file, and for the first frame of lidar point cloud, generate a 40-row and 60-column annular bird's-eye view feature descriptor and adjust it to a vector of 2400 rows and 1 column.
然后,计算实时描述符在地图描述符中的击中率作为评价函数,在全部地图描述符中暴力搜索计算出评价函数最优的地图描述符。Then, the hit rate of the real-time descriptor in the map descriptor is calculated as the evaluation function, and the map descriptor with the best evaluation function is calculated by brute force search in all map descriptors.
最后,根据最优地图描述符的索引和地图参考点云,估算出车辆的位置。Finally, the vehicle’s position is estimated based on the index of the optimal map descriptor and the map reference point cloud.
本实施例中,步骤3的具体实现方式如下:In this embodiment, the specific implementation of step 3 is as follows:
步骤3.1,在开启实时运行系统后,只需加载一次离线地图文件。针对第一帧激光雷达点云,进行高度滤波,其中高度保留范围应尽可能与地图取值范围一致。生成nr行nc列鸟瞰图描述符,并调整为nr×nc行1列的特征向量。Step 3.1, after starting the real-time system, only one offline map file needs to be loaded. For the first frame of the lidar point cloud, perform height filtering, where the height retention range should be as consistent as possible with the map value range. Generate an n r row and n c column bird's-eye view descriptor and adjust it to a feature vector of n r × n c rows and 1 column.
步骤3.2,以地图描述符为参考,计算实时描述符在地图描述符的击中率作为评价函数。在所有地图描述符集中,选取最大评价函数对应的地图描述符向量作为最佳候选。评价函数L计算公式如下:
Step 3.2, using the map descriptor as a reference, calculate the hit rate of the real-time descriptor in the map descriptor as the evaluation function. In all map descriptor sets, select the map descriptor vector corresponding to the maximum evaluation function As the best candidate. The evaluation function L is calculated as follows:
式中fs和fm分别为实时与地图描述符向量。n(1,0)和n(1,1)分别表示fs和fm中为(1,0)和(0,1)的元素数。Where fs and fm are the real-time and map descriptor vectors, respectively. n(1,0) and n(1,1) represent the number of elements in fs and fm that are (1,0) and (0,1), respectively.
步骤3.3,根据最佳匹配向量索引和地图参考点Mr,估算当前车辆的位置。位置计算方式如下:
(xv,yv)∈Mr,v=mod(idbest,nθ)      (6)
Step 3.3, based on the best matching vector The index and map reference point Mr are used to estimate the current vehicle position. The position is calculated as follows:
(x v ,y v )∈M r ,v=mod(id best ,n θ ) (6)
式中(xv,yv)为当前车辆的位置估计,v为地图参考点索引,idbest为最佳匹配的地图描述向量索引。Where (x v ,y v ) is the estimated position of the current vehicle, v is the map reference point index, and id best is the index of the best matching map description vector.
步骤4,如图4为第二帧数据的位置计算示意图。针对轨迹第二帧点云,借助第一帧位置估计,在地图描述集中采用局部搜索计算出最佳匹配的地 图描述子,基于地图参考点估算车辆位置。Step 4, as shown in Figure 4, is a schematic diagram of the position calculation of the second frame data. For the second frame point cloud of the trajectory, with the help of the first frame position estimation, a local search is used in the map description set to calculate the best matching map. Graph descriptor, estimates the vehicle position based on map reference points.
首先,针对第二帧点云,利用第一帧的位置结果,在地图参考点云中搜索20个最近邻点,根据地图描述符索引,构建局部描述符集。First, for the second frame point cloud, the position results of the first frame are used to search for 20 nearest neighbor points in the map reference point cloud, and a local descriptor set is constructed according to the map descriptor index.
然后,在局部描述符集中,采暴力搜索出最佳匹配的地图描述符,根据索引估算出车辆当前位置。Then, in the local descriptor set, a brute force search is performed to find the best matching map descriptor, and the current position of the vehicle is estimated based on the index.
本实施例中,步骤4的具体实现方式如下:In this embodiment, the specific implementation of step 4 is as follows:
步骤4.1,针对轨迹中第二帧激光雷达点云,利用第一帧的位置估计,在地图参考点KD树中搜索出最近邻点云P,其包含点数为Nk,根据地图描述符集索引,构建局部描述符集合FL
FL={fi,i=[j·nf·nθ,(j+1)·nf·nθ],pj∈P}    (7)
Step 4.1: for the second frame of the laser radar point cloud in the trajectory, use the position estimation of the first frame to search for the nearest neighbor point cloud P in the map reference point KD tree, which contains N k points. According to the map descriptor set index, construct the local descriptor set FL :
F L ={ fi ,i=[j· nf · ,(j+1)· nf · ],pj∈P} ( 7)
式中j是地图参考点索引,i是地图特征描述集索引。Where j is the map reference point index and i is the map feature description set index.
步骤4.2,在局部描述符集中,采用暴力搜索计算出与实时描述符最匹配的地图描述符、根据描述符索引,计算当前车辆的位置和朝向。In step 4.2, in the local descriptor set, a brute force search is used to calculate the map descriptor that best matches the real-time descriptor, and the position and orientation of the current vehicle are calculated based on the descriptor index.
步骤5,如图5为实时数据的位置计算示意图。针对实时点云数据,采用描述符相似性作为可靠性度量。借助前两帧位置估计,结合车辆运动模型和局部搜索,采用卡尔曼滤波估算当前车辆位置。进一步对比历史轨迹判断是否存在回环。Step 5, as shown in Figure 5, is a schematic diagram of the position calculation of real-time data. For real-time point cloud data, descriptor similarity is used as a reliability metric. With the help of the position estimation of the first two frames, combined with the vehicle motion model and local search, the Kalman filter is used to estimate the current vehicle position. The historical trajectory is further compared to determine whether there is a loop.
首先,对于实时帧激光雷达点云,利用前两帧的位置估计,采用匀速运动模型预测当前车辆位置。发明中假定位置估计误差不会超过地图参考点采样距离,设定位置估计误差为1m。First, for the real-time frame lidar point cloud, the position estimation of the first two frames is used to predict the current vehicle position using a uniform motion model. In the invention, it is assumed that the position estimation error will not exceed the sampling distance of the map reference point, and the position estimation error is set to 1m.
然后,利用前一帧的位置估计,采用类似轨迹第二帧的局部搜索方式,估算出车辆的位置。Then, using the position estimate from the previous frame, a local search similar to the second frame of the trajectory is used to estimate the position of the vehicle.
最后,利用特征描述符的相似性作为位置偏差的度量,采用卡尔曼滤波算法估算当前位置,对比历史轨迹判断是否存在回环。如果与历史位置的距离差小于设定阈值,则认为是回环,否则非回环,以此实现位置识别。Finally, the similarity of feature descriptors is used as a measure of position deviation, and the Kalman filter algorithm is used to estimate the current position, and the historical trajectory is compared to determine whether there is a loop. If the distance difference with the historical position is less than the set threshold, it is considered a loop, otherwise it is not a loop, thereby achieving position recognition.
本实施例中,步骤5的具体实现方式如下:In this embodiment, the specific implementation of step 5 is as follows:
步骤5.1,对于轨迹初始化后的实时激光雷达点云,利用前两帧的位置估计,采用车辆的匀速运动模型,预测出车辆的运动位置。其中车辆的运动模型如下:

Step 5.1: For the real-time LiDAR point cloud after trajectory initialization, use the position estimation of the first two frames and the uniform motion model of the vehicle to predict the vehicle's motion position. The vehicle's motion model is as follows:

式中^分别代表后验和先验变量。xt+1表示在t+1时刻的位置变量,F(·)表示车辆的运动模型,A为运动模型中误差传播的雅克比系数矩阵,V表示方差矩阵。ωx为位置估计误差,为运动预测的方差。 Where ~ and ^ represent the posterior and prior variables respectively. x t+1 represents the position variable at time t+1, F(·) represents the motion model of the vehicle, A is the Jacobian coefficient matrix of error propagation in the motion model, and V represents the variance matrix. ω x is the position estimation error, is the variance of motion prediction.
步骤5.2,利用前一帧的位置估计,在地图参考点中采用局部搜索,在邻域描述符集中寻找最佳匹配的描述符,计算出邻域相似的地图参考点,以此作为观测模型:

Step 5.2, using the position estimate of the previous frame, a local search is performed in the map reference point to find the best matching descriptor in the neighborhood descriptor set, and the map reference point with similar neighborhood is calculated as the observation model:

式中H(·)是观测模型,该发明使用局部搜索作为观测模型。是使用局部搜索估计的车辆位置,ωs是观测误差。为利用实时描述符fs和地图描述符fm计算得到的公式(5)中评价函数结果;每一次计算中,使用评价函数结果和采样距离dx、dy,计算位置估计的方差σsWherein H(·) is the observation model, and the present invention uses local search as the observation model. is the vehicle position estimated using local search and ωs is the observation error. is the evaluation function result in formula (5) calculated using the real-time descriptor fs and the map descriptor fm ; in each calculation, the variance σs of the position estimate is calculated using the evaluation function result and the sampling distances dx , dy ;
步骤5.3,在确定运动模型和观测模型后,采用卡尔曼滤波估算出当前车辆的位置,计算方式如下:


Step 5.3, after determining the motion model and observation model, use Kalman filtering to estimate the current vehicle position. The calculation method is as follows:


式中K为卡尔曼增益,B对为观测模型的误差传播方程的雅克比矩阵。本发明通过上述公式可以快速地计算出车辆位置。Where K is the Kalman gain, and B is the Jacobian matrix of the error propagation equation of the observation model. The present invention can quickly calculate the vehicle position through the above formula.
步骤5.4,对比车辆的历史轨迹,判断是否存在回环。如图6是在三个动态环境下的重访位置匹配效果示意图,水平方向黑色线为车辆的轨迹,竖直方向灰色线为重访位置匹配,竖直方向黑色线为错误的匹配结果。选取三个典型的室外动态场景:城市(DCC 02,#1),大学(KAIST 02,#2)和乡村(KITTI 05,#3)作为实验场景。位姿均由高精度的组合导航计算得到,实验中仅使用激光雷达点云解算位置信息。三个场景分别采集3151帧(#1)、4190帧(#2)和2761帧(#3)点云,室外环境下设置相距8米时为回环匹配,各数据序列均包含一定量的重访位置。为了阈值统一,同样 设置8m以内的位置误差认为是正确的位置计算结果。实验结果如表1所示,在大范围的动态环境下,位置计算的成功率可以达到99%以上,平均误差在0.5m左右,具有良好的位置精度。轻量级的描述符创建耗时约6ms,实时位置解算速度可以达到200FPS。在重访位置识别方面,最大F1分数可达到0.99以上,具有良好的位置识别精度。该方法可为自动驾驶提供一种高效、准确的位置识别方法,提高系统整体的稳定性能。Step 5.4, compare the historical trajectory of the vehicle to determine whether there is a loop. Figure 6 is a schematic diagram of the revisit position matching effect in three dynamic environments. The horizontal black line is the trajectory of the vehicle, the vertical gray line is the revisit position matching, and the vertical black line is the wrong matching result. Three typical outdoor dynamic scenes are selected as experimental scenes: city (DCC 02, #1), university (KAIST 02, #2) and rural (KITTI 05, #3). The poses are all calculated by high-precision combined navigation. Only the lidar point cloud is used to solve the position information in the experiment. The three scenes collect 3151 frames (#1), 4190 frames (#2) and 2761 frames (#3) of point clouds respectively. In the outdoor environment, the distance is set to 8 meters for loop matching. Each data sequence contains a certain amount of revisited positions. In order to unify the threshold, the same The position error within 8m is considered to be the correct position calculation result. The experimental results are shown in Table 1. In a large-scale dynamic environment, the success rate of position calculation can reach more than 99%, and the average error is about 0.5m, which has good position accuracy. The lightweight descriptor creation takes about 6ms, and the real-time position solution speed can reach 200FPS. In terms of revisited position recognition, the maximum F1 score can reach more than 0.99, which has good position recognition accuracy. This method can provide an efficient and accurate position recognition method for autonomous driving and improve the overall stability of the system.
表1 在不同场景下位置识别的测试性能
Table 1 Test performance of location recognition in different scenarios
以上所述仅为本发明的优选实施例而已,并不用以限制本发明,对于本领域的技术人员来说,本发明可以有各种更改和变化,凡在本发明的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本发明的保护范围之内。 The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims (10)

  1. 一种融合点云地图、运动模型和局部特征的位置识别方法,其特征在于,包括以下步骤:A location recognition method integrating a point cloud map, a motion model and local features, characterized in that it comprises the following steps:
    步骤1,针对离线记录的多帧激光雷达点云,基于高精度组合导航计算相对位姿,拼接场景先验点云地图,在地图内部生成参考点作为虚拟地标;Step 1: For the multi-frame lidar point cloud recorded offline, the relative pose is calculated based on high-precision integrated navigation, the scene prior point cloud map is stitched, and reference points are generated inside the map as virtual landmarks;
    步骤2,遍历地图参考点,结合点云地图,构建不同朝向下鸟瞰图特征描述,生成地图描述集,并离线保存地图相关数据;Step 2: traverse the map reference points, combine the point cloud map, construct the feature description of the bird's-eye view under different directions, generate a map description set, and save the map-related data offline;
    步骤3,针对轨迹第一帧点云,在地图描述集中采用全局暴力搜索计算出最佳匹配的地图描述子,基于地图参考点估算车辆位置;Step 3: For the point cloud of the first frame of the trajectory, a global brute force search is used in the map description set to calculate the best matching map descriptor, and the vehicle position is estimated based on the map reference point;
    步骤4,针对轨迹第二帧点云,借助起始位置估计,在地图描述集中采用局部搜索计算出最佳匹配的地图描述子,基于地图参考点估算车辆位置;Step 4: for the point cloud of the second frame of the trajectory, with the help of the starting position estimation, the best matching map descriptor is calculated by local search in the map description set, and the vehicle position is estimated based on the map reference point;
    步骤5,针对实时点云数据,采用描述符相似性作为可靠性度量,借助前两帧位置估计,结合车辆运动模型和局部搜索,采用卡尔曼滤波估算当前车辆位置。Step 5: For real-time point cloud data, descriptor similarity is used as a reliability metric. With the help of the position estimation of the first two frames, combined with the vehicle motion model and local search, the Kalman filter is used to estimate the current vehicle position.
  2. 根据权利要求1所述的一种融合点云地图、运动模型和局部特征的位置识别方法,其特征在于:步骤1的具体实现包括如下子步骤;The location recognition method integrating point cloud map, motion model and local features according to claim 1 is characterized in that: the specific implementation of step 1 includes the following sub-steps;
    步骤1.1,采用搭载多线激光雷达的自动驾驶车辆预先在场景内部环绕采集数据,利用全球定位系统(GPS,Global Positioning System)和惯性导航系统(INS,Inertial Navigation System)组合导航,计算高精度位姿,并拼接点云地图;Step 1.1: Use an autonomous driving vehicle equipped with a multi-line laser radar to collect data in advance inside the scene, use the combined navigation of the global positioning system (GPS) and the inertial navigation system (INS) to calculate the high-precision pose and stitch the point cloud map;
    步骤1.2,结合传感器的高度利用两个高度阈值z1和z2滤除地图中部分点云,采用边长为lm的体素对地图进行降采样,将其投影到XOY平面内;其中,激光雷达的坐标系方向为X轴指向前进方向,Y轴向左,Z轴竖直向上;Step 1.2: Combine the height of the sensor and use two height thresholds z1 and z2 to filter out some point clouds in the map, downsample the map using voxels with a side length of 1 m , and project it into the XOY plane; the coordinate system of the laser radar is oriented in the direction of travel with the X axis pointing to the left and the Z axis pointing vertically upward.
    步骤1.3,计算投影地图的二维坐标极值(xmin,ymin,xmax,ymax),按照等距离采样在地图内生成参考点Mr作为虚拟地标,利用点云处理软件裁剪部分不合理参考点,参考点坐标计算如下:
    Step 1.3, calculate the two-dimensional coordinate extreme values (x min , y min , x max , y max ) of the projection map, generate reference points Mr in the map as virtual landmarks according to equidistant sampling, and use point cloud processing software to crop some unreasonable reference points. The coordinates of the reference points are calculated as follows:
    式中i和j为采样点行列号,dx和dy为采样距离,(x,y)为采样点坐标。Where i and j are the row and column numbers of the sampling points, dx and dy are the sampling distances, and (x, y) are the coordinates of the sampling points.
  3. 根据权利要求1所述的一种融合点云地图、运动模型和局部特征的位置识别方法,其特征在于:步骤2的具体实现包括如下子步骤; The location recognition method integrating point cloud map, motion model and local features according to claim 1 is characterized in that: the specific implementation of step 2 includes the following sub-steps;
    步骤2.1,在地图参考点中构建KD树treec,遍历所有地图参考点,在每一点处:以当前点为中心,依次对地图旋转不同角度θ,其中角度计算公式如下:
    θi=i·θ',i∈N+,θ∈(0,2π]   (2)
    Step 2.1, construct a KD tree tree c in the map reference point, traverse all map reference points, and at each point: take the current point as the center, rotate the map by different angles θ in turn, where the angle calculation formula is as follows:
    θ i =i·θ',i∈N + ,θ∈(0,2π] (2)
    式中θ'为角度分辨率,i为朝向角度序号,N+表示正整数;Where θ' is the angle resolution, i is the orientation angle number, and N + represents a positive integer;
    步骤2.2,针对地图参考点pk(xk,yk)∈Mr,pk表示参考点Mr中的第k个点,xk,yk表示pk的横纵坐标,其中k为地图点索引,旋转角度θi下生成nr行nc列的水平面内的环形特征描述符,模拟车辆在该点朝向为θi时所采集的数据,将描述符转为nr×nc行1列的向量f,并将所有描述符向量拼接为地图特征矩阵F=[f1,f2,...,fk],特征描述符的具体计算方式如下:
    Step 2.2, for the map reference point pk ( xk , yk ) ∈Mr , pk represents the kth point in the reference point Mr , xk , yk represents the horizontal and vertical coordinates of pk , where k is the map point index, generate a circular feature descriptor in the horizontal plane with nr rows and nc columns under the rotation angle θi , simulate the data collected when the vehicle is oriented at this point with θi , convert the descriptor into a vector f with nr × nc rows and 1 column, and concatenate all descriptor vectors into a map feature matrix F=[ f1 , f2 ,..., fk ]. The specific calculation method of the feature descriptor is as follows:
    式中j为邻域点索引,r和c为描述符行列索引,dmax为最大距离,d'为距离分辨率,α'为角度分辨率;Where j is the neighborhood point index, r and c are the descriptor row and column indices, d max is the maximum distance, d' is the distance resolution, and α' is the angle resolution;
    步骤2.3,将地图相关数据保存为离线的二进制文件,其中包括:(1)地图参考点云;(2)地图参考点KD树;(3)地图特征矩阵;其中,地图特征矩阵文件中记录了每一个占据元素的全局索引,具体如下:
    Step 2.3, save the map-related data as an offline binary file, which includes: (1) map reference point cloud; (2) map reference point KD tree; (3) map feature matrix; the map feature matrix file records the global index of each occupied element, as follows:
    式中idm、idθ和idf分别为地图参考点、朝向角度和描述符向量元素的索引,s为离线记录的数据,nθ为角度数量,nf=nr·nc为描述符元素数,mod和rem分别表示取模和取余运算。where id m , id θ and id f are the map reference point, orientation angle and descriptor vector element indices respectively, s is the offline recorded data, n θ is the number of angles, n f = n r · n c is the number of descriptor elements, mod and rem represent modulo and remainder operations respectively.
  4. 根据权利要求1所述的一种融合点云地图、运动模型和局部特征的位置识别方法,其特征在于:步骤3的具体实现方式如下;According to the method for position recognition integrating point cloud map, motion model and local features of claim 1, it is characterized in that: the specific implementation of step 3 is as follows;
    步骤3.1,在开启实时运行系统后,只需加载一次离线地图文件,针对第一帧激光雷达点云,进行高度滤波,其中高度保留范围与地图取值范围一致,生成nr行nc列鸟瞰图描述符,并调整为nr×nc行1列的特征向量;Step 3.1, after starting the real-time system, only need to load the offline map file once, perform height filtering on the first frame of lidar point cloud, where the height retention range is consistent with the map value range, generate n r rows and n c columns of bird's-eye view descriptor, and adjust it to n r × n c rows and 1 column feature vector;
    步骤3.2,以地图描述符为参考,计算实时描述符在地图描述符的击中率作 为评价函数,在所有地图描述符集中,选取最大评价函数对应的地图描述符向量作为最佳候选,评价函数L计算公式如下:
    Step 3.2, using the map descriptor as a reference, calculate the hit rate of the real-time descriptor in the map descriptor. is the evaluation function. Among all the map descriptor sets, the map descriptor vector corresponding to the maximum evaluation function is selected As the best candidate, the evaluation function L is calculated as follows:
    式中fs和fm分别为实时与地图描述符向量,n(1,0)和n(1,1)分别表示fs和fm中为(1,0)和(0,1)的元素数;Where fs and fm are real-time and map descriptor vectors, respectively, n(1,0) and n(1,1) represent the number of elements with (1,0) and (0,1) in fs and fm , respectively;
    步骤3.3,根据最佳匹配向量索引和地图参考点Mr,估算当前车辆的位置,位置计算方式如下:
    (xv,yv)∈Mr,v=mod(idbest,nθ)   (6)
    Step 3.3, based on the best matching vector The index and map reference point Mr are used to estimate the current vehicle position. The position is calculated as follows:
    (x v ,y v )∈M r ,v=mod(id best ,n θ ) (6)
    式中(xv,yv)为当前车辆的位置估计,v为地图参考点索引,nθ为角度数量,idbest为最佳匹配的地图描述向量索引。Where (x v ,y v ) is the estimated position of the current vehicle, v is the map reference point index, n θ is the number of angles, and id best is the index of the best matching map description vector.
  5. 根据权利要求3所述的一种融合点云地图、运动模型和局部特征的位置识别方法,其特征在于:步骤4的具体实现方式如下;According to the method for position recognition integrating point cloud map, motion model and local features of claim 3, it is characterized in that: the specific implementation of step 4 is as follows;
    步骤4.1,针对轨迹中第二帧激光雷达点云,利用第一帧的位置估计,在地图参考点KD树中搜索出最近邻点云P,其包含点数为Nk,根据地图描述符集索引,构建局部描述符集合FL
    FL={fi,i=[j·nf·nθ,(j+1)·nf·nθ],pj∈P}   (7)
    Step 4.1: for the second frame of the laser radar point cloud in the trajectory, use the position estimation of the first frame to search for the nearest neighbor point cloud P in the map reference point KD tree, which contains N k points. According to the map descriptor set index, construct the local descriptor set FL :
    F L ={ fi ,i=[j· nf · ,(j+1)· nf · ],pj∈P} ( 7)
    式中j是地图参考点索引,i是地图特征描述集索引,nθ为角度数量;Where j is the map reference point index, i is the map feature description set index, and n θ is the number of angles;
    步骤4.2,在局部描述符集中,采用暴力搜索计算出与实时描述符最匹配的地图描述符、根据描述符索引,计算当前车辆的位置和朝向。In step 4.2, in the local descriptor set, a brute force search is used to calculate the map descriptor that best matches the real-time descriptor, and the position and orientation of the current vehicle are calculated based on the descriptor index.
  6. 根据权利要求4所述的一种融合点云地图、运动模型和局部特征的位置识别方法,其特征在于:步骤5的具体实现方式如下;According to the method for position recognition integrating point cloud map, motion model and local features of claim 4, it is characterized in that: the specific implementation of step 5 is as follows;
    步骤5.1,对于轨迹初始化后的实时激光雷达点云,利用前两帧的位置估计,采用车辆的匀速运动模型,预测出车辆的运动位置,其中车辆的运动模型如下:

    Step 5.1: For the real-time lidar point cloud after trajectory initialization, the position estimation of the first two frames is used to predict the vehicle's motion position using the vehicle's uniform motion model, where the vehicle's motion model is as follows:

    式中~和^分别代表后验和先验变量,xt+1表示在t+1时刻的位置变量,F(·)表示车辆的运动模型,A为运动模型中误差传播的雅克比系数矩阵,V表示方差矩阵,ωx为位置估计误差,为运动预测的方差; where ˜ and ˆ represent the posterior and prior variables, respectively; x t+1 represents the position variable at time t+1; F(·) represents the motion model of the vehicle; A is the Jacobian coefficient matrix of error propagation in the motion model; V represents the variance matrix; ω x is the position estimation error; is the variance of motion prediction;
    步骤5.2,利用前一帧的位置估计,在地图参考点中采用局部搜索,在邻域描述符集中寻找最佳匹配的描述符,计算出邻域相似的地图参考点,以此作为观测模型:

    Step 5.2, using the position estimate of the previous frame, a local search is performed in the map reference point to find the best matching descriptor in the neighborhood descriptor set, and the map reference point with similar neighborhood is calculated as the observation model:

    式中H(·)是观测模型,使用局部搜索作为观测模型;是使用局部搜索估计的车辆位置,ωs是观测误差;为利用实时描述符fs和地图描述符fm计算得到的公式(5)中评价函数结果;每一次计算中,使用评价函数结果和采样距离dx、dy,计算位置估计的方差σsWhere H(·) is the observation model, and local search is used as the observation model; is the vehicle position estimated using local search, ω s is the observation error; is the evaluation function result in formula (5) calculated using the real-time descriptor fs and the map descriptor fm ; in each calculation, the variance σs of the position estimate is calculated using the evaluation function result and the sampling distances dx , dy ;
    步骤5.3,在确定运动模型和观测模型后,采用卡尔曼滤波估算出当前车辆的位置,计算方式如下:


    Step 5.3, after determining the motion model and observation model, use Kalman filtering to estimate the current vehicle position. The calculation method is as follows:


    式中K为卡尔曼增益,B对为观测模型的误差传播方程的雅克比矩阵,通过上述公式可以快速地计算出车辆位置。Where K is the Kalman gain, and B is the Jacobian matrix of the error propagation equation of the observation model. The vehicle position can be quickly calculated using the above formula.
  7. 根据权利要求1所述的一种融合点云地图、运动模型和局部特征的位置识别方法,其特征在于:步骤5还包括对比历史轨迹判断是否存在回环,如果与历史位置的距离差小于设定阈值,则认为是回环,否则非回环,以此实现位置识别。According to the location recognition method of integrating point cloud maps, motion models and local features as described in claim 1, it is characterized in that: step 5 also includes comparing historical trajectories to determine whether there is a loop, if the distance difference with the historical position is less than a set threshold, it is considered to be a loop, otherwise it is not a loop, thereby achieving location recognition.
  8. 根据权利要求3所述的一种融合点云地图、运动模型和局部特征的位置识别方法,其特征在于:步骤2.1中地图旋转角度分辨率设置为3度,即θ为3。According to the location recognition method integrating point cloud map, motion model and local features as described in claim 3, it is characterized in that: in step 2.1, the map rotation angle resolution is set to 3 degrees, that is, θ is 3.
  9. 根据权利要求3所述的一种融合点云地图、运动模型和局部特征的位置识别方法,其特征在于:步骤2.2中环形鸟瞰图特征描述尺寸设置为40行60列。According to the location recognition method that integrates point cloud maps, motion models and local features as described in claim 3, it is characterized in that the feature description size of the annular bird's-eye view in step 2.2 is set to 40 rows and 60 columns.
  10. 根据权利要求5所述的一种融合点云地图、运动模型和局部特征的位置识别方法,其特征在于:步骤4.1中在地图参考点中搜索最近邻个数为20。 According to the location recognition method integrating point cloud map, motion model and local features as described in claim 5, it is characterized in that: in step 4.1, the number of nearest neighbors searched in the map reference point is 20.
PCT/CN2023/134918 2022-12-05 2023-11-29 Position recognition method for fusing point cloud map, motion model and local feature WO2024120269A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211552272.3 2022-12-05
CN202211552272.3A CN116127405A (en) 2022-12-05 2022-12-05 Position identification method integrating point cloud map, motion model and local features

Publications (1)

Publication Number Publication Date
WO2024120269A1 true WO2024120269A1 (en) 2024-06-13

Family

ID=86303510

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/134918 WO2024120269A1 (en) 2022-12-05 2023-11-29 Position recognition method for fusing point cloud map, motion model and local feature

Country Status (2)

Country Link
CN (1) CN116127405A (en)
WO (1) WO2024120269A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116127405A (en) * 2022-12-05 2023-05-16 武汉大学 Position identification method integrating point cloud map, motion model and local features

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180075643A1 (en) * 2015-04-10 2018-03-15 The European Atomic Energy Community (Euratom), Represented By The European Commission Method and device for real-time mapping and localization
US11002859B1 (en) * 2020-02-27 2021-05-11 Tsinghua University Intelligent vehicle positioning method based on feature point calibration
CN114353799A (en) * 2021-12-30 2022-04-15 武汉大学 Indoor rapid global positioning method for unmanned platform carrying multi-line laser radar
CN115388880A (en) * 2022-10-27 2022-11-25 联友智连科技有限公司 Low-cost memory parking map building and positioning method and device and electronic equipment
CN116127405A (en) * 2022-12-05 2023-05-16 武汉大学 Position identification method integrating point cloud map, motion model and local features

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180075643A1 (en) * 2015-04-10 2018-03-15 The European Atomic Energy Community (Euratom), Represented By The European Commission Method and device for real-time mapping and localization
US11002859B1 (en) * 2020-02-27 2021-05-11 Tsinghua University Intelligent vehicle positioning method based on feature point calibration
CN114353799A (en) * 2021-12-30 2022-04-15 武汉大学 Indoor rapid global positioning method for unmanned platform carrying multi-line laser radar
CN115388880A (en) * 2022-10-27 2022-11-25 联友智连科技有限公司 Low-cost memory parking map building and positioning method and device and electronic equipment
CN116127405A (en) * 2022-12-05 2023-05-16 武汉大学 Position identification method integrating point cloud map, motion model and local features

Also Published As

Publication number Publication date
CN116127405A (en) 2023-05-16

Similar Documents

Publication Publication Date Title
Yin et al. Locnet: Global localization in 3d point clouds for mobile vehicles
Huang Review on LiDAR-based SLAM techniques
Wang et al. Intelligent vehicle self-localization based on double-layer features and multilayer LIDAR
CN112014857A (en) Three-dimensional laser radar positioning and navigation method for intelligent inspection and inspection robot
CN114526745B (en) Drawing construction method and system for tightly coupled laser radar and inertial odometer
CN110132284B (en) Global positioning method based on depth information
US11790542B2 (en) Mapping and localization system for autonomous vehicles
Wang et al. Bionic vision inspired on-road obstacle detection and tracking using radar and visual information
Agrawal et al. Rough terrain visual odometry
Cao et al. Robust place recognition and loop closing in laser-based SLAM for UGVs in urban environments
WO2024120269A1 (en) Position recognition method for fusing point cloud map, motion model and local feature
CN108549376A (en) A kind of navigation locating method and system based on beacon
Chen et al. PSF-LO: Parameterized semantic features based LiDAR odometry
CN114088081B (en) Map construction method for accurate positioning based on multistage joint optimization
CN114325634A (en) Method for extracting passable area in high-robustness field environment based on laser radar
CN116592897B (en) Improved ORB-SLAM2 positioning method based on pose uncertainty
Shi et al. A fast LiDAR place recognition and localization method by fusing local and global search
Carrera et al. Lightweight SLAM and Navigation with a Multi-Camera Rig.
CN114593739A (en) Vehicle global positioning method and device based on visual detection and reference line matching
CN113759928A (en) Mobile robot high-precision positioning method for complex large-scale indoor scene
CN117053779A (en) Tightly coupled laser SLAM method and device based on redundant key frame removal
Ma et al. RoLM: Radar on LiDAR map localization
Cui et al. Precise landing control of UAV based on binocular visual SLAM
Wang et al. PDLC-LIO: A Precise and Direct SLAM System Toward Large-Scale Environments With Loop Closures
Wu et al. Visual data driven approach for metric localization in substation