CN117554984A - A single-line lidar indoor SLAM positioning method and system based on image understanding - Google Patents

Abstract

The invention discloses a single-line lidar indoor SLAM positioning method and system based on image understanding, which belongs to the technical field of robot autonomous navigation and includes the following steps: S1. Select an experimental platform and design the hardware of the robot; S2. Perform the input sensor data Acquisition and preprocessing; S3, implement local SLAM mapping; S4, implement global SLAM mapping, and complete indoor SLAM positioning; by using RGB-D cameras to capture images and depth information, this invention can extract semantic features from images and build an environment The visual map facilitates the registration of consecutive frames obtained by assisting single-line lidar, and achieves accurate and robust positioning and mapping in indoor environments. It is superior to the traditional single-line lidar SLAM method without image interpretation and incorporates graph optimization. The laser SLAM algorithm and the understanding of environmental depth information achieve accurate and robust positioning and mapping in indoor environments, with stronger applicability.

Description

A single-line lidar indoor SLAM positioning method and system based on image understanding

Technical field

The present invention relates to the technical field of robot autonomous navigation, and in particular to a single-line laser radar indoor SLAM positioning method and system based on image understanding.

Background technique

Simultaneous positioning and map construction have been widely used in various fields, such as robotics, autonomous driving, and virtual reality, and indoor SLAM is particularly important for applications such as indoor navigation, building inspection, and rescue missions. Due to its low cost and high accuracy, single-line lidar has become a commonly used sensor in indoor SLAM. However, its limited field of view makes it difficult to obtain comprehensive 3D point clouds. In addition, the traditional single-line lidar SLAM method is prone to drift and requires frequent closed-loop correction.

There are many types of SLAM algorithms, and there are different solutions for different problems. Nonlinear optimization multi-threading is used in SLAM. This algorithm successfully solves the real-time attitude estimation problem of SLAM, but it lacks loop detection function and is only suitable for building small-scale maps; Some low-drift SLAM methods based on point-line-plane pose estimation for indoor scenes have successfully solved the SLAM mapping problem of low-texture indoor scenes, but their recognition rate is still low in complex scenes; and the lili-om solid-state laser Tightly coupled mapping of solid-state and mechanical lidar is implemented in the radar inertial ranging SLAM solution. This solution designs a new feature extraction method for the irregular and unique scanning pattern of Livox Horizon, improving feature extraction in complex scenes. Tracking and mapping accuracy. However, because it uses Livox Horizon, it doesn't come cheap.

Analyzing existing related lidar SLAM positioning algorithms and technologies, the semantic SLAM method enables the SLAM system to obtain a higher level of scene understanding, but there are certain limitations in practical application.

In response to the above problems, this invention document proposes a single-line lidar indoor SLAM positioning method and system based on image understanding.

Contents of the invention

The present invention provides a single-line laser radar indoor SLAM positioning method and system based on image understanding, which solves the above-mentioned problems.

The present invention provides the following technical solutions:

A single-line lidar indoor SLAM positioning method based on image understanding, including the following steps:

S1. Select the experimental platform and design the robot hardware;

S2. Collect and preprocess the data input from the sensor;

S3. Implement local SLAM mapping;

S4. Implement global SLAM mapping and complete indoor SLAM positioning.

In a possible design, in step S2, the specific steps are:

S2.1. Preprocess the laser ranging data collected by the single-line lidar through the voxel filter and the adaptive voxel filter, and input the results into the local SLAM module;

S2.2. Send the image frames captured by the RGB-D camera to the YOLOv5 target detection network to detect a priori dynamic targets, obtain the next frame, extract semantic features, and output accurate camera pose estimation;

S2.3. Then use the inertial tracker to preprocess the IMU data, and send the results to the pose extrapolator together with the odometry pose data and camera pose estimation.

In a possible design, in step S3, the specific steps are:

S3.1. Calculate attitude estimation using odometry and IMU data;

S3.2. Use the attitude estimate as the initial guess, match the lidar data and update the value of the attitude estimator;

S3.3. After performing motion filtering on each frame of lidar data, the frames are accumulated and combined to form a subgraph.

In one possible design, step S4 includes the following steps:

S4.1. Insert the new scan frame and all previous scan frames into the completed subgraph;

S4.2. Then use the optimization algorithm branch and bound to detect loops;

S4.3. Calculate the constraint relationship between postures based on the detected loop constraints;

S4.4. Finally, use the attitude optimization algorithm to optimize all constraints to obtain more accurate attitude estimation results.

A system applied to a single-line lidar indoor SLAM positioning method based on image understanding. The system includes: a sensor data input module, a semantic detection module, a local SLAM module and a global SLAM module;

The sensor data input module is used to input data to the semantic detection module, the local SLAM module and the global SLAM module;

The semantic detection module is used to extract semantic information and detect dynamic targets in the scene;

The local SLAM module is used to construct a local map and update attitude estimation;

The global SLAM module is used to detect and perform global optimization to eliminate accumulated errors.

In a possible design, the input data of the sensor data input module includes laser scanning data, odometry attitude data, IMU measurement data, fixed frame attitude data and image frames.

It should be understood that the above general description and the following detailed description are only exemplary and do not limit the present invention.

By using an RGB-D camera to capture images and depth information, the present invention can extract semantic features from images, build a visual map of the environment, facilitate the registration of continuous frames obtained by assisting single-line lidar, and achieve accurate detection in indoor environments. Robust positioning and mapping, which is better than the traditional single-line lidar SLAM method without image interpretation. It integrates the graph-optimized laser SLAM algorithm and the understanding of environmental depth information to achieve accurate and robust positioning and mapping in indoor environments. Applicability Stronger.

Description of the drawings

Figure 1 is a schematic flow chart of the single-line lidar indoor SLAM positioning method based on image understanding provided by an embodiment of the present invention;

Figure 2 is a schematic diagram of the robot hardware design provided by the embodiment of the present invention;

Figure 3 is a schematic diagram of mapping effect evaluation and comparison results provided by an embodiment of the present invention.

Detailed ways

The embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention.

In order to make the above objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, the present invention can be implemented in many other ways different from those described here. Those skilled in the art can make similar improvements without departing from the connotation of the present invention. Therefore, the present invention is not limited to the specific implementation disclosed below.

Example 1

Referring to Figure 1, a system in this embodiment is applied to the single-line lidar indoor SLAM positioning method based on image understanding. The system is based on the Cartographer framework and includes a sensor data input module, a semantic detection module, a local SLAM module and a global SLAM module. .

The sensor data input module is used to input data to the semantic detection module, local SLAM module and global SLAM module. The sensor data input module uses laser scanning data, odometer attitude, IMU measurement data and fixed frame attitude as input sensor data, and also adds Image frames captured by RGB-D cameras are used to extract semantic features and depth information from these images;

In the processing of laser scanning data, the entire positioning and mapping process is based on optimizing the robot's attitude estimation and map occupation grid probability through scanning data. In the process, the Cartographer system first preprocesses the scanning data through two voxel filters, and then It is input into the local SLAM module, and the SLAM algorithm is used to construct a local map and generate sub-maps, so that the Cartographer system can efficiently and accurately map and position the indoor environment, providing reliable technical support for applications such as indoor navigation and mobile robots;

Odometer attitude data, IMU measurement data and fixed frame attitude data are mainly used to provide better initial values for the optimization solution process. IMU data is the main part. Through the use of IMU trackers and attitude extrapolators, it provides initial values for local SLAM. Pose estimation, and the raw data from these sensors will also be used in the sparse pose adjustment of global SLAM.

The semantic detection module is used to extract semantic information and detect dynamic targets in the scene. The extracted semantic information can be used to identify dynamic targets in the scene, thereby improving the robustness of SLAM in dynamic environments. In the semantic detection module, YOLOv5 is used As a semantic detection network to extract semantic information, it can detect small targets and ensure real-time performance while improving accuracy, and YOLOv5 contains more than 80 different categories of targets, and the training category is very suitable for indoor SLAM scenes;

The input image frame is semantically detected using the YOLOv5 target detection network. During the detection process, only a priori dynamic targets are detected, and other targets are not detected. The detection results are passed to the tracking thread for preprocessing, and features on the dynamic targets are removed. point to obtain accurate camera pose estimation.

The local SLAM module is used to build a local map and update the attitude estimate. The input of the process is the initial attitude generated by the attitude extrapolator, and the output is the local subgraph. Its implementation includes:

Scan matching, motion filtering static and garbage discard components;

Scan matching: The goal of this module is to calculate the optimal posture at the current moment relative to the previous period of time, using a nonlinear optimization method to establish a least squares problem, with the initial posture obtained by the extrapolator as input, and the output is the optimal scan matching Attitude, Google's Ceres library is used in Cartographer to solve the least squares problem;

Motion Filter: The goal of this module is to reduce the number of scan frames inserted into each subgraph. Once the scan matcher generates a new pose, calculate the change of this pose from the last pose and then call the motion filter if the pose If the change is not obvious or too small, the scan will be removed. Only when the distance, angle or time threshold between poses changes significantly, the scan will be inserted into the current subgraph to ensure that the number of scans in the subgraph is not too many. , also reduces noise and errors in the map;

Subgraph: Whenever a scan data is obtained, it is matched with the recently established subgraph to insert the scan data of this frame into the best position in the subgraph. While new data frames are continuously inserted, the subgraph is also Update, a certain amount of data is combined into a subgraph, and when no new scans are inserted into the subgraph, the subgraph is considered complete and the algorithm creates the next subgraph.

The global SLAM module is used to detect and perform global optimization to eliminate accumulated errors. The input is the new scan frame and the completed subgraph, and the output is the optimized pose estimation. Its implementation includes:

Computational constraints, sparse pose adjustment and loop detection;

Computational constraints: Computational constraints refer to establishing the relationship between adjacent frames based on various forms of information, such as pose differences, visual feature matching, IMU data, etc. These constraints will be used as input to the optimizer to estimate the trajectory of the camera and robot. and adjust the map;

Sparse pose adjustment: If a good match is obtained, the loop detection process ends and the existence of a loop is detected. Then, all poses in the subgraph are optimized based on the current scan pose and the pose that best matches it in the subgraph. The target It is to minimize the residual error. The loop optimization problem is also a nonlinear least squares problem, which is solved using Google's Ceres library in Cartographer;

Loop detection: During loop detection, all created subgraphs and the current laser scan are used for matching. If the current scan and all completed subgraphs are close enough, a suitable matching strategy can find the loop closure, which reduces To reduce computational complexity and improve real-time loop detection efficiency, branch-and-bound optimization methods are applied for efficient search.

Example 2

Referring to Figures 1-3, a single-line lidar indoor SLAM positioning method based on image understanding includes the following steps:

S1. Select the experimental platform and design the robot’s hardware:

A 64-bit Ubuntu system is selected. The robot's hardware sensors include: Raspberry Pi 4B for the upper motherboard; NVIDIA Jetson NanoB01 for the bottom motherboard; STM32F103RET6 chip for the main control; integrated Silan A1 single-line lidar, nine-axis IMU (MPU9250), and odometer. ; At the same time, it adopts Ackerman steering chassis, and the rear wheels of the chassis are driven by 2 DC motors with encoders.

S2. Collect and preprocess the input sensor data:

S2.1. Preprocess the laser ranging data collected by the single-line lidar through the voxel filter and the adaptive voxel filter, and input the results into the local SLAM module;

S2.2. Send the image frames captured by the RGB-D camera to the YOLOv5 target detection network to detect a priori dynamic targets, obtain the next frame, extract semantic features, and output accurate camera pose estimation;

S2.3. Then use the inertial tracker to preprocess the IMU data, and send the results to the pose extrapolator together with the odometry pose data and camera pose estimation.

S3. Implement local SLAM mapping:

S3.1. Calculate attitude estimation using odometry and IMU data;

S3.2. Use the attitude estimate as the initial guess, match the lidar data and update the value of the attitude estimator;

S3.3. After performing motion filtering on each frame of lidar data, the frames are accumulated and combined to form a subgraph.

S4. Implement global SLAM mapping and complete indoor SLAM positioning:

S4.1. Insert the new scan frame and all previous scan frames into the completed subgraph;

S4.2. Then use the optimization algorithm branch and bound to detect loops;

S4.3. Calculate the constraint relationship between postures based on the detected loop constraints;

S4.4. Finally, use the attitude optimization algorithm to optimize all constraints to obtain more accurate attitude estimation results.

The method and system capture images using RGB-D cameras, extract semantic features and depth information from the images, and build a visual map of the environment to help the registration of continuous frames obtained by single-line lidar, and are ultimately implemented in indoor environments. Accurate and robust positioning and mapping enable robots to autonomously position and map complex indoor scenes.

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all of them should be covered. Within the scope of the present invention; without conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims (6)

1. A single-line lidar indoor SLAM positioning method based on image understanding, which is characterized by including the following steps: S1. Select the experimental platform and design the robot hardware; S2. Collect and preprocess the data input from the sensor; S3. Implement local SLAM mapping; S4. Implement global SLAM mapping and complete indoor SLAM positioning. 2. A single-line lidar indoor SLAM positioning method based on image understanding according to claim 1, characterized in that in step S2, the specific steps are: S2.1. Preprocess the laser ranging data collected by the single-line lidar through the voxel filter and the adaptive voxel filter, and input the results into the local SLAM module; S2.2. Send the image frames captured by the RGB-D camera to the YOLOv5 target detection network to detect a priori dynamic targets, obtain the next frame, extract semantic features, and output accurate camera pose estimation; S2.3. Then use the inertial tracker to preprocess the IMU data, and send the results to the pose extrapolator together with the odometry pose data and camera pose estimation. 3. A single-line lidar indoor SLAM positioning method based on image understanding according to claim 1, characterized in that in step S3, the specific steps are: S3.1. Calculate attitude estimation using odometry and IMU data; S3.2. Use the attitude estimate as the initial guess, match the lidar data and update the value of the attitude estimator; S3.3. After performing motion filtering on each frame of lidar data, the frames are accumulated and combined to form a subgraph. 4. A single-line lidar indoor SLAM positioning method based on image understanding according to claim 1, characterized in that, in step S4, the following steps are: S4.1. Insert the new scan frame and all previous scan frames into the completed subgraph; S4.2. Then use the optimization algorithm branch and bound to detect loops; S4.3. Calculate the constraint relationship between postures based on the detected loop constraints; S4.4. Finally, use the attitude optimization algorithm to optimize all constraints to obtain more accurate attitude estimation results. 5. A system applied to the single-line lidar indoor SLAM positioning method based on image understanding, characterized in that the system includes: a sensor data input module, a semantic detection module, a local SLAM module and a global SLAM module; The sensor data input module is used to input data to the semantic detection module, the local SLAM module and the global SLAM module; The semantic detection module is used to extract semantic information and detect dynamic targets in the scene; The local SLAM module is used to construct a local map and update attitude estimation; The global SLAM module is used to detect and perform global optimization to eliminate accumulated errors. 6. A single-line lidar indoor SLAM positioning method based on image understanding according to claim 5, characterized in that the input data of the sensor data input module includes laser scanning data, odometer attitude data, IMU measurement data, Fixed frame pose data and image frames.