CN114627309A - Visual SLAM method based on dotted line features in low texture environment - Google Patents
Visual SLAM method based on dotted line features in low texture environment Download PDFInfo
- Publication number
- CN114627309A CN114627309A CN202210241165.2A CN202210241165A CN114627309A CN 114627309 A CN114627309 A CN 114627309A CN 202210241165 A CN202210241165 A CN 202210241165A CN 114627309 A CN114627309 A CN 114627309A
- Authority
- CN
- China
- Prior art keywords
- point
- features
- line
- camera
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Abstract
A visual SLAM method based on point-line characteristics in a low-texture environment relates to the technical field of computer vision. The method is expanded on an ORB-SLAM2 system, the point features and the line features in the image are extracted, and the building structure line features are screened out on the basis of the common line features to obtain the point features and the building structure line features; performing feature matching by using the point features and the building structure line features, and eliminating mismatching pairs by using an RANSAC algorithm; estimating the pose of the camera according to a reprojection error model deduced from adjacent frames; and sequentially executing a back-end optimization thread and a loop detection thread, thereby improving the stability and robustness of the system in a low-texture scene, and finally realizing the visual SLAM method based on the dotted line characteristics in the low-texture environment.
Description
The technical field is as follows:
the invention relates to the technical field of computer vision, in particular to a visual SLAM method based on point-line characteristics in a low-texture environment.
Background art:
meanwhile, positioning And Mapping (SLAM) means that under the conditions of unknown environment And uncertain self position, data are acquired through a sensor to estimate self pose And map, a visual SLAM adopts a camera as the sensor to acquire rich information such as geometry, texture And color in the environment, positioning And map building are carried out by capturing characteristic information in an image, the information is used as a prerequisite condition for completing an intelligent task by a mobile robot, And the visual SLAM technology becomes a research hotspot in the robot field.
Although many mature visual SLAM systems are developed at present, many problems still exist in practical application, so that the visual SLAM technology is difficult to really popularize; for example, most of the existing visual SLAM algorithms use point features for tracking and matching, so that rich straight line information in an image is not fully utilized, when a camera is in an environment with missing textures (such as an open room, a corridor and the like), the number of the point features is sharply reduced and the point features are unevenly distributed, so that the tracking of the camera fails, or the problem that pose estimation and positioning cannot be performed due to the fact that sufficient feature matching pairs are not available occurs.
Compared with point features, the line features are higher-level features, have stronger robustness to illumination and visual angle changes, can express geometric information of a scene more visually, and can improve the stability and robustness of a system in a low-texture environment lacking in feature points.
The invention content is as follows:
the invention aims to provide a visual SLAM method based on dotted line features in a low-texture environment, which can improve the stability and robustness of a system in the low-texture environment.
The technical scheme for realizing the purpose of the invention is as follows:
a visual SLAM method based on point-line characteristics under a low-texture environment comprises the following steps:
the method comprises the following steps that (1) point features and line features in an image are extracted, building structure line features are screened out on the basis of common line features, and the point features and the building structure line features are obtained;
performing feature matching according to the utilization point features and the building structure line features, and eliminating mismatching pairs by using a RANSAC algorithm;
step (3) estimating the pose of the camera according to a reprojection error model deduced from adjacent frames;
and (4) sequentially executing a back-end optimization and loop detection thread, thereby improving the stability and robustness of the system in a low-texture scene, and finally realizing the visual SLAM method based on the dotted line characteristics and oriented to the low-texture scene.
Preferably, the specific method of step (1) of the present invention is as follows: firstly, ORB point characteristics and LSD line characteristics in an image are extracted, binary descriptors of the characteristics are calculated, and then building structure line characteristics which accord with the main direction of the Manhattan world are screened out on the basis of common line characteristics.
Preferably, in the online feature screening stage, the method calculates the dominant direction of the Manhattan world through the vanishing point, and screens the structural line features consistent with the dominant direction in the common line features; in a three-dimensional space, a group of parallel straight lines cannot intersect, or intersect at the same infinity point, the infinity point may be imaged in an image plane under the perspective projection effect of a camera, an imaged point is called a vanishing point, all building structure lines in a leading direction can be oriented by using one vanishing point, the middle point of a line segment is respectively connected with each vanishing point to obtain a corresponding reference straight line, and if the distance between the line segment and one reference straight line is close, the line segment is considered to belong to the vanishing point and simultaneously belongs to the leading direction corresponding to the vanishing point.
Preferably, the specific method of step (2) of the present invention is as follows: by using the descriptors of the ORB point features and the LSD line features of the extracted images, firstly, the descriptors are used for feature matching, and then, on the basis, the RANSAC algorithm is used for carrying out mismatching elimination.
Preferably, the method uses RANSAC algorithm to carry out mismatch elimination, the RANSAC algorithm assumes that data contains correct data and abnormal data (noise), and assumes a method capable of calculating model parameters conforming to the correct data.
Preferably, in step (3) of the present invention, the pose of the camera is estimated according to a re-projection error model derived from adjacent frames, and the specific method is as follows:
and (3) combining the reprojection errors of the space point characteristics and the space line characteristics, increasing the reprojection errors of the building structure line characteristics on the basis, obtaining the sum of the errors of all the space points, the space lines and the structure lines in the three-dimensional space, and solving the camera pose by using the reprojection errors.
Preferably, the method for sequentially executing the back-end optimization and loop detection threads in step (4) of the present invention is as follows:
the back-end optimization comprises local map-based optimization and loop detection-based optimization; optimizing the local map, firstly judging whether the current frame is a key frame, if so, establishing a local map associated with the current key frame, and then establishing a map optimization model according to the local map to optimize the pose of the camera; and firstly judging whether the current key frame is a loop frame based on loop detection optimization, and if so, constructing a pose graph optimization model according to loop information so as to eliminate the accumulated error of the camera.
Preferably, the vanishing point in three-dimensional space is determined using equation (1):
uTv＝0 (1)
where u represents a 3 × N matrix, N is the number of straight lines, and v represents the 3 × 1 homogeneous coordinate of the vanishing point being found.
Preferably, the direction of the structural line in the world coordinate system is calculated using formula (2):
η∞RwcK-1v (2)
where η is the direction of the structure line in the world coordinate system, also called the dominant direction, RwcAs a rotation matrix from the camera coordinate system to the world coordinate system, K-1Is the inverse of the camera's intrinsic parameters.
Preferably, the vanishing point in the image is calculated using equation (3):
vi＝KRcwηi (3)
wherein v isiIs a vanishing point in the image, K is camera internal parameter, RcwIs a rotation matrix from the world coordinate system to the camera coordinate system, ηiIs the dominant direction under the world coordinate system.
Preferably, the inter-frame reprojection error sum is calculated using equation (4):
wherein the k frame pose of the camera is Tcw,kI-th spatial point p observed in the k-th framew,iReprojection error Epk,iThe jth space line L is observed in the kth framew,jWith a reprojection error of Elk,jThe k frame camera maintains a pose of RkThe k frame camera has a pose Z estimated by the structured linekΣ p and Σ l denote the observed covariance of the point-and-line, ρp、ρl、ρsIs a Huber robust kernel function.
Compared with the prior art, the method introduces line characteristics on the basis of the ORB-SLAM2 algorithm, screens out building structure line characteristics containing building structure information on the basis of the traditional line characteristics, uses the point characteristics and the building structure line characteristics to carry out pose estimation, and can improve the pose estimation precision.
The invention can estimate the pose by combining the point characteristics and the building structure line characteristics in the environment, reduce the pose estimation error of the system and improve the robustness of the system in the structured and low-texture environment.
Drawings
FIG. 1 is a schematic flow diagram of the present invention;
FIG. 2 is a schematic diagram of a structure consistent with the Manhattan world hypothesis in the method of the present invention;
FIG. 3 is a schematic diagram of the extraction point features and building structure line features in the method of the present invention:
FIG. 4 is a schematic diagram of a map and a camera track constructed by the method of the present invention.
Detailed Description
The present invention will be further described with reference to the following examples and drawings, but the present invention is not limited to the embodiments.
As shown in fig. 1, a visual SLAM method based on dotted line features in a low texture environment is implemented as follows:
the method comprises the following steps that (1) point features and line features in an image are extracted, building structure line features are screened out on the basis of common line features, and the point features and the building structure line features are obtained;
the specific process is as follows:
firstly, ORB point characteristics and LSD line characteristics in an image are extracted, binary descriptors of the characteristics are calculated, and then building structure line characteristics which accord with the main direction of the Manhattan world are screened out on the basis of common line characteristics.
In the online characteristic screening stage, the method calculates the dominant direction of the Manhattan world through the vanishing point, and screens the structural line characteristics which are consistent with the dominant direction in the common line characteristics; in a three-dimensional space, a group of parallel straight lines cannot intersect, or intersect at the same infinity point, the infinity point may be imaged in an image plane under the perspective projection effect of a camera, an imaged point is called a vanishing point, all building structure lines in a leading direction can be oriented by using one vanishing point, the middle point of a line segment is respectively connected with each vanishing point to obtain a corresponding reference straight line, and if the distance between the line segment and one reference straight line is close, the line segment is considered to belong to the vanishing point and simultaneously belongs to the leading direction corresponding to the vanishing point.
Step (2) performing feature matching by using the point features and the building structure line features, and eliminating mismatching pairs by using a RANSAC algorithm;
the specific process is as follows:
the method comprises the steps of firstly using descriptors to carry out feature matching by using descriptors for extracting ORB point features and LSD line features of an image, then using RANSAC algorithm to carry out mismatching elimination on the basis, using RANSAC algorithm to carry out mismatching elimination, assuming that data comprises correct data and abnormal data (noise) by RANSAC algorithm, assuming a method capable of calculating model parameters conforming to the correct data, wherein the algorithm has the characteristics of randomness and hypothesis, randomly selecting sample data according to the probability of the correct data, randomly simulating approximate correct results, assuming that the sample data selected by the method are correct data, and calculating other points by using the data to obtain the best model parameters.
Step (3) estimating the pose of the camera according to a reprojection error model deduced from adjacent frames;
the specific process is as follows:
and (3) combining the reprojection errors of the space point characteristics and the space line characteristics, increasing the reprojection errors of the building structure line characteristics on the basis, obtaining the sum of the errors of all the space points, the space lines and the structure lines in the three-dimensional space, and solving the camera pose by using the reprojection errors. .
And (4) sequentially executing a back-end optimization and loop detection thread, so that the stability and robustness of the system in a low-texture scene are improved, and finally, the low-texture scene-oriented visual SLAM method based on the dotted line features is realized.
The specific process is as follows:
the back-end optimization comprises local map-based optimization and loop detection-based optimization; optimizing the local map, firstly judging whether the current frame is a key frame, if so, establishing a local map associated with the current key frame, and then establishing a map optimization model according to the local map to optimize the pose of the camera; and firstly judging whether the current key frame is a loop frame based on loop detection optimization, and if so, constructing a pose graph optimization model according to loop information so as to eliminate the accumulated error of the camera.
Determining a vanishing point in three-dimensional space using equation (1):
uTv＝0 (1)
where u represents a 3 × N matrix, N is the number of straight lines, and v represents the 3 × 1 homogeneous coordinate of the vanishing point being found.
And (3) calculating the direction of the structural line in the world coordinate system by adopting the formula (2):
η∞RwcK-1v (2)
where η is the direction of the structure line in the world coordinate system, also called the dominant direction, RwcAs a rotation matrix from the camera coordinate system to the world coordinate system, K-1Is the inverse of the camera's intrinsic parameters.
The vanishing point in the image is calculated using equation (3):
vi＝KRcwηi (3)
wherein v isiIs a vanishing point in the image, K is camera internal parameter, RcwIs a rotation matrix from the world coordinate system to the camera coordinate system, ηiIs the dominant direction under the world coordinate system.
Calculating the sum of the frame-to-frame reprojection errors by using formula (4):
wherein the k frame pose of the camera is Tcw,kI-th spatial point p observed in the k-th framew,iReprojection error Epk,iThe kth frame observes the jth space line Lw,jWith a reprojection error of Elk,jThe k frame camera maintains a pose of RkThe k frame camera has an estimated attitude Z through the structured linekΣ p and Σ l denote the observed covariance of the point-and-line, ρp、ρl、ρsIs a Huber robust kernel function.
Example 1
A visual SLAM method based on point-line features in a low-texture environment comprises the following steps:
step 1, firstly, extracting ORB point characteristics and LSD line characteristics in an image, calculating a binary descriptor of the characteristics, and screening out building structure line characteristics conforming to the principal direction of the Manhattan world on the basis of common line characteristics; in the online characteristic screening stage, the method calculates the dominant direction of the Manhattan world through the vanishing point, and screens the structural line characteristics which are consistent with the dominant direction in the common line characteristics.
And 2, by using descriptors of ORB point features and LSD line features of the extracted image, firstly, performing feature matching by using the descriptors, and then performing mismatching elimination by using a RANSAC algorithm on the basis.
And 3, integrating the reprojection errors of the space point characteristics and the space line characteristics, increasing the reprojection errors of the building structure line characteristics on the basis, obtaining the error sum of all the space points, the space lines and the structure lines in the three-dimensional space, and solving the camera pose by using the reprojection errors.
Step 4, the back-end optimization comprises optimization based on a local map and optimization based on loop detection; optimizing the local map, firstly judging whether the current frame is a key frame, if so, establishing a local map associated with the current key frame, and then establishing a map optimization model according to the local map to optimize the pose of the camera; and firstly judging whether the current key frame is a loop frame or not based on loop detection optimization, and if so, constructing a pose graph optimization model according to loop information so as to eliminate the accumulated error of the camera.
As shown in fig. 1, the present embodiment can improve the pose estimation accuracy by performing pose estimation by combining point features in the environment with building structure line features; a globally consistent map is constructed through a back-end optimization and loop detection thread, so that the robustness of the system in a structured and low-texture environment is improved.
Claims (9)
1. A visual SLAM method based on dotted line features in a low-texture environment is characterized by comprising the following steps:
the method comprises the following steps that (1) point features and line features in an image are extracted, building structure line features are screened out on the basis of common line features, and the point features and the building structure line features are obtained;
performing feature matching by using point features and building structure line features, and eliminating mismatching pairs by using a RANSAC algorithm;
step (3) estimating the pose of the camera according to a reprojection error model deduced from adjacent frames;
and (4) sequentially executing a back-end optimization and loop detection thread, thereby improving the stability and robustness of the system in a low-texture environment and finally realizing the visual SLAM method based on the point-line characteristics in the low-texture environment.
2. The visual SLAM method based on dotted line features in a low texture environment of claim 1, wherein: the specific process of the step (1) is as follows:
firstly, extracting ORB point characteristics and LSD line characteristics in an image, calculating a binary descriptor of the characteristics, and screening out building structure line characteristics conforming to the principal direction of the Manhattan world on the basis of common line characteristics; in the online characteristic screening stage, the method calculates the dominant direction of the Manhattan world through the vanishing point, and screens the structural line characteristics which are consistent with the dominant direction in the common line characteristics; in a three-dimensional space, a group of parallel straight lines cannot intersect, or intersect at the same infinity point, the infinity point may be imaged in an image plane under the perspective projection effect of a camera, an imaged point is called a vanishing point, all building structure lines in a leading direction can be oriented by using one vanishing point, the middle point of a line segment is respectively connected with each vanishing point to obtain a corresponding reference straight line, and if the distance between the line segment and one reference straight line is close, the line segment is considered to belong to the vanishing point and simultaneously belongs to the leading direction corresponding to the vanishing point.
3. The visual SLAM method based on dotted line features in a low texture environment of claim 1, wherein: the specific process of the step (2) is as follows:
by using descriptors of ORB point features and LSD line features of the extracted image, firstly, performing feature matching by using the descriptors, and then performing mismatching elimination by using a RANSAC algorithm on the basis; the RANSAC algorithm assumes that data contains correct data and abnormal data (noise), and assumes a method capable of calculating model parameters conforming to the correct data, the algorithm has the characteristics of randomness and hypothesis, the randomness is that sampling data is randomly selected according to the probability of the occurrence of the correct data, the randomness simulates approximate correct results, the hypothesis is that the sampling data selected are correct data, and other points are calculated by using the data to obtain the best model parameters.
4. The visual SLAM method based on dotted line features in a low texture environment of claim 1, wherein: and (3) estimating the pose of the camera according to the reprojection error model deduced from the adjacent frames, wherein the specific process is as follows:
and (3) combining the reprojection errors of the space point characteristics and the space line characteristics, increasing the reprojection errors of the building structure line characteristics on the basis, obtaining the sum of the errors of all the space points, the space lines and the structure lines in the three-dimensional space, and solving the camera pose by using the sum of the errors.
5. The visual SLAM method based on dotted line features in a low texture environment of claim 1, wherein: the method for sequentially executing the back-end optimization and loop detection threads in the step (4) is as follows:
the back-end optimization comprises local map-based optimization and loop detection-based optimization; optimizing the local map, firstly judging whether the current frame is a key frame, if so, establishing a local map associated with the current key frame, and then establishing a map optimization model according to the local map to optimize the pose of the camera; and firstly judging whether the current key frame is a loop frame based on loop detection optimization, and if so, constructing a pose graph optimization model according to loop information so as to eliminate the accumulated error of the camera.
6. The visual SLAM method in low texture environment based on dotted line features of claim 2, wherein the vanishing point in three-dimensional space is determined using formula (1):
uTv＝0 (1)
where u represents a 3 × N matrix, N is the number of straight lines, and v represents the 3 × 1 homogeneous coordinate of the vanishing point being found.
7. The visual SLAM method based on dotted line features in low texture environment as claimed in claim 2, wherein the direction of the structural lines in the world coordinate system is calculated by formula (2):
η∞RwcK-1v (2)
where η is the direction of the structure line in the world coordinate system, also called the dominant direction, RwcAs a rotation matrix from the camera coordinate system to the world coordinate system, K-1Is the inverse of the camera's internal reference.
8. The visual SLAM method based on dotted line features in low texture environment as claimed in claim 2, wherein formula (3) is used to calculate vanishing points in the image:
vi＝KRcwηi (3)
wherein v isiIs a vanishing point in the image, K is camera internal parameter, RcwIs a rotation matrix from the world coordinate system to the camera coordinate system, ηiIs the dominant direction under the world coordinate system.
9. The visual SLAM method based on dotted line features in a low texture environment as claimed in claim 4 wherein the sum of the frame-to-frame reprojection errors is calculated using equation (4):
wherein the k frame pose of the camera is Tcw,kI-th spatial point p observed in the k-th framew,iReprojection error Epk,iThe jth space line L is observed in the kth framew,jWith a reprojection error of Elk,jThe k frame camera maintains a pose of RkThe k frame camera has an estimated attitude Z through the structured linekΣ p and Σ l denote the observed covariance of the point-and-line, ρp、ρl、ρsIs a Huber robust kernel function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210241165.2A CN114627309A (en) | 2022-03-11 | 2022-03-11 | Visual SLAM method based on dotted line features in low texture environment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210241165.2A CN114627309A (en) | 2022-03-11 | 2022-03-11 | Visual SLAM method based on dotted line features in low texture environment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114627309A true CN114627309A (en) | 2022-06-14 |
Family
ID=81902070
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210241165.2A Pending CN114627309A (en) | 2022-03-11 | 2022-03-11 | Visual SLAM method based on dotted line features in low texture environment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114627309A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116468786A (en) * | 2022-12-16 | 2023-07-21 | 中国海洋大学 | Semantic SLAM method based on point-line combination and oriented to dynamic environment |
-
2022
- 2022-03-11 CN CN202210241165.2A patent/CN114627309A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116468786A (en) * | 2022-12-16 | 2023-07-21 | 中国海洋大学 | Semantic SLAM method based on point-line combination and oriented to dynamic environment |
| CN116468786B (en) * | 2022-12-16 | 2023-12-26 | 中国海洋大学 | Semantic SLAM method based on point-line combination and oriented to dynamic environment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106909888B (en) | Face key point tracking system and method applied to mobile equipment terminal | |
| CN110060277A (en) | A kind of vision SLAM method of multiple features fusion | |
| CN110490158B (en) | Robust face alignment method based on multistage model | |
| CN109974743B (en) | Visual odometer based on GMS feature matching and sliding window pose graph optimization | |
| CN105374049B (en) | Multi-corner point tracking method and device based on sparse optical flow method | |
| CN110825900A (en) | Training method of feature reconstruction layer, reconstruction method of image features and related device | |
| CN110531618B (en) | Closed loop detection robot self-positioning error elimination method based on effective key frame | |
| CN107403451B (en) | Self-adaptive binary characteristic monocular vision odometer method, computer and robot | |
| CN109784155B (en) | Visual target tracking method based on verification and error correction mechanism and intelligent robot | |
| CN114529593A (en) | Infrared and visible light image registration method, system, equipment and image processing terminal | |
| CN112509044A (en) | Binocular vision SLAM method based on dotted line feature fusion | |
| CN112132874A (en) | Calibration-board-free different-source image registration method and device, electronic equipment and storage medium | |
| CN110599522A (en) | Method for detecting and removing dynamic target in video sequence | |
| CN114119732A (en) | Joint optimization dynamic SLAM method based on target detection and K-means clustering | |
| CN112652020A (en) | Visual SLAM method based on AdaLAM algorithm | |
| CN112329764A (en) | Infrared dim target detection method based on TV-L1 model | |
| CN114627309A (en) | Visual SLAM method based on dotted line features in low texture environment | |
| CN112308128A (en) | Image matching method based on attention mechanism neural network | |
| CN112614167A (en) | Rock slice image alignment method combining single-polarization and orthogonal-polarization images | |
| CN111914832B (en) | SLAM method of RGB-D camera under dynamic scene | |
| CN109584269A (en) | A kind of method for tracking target | |
| CN109472813A (en) | It is a kind of based on background weighting Mean Shift algorithm and Kalman prediction fusion block tracking | |
| CN117315547A (en) | Visual SLAM method for solving large duty ratio of dynamic object | |
| CN111767826A (en) | Timing fixed-point scene abnormity detection method | |
| CN112183287A (en) | People counting method of mobile robot under complex background |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |