CN110853100A - Structured scene vision SLAM method based on improved point-line characteristics - Google Patents

Structured scene vision SLAM method based on improved point-line characteristics Download PDF

Info

Publication number
CN110853100A
CN110853100A CN201911015560.3A CN201911015560A CN110853100A CN 110853100 A CN110853100 A CN 110853100A CN 201911015560 A CN201911015560 A CN 201911015560A CN 110853100 A CN110853100 A CN 110853100A
Authority
CN
China
Prior art keywords
point
line
feature
camera
structured scene
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201911015560.3A
Other languages
Chinese (zh)
Other versions
CN110853100B (en
Inventor
张小国
刘启汉
王慧青
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southeast University
Original Assignee
Southeast University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southeast University filed Critical Southeast University
Priority to CN201911015560.3A priority Critical patent/CN110853100B/en
Publication of CN110853100A publication Critical patent/CN110853100A/en
Application granted granted Critical
Publication of CN110853100B publication Critical patent/CN110853100B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/05—Geographic models

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Remote Sensing (AREA)
  • Computer Graphics (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a structured scene vision SLAM method based on improved dotted line characteristics, which comprises the steps of firstly, conducting basic calibration on an RGB-D camera to obtain internal reference information, and then conducting SLAM initialization on a structured scene through a depth camera; extracting ORB point features and LSD line features in the structured scene, establishing an error model according to space points and space straight lines corresponding to the point line features, and estimating the pose of the camera by minimizing the model to generate three-dimensional map points of the structured scene; making a decision to generate a key frame in a video frame, establishing a bag-of-words model by using a key frame set, and carrying out closed-loop detection on the three-dimensional map points; and after a closed-loop condition is detected, optimizing the camera pose and the three-dimensional map points of the structured scene through an error model, and improving the SLAM effect. The method solves the problem that the precision and the efficiency of the visual SLAM in the structured scene are not high for closed-loop detection, and provides great convenience for the visual SLAM.

Description

Structured scene vision SLAM method based on improved point-line characteristics
Technical Field
The invention belongs to the technical field of machine vision, and particularly relates to a structured scene vision SLAM method based on improved point-line characteristics.
Background
With the rapid development of the internet economy, the industries such as logistics, express delivery, machining production and the like develop rapidly. Due to the fact that the workload is large, the work task is single, and the requirement on work accuracy is high, an industrial robot is often selected to complete specific work. In the field of image measurement processes of robots and machine vision, there is an increasing demand for positioning and mapping in structured scenes.
Meanwhile, machine vision technology based on vision synchronous positioning and mapping (vision SLAM) is increasingly perfected, and compared with the traditional mode, the machine vision technology based on vision synchronous positioning and mapping has great progress in processing speed, positioning and mapping precision. However, at present, a complete set of visual SLAM technology aiming at indoor structured scenes such as express sorting, unmanned logistics transportation and the like does not exist. This also leads to the problem that the simultaneous positioning and mapping of the robot in such scenarios is inefficient and susceptible to environmental factors with low accuracy. Therefore, the requirement for improving the positioning and mapping precision under the structured scene is increasingly highlighted while the speed of the robot positioning and mapping work is ensured.
Disclosure of Invention
The purpose of the invention is as follows: in order to solve the problems that the robot is low in efficiency of simultaneous positioning and mapping and is easily influenced by environmental factors and low in precision, the invention provides a structured scene vision SLAM method based on improved point-line characteristics, and great convenience is provided for simultaneous positioning and mapping in some types of environments.
The technical scheme is as follows: in order to realize the purpose of the invention, the technical scheme adopted by the invention is as follows: a structured scene vision SLAM method based on improved dotted line characteristics comprises the following steps:
s1, performing basic calibration on the RGB-D depth camera to obtain internal reference information; visual synchronous positioning and image building (visual SLAM) initialization are carried out on the structured scene through the depth camera;
s2, extracting ORB point features and LSD line features in the structured scene through the video frame obtained by the camera, and respectively corresponding to the space points and the space straight lines in the structured scene;
s3, establishing an error model based on the improved point characteristics and the line characteristics of the structured scene according to the space point and the space straight line corresponding to the ORB point characteristics and the LSD line characteristics;
s4, estimating the pose of the camera through the minimized error model and generating three-dimensional map points of the structured scene;
s5, according to the obtained camera pose and the three-dimensional map points of the structured scene, in the video frame, generating a key frame set through the tracking condition of the point and line characteristics of the structured scene and the camera positioning and map building state;
s6, establishing a bag-of-words model based on the improved point-line characteristics by using the key frame set, and carrying out closed-loop detection on the three-dimensional map points;
and S7, optimizing the camera pose and the three-dimensional map points of the structured scene through a point-line characteristic error model according to the closed loop detection result obtained in the step S6 after the closed loop condition is detected, and improving the effect of positioning and map building at the same time.
Further, in step S1, performing basic calibration on the RGB-D depth camera to obtain internal reference information; initializing visual SLAM on the structured scene through the depth camera; the method comprises the following specific steps:
s1.1, taking a black and white checkerboard in a structured scene as a calibration object, shooting photos in different directions for the calibration object by adjusting the directions of the calibration object and a camera, and extracting checkerboard angular points from the photos;
s1.2, setting the camera coordinate of a certain space point P in the real environment as [ X, Y, Z]TX, Y and Z are coordinates of the camera in a three-dimensional space stereo coordinate system respectively;
obtaining the coordinates of a corresponding imaging point P' in an imaging plane according to the similarity relation of triangles in a pinhole camera model
Figure BDA0002245596790000021
f is the focal length of the camera; converting the point P' to pixel coordinates U, V]Is provided with
Figure BDA0002245596790000022
Wherein K is a camera reference matrix;
s1.3, reasonably selecting a correction parameter k according to actual conditions1，k2，l1，l2，l3Correcting radial distortion and tangential distortion in the basic calibration calculation of the camera; wherein k is1，k2For radial distortion correction factor,/1，l2，l3Is a tangential distortion correction coefficient;
s1.4, before extraction frORB point characteristics of the frame image are selected, and the number N of the ORB point characteristics is selectedORBThe video frame exceeding a certain value is taken as an initial frame; typically selected within the first 20 frames of the image, NORBNot less than 50; and taking the selected initial frame as a reference, and carrying out synchronous positioning and image building initialization on the structured scene through the depth camera.
Further, in step S2, extracting ORB point features in the structured scene from the video frame obtained by the camera, and corresponding to the spatial points in the structured scene; the method comprises the following specific steps:
s2.1.1, extracting pixel points in the image to obtain ORB feature points, and expressing the direction of the ORB feature points by a gray centroid method, wherein the direction of the feature points is the direction of a connecting line between the geometric center of the adjacent a x a image block and the gray centroid of the point;
s2.1.2, selecting q pixel pairs (j, k) by greedy search in b x b image blocks adjacent to ORB feature points, wherein the gray values of the pixel pairs are Ij，IkComparing the brightness relation of two pixels in the pixel pair to obtain a q-dimensional vector descriptor composed of 1 and 0 to describe the relation between the feature point and the image, if Ij＞IkIf so, taking 1 as the corresponding bit of the descriptor, otherwise, taking 0, wherein the descriptor is the ORB feature descriptor in the structured scene; the ORB point characteristics are extracted by constructing an ORB characteristic descriptor for the video frame image;
s2.1.3, describing each ORB feature in the video frame image as its three-dimensional space point P in the structured scenewAnd correspondingly.
Further, in step S2, extracting LSD line features in the structured scene from the video frame obtained by the camera, and corresponding to the spatial straight lines in the structured scene; the method comprises the following specific steps:
the method for representing the spatial straight line in the structured scene is realized by extracting the LSD line characteristics of the video frame image;
s2.2.1, extracting LSD line feature in the image, and setting the line segment support domain of the LSD line feature as thetalSince lines are regular and dense in a structured scene, the line segment support domain is at the threshold θll′Merging the end points of the inner line segments l and l ', and taking the average value of the end point coordinates of the line segments l and l' as the coordinates of the merged end points;
s2.2.2, representing spatial straight lines in the structured scene corresponding to LSD line features using Pliicker coordinates;
setting the homogeneous coordinates of the two end points of the space straight line corresponding to the LSD line feature obtained in step S2.2.1 as:
P1(x1，y1，z1，1)，P2(x2，y2，z2，1)
the Pliicker matrix T is expressed as follows:
Figure BDA0002245596790000031
the spatial straight line L in the structured scene corresponding to the LSD line featurewExpressed using Pliicker coordinates as:
Figure BDA0002245596790000032
wherein n and v are space straight lines LwTwo end point coordinate vectors in the Pliicker coordinates, the values of which are calculated from the Pliicker matrix T.
Further, in step S3, an error model based on the improved point feature and the line feature of the structured scene is established according to the spatial point and the spatial straight line corresponding to the ORB point feature and the LSD line feature; the method comprises the following specific steps:
s3.1, establishing a space point P corresponding to the ORB point characteristicswThe error model of (2);
spatial point P in structured scene according to camera motion equationwSpatial point P converted to camera coordinatesc；
Obtaining a pixel coordinate system projection point P 'according to the pinhole imaging model'uvCalculating the actual coordinates PuvAnd P'uvAn error of (2);
the spatial point P corresponding to the ORB point featurewThe error model of (a) is expressed as:
Figure BDA0002245596790000034
wherein the content of the first and second substances,
Figure BDA0002245596790000035
representing the ith spatial point Pw，iProjection on pixel coordinates and its actual coordinates P on pixel planeuv，k，iThe error between; k represents the K frame image frame under the structured scene, K is the camera internal parameter, Tcw，kA pose transformation matrix of a camera coordinate system relative to a structured scene coordinate system;
s3.2, establishing a space straight line L corresponding to the LSD line characteristicswThe error model of (2);
projection L 'of space straight line corresponding to LSD line feature on pixel plane'uvActual coordinate L of the pixel plane with the spatial straight lineuvError between, constructing an error model:
Figure BDA0002245596790000033
wherein p iss，peIs a space line LwN, v are spatial straight lines LwTwo endpoint coordinate vectors in Pliicker coordinates;
then the spatial straight line L corresponding to the LSD line featurewThe error model of (a) is expressed as:
wherein the content of the first and second substances,
Figure BDA0002245596790000043
indicates the jth nullStraight line L betweenw，jProjection on pixel coordinate and its actual coordinate L on pixel planeuv，k，jThe error between; k represents the K frame image frame under the structured scene, K is the camera internal parameter, Tcw，kA pose transformation matrix of a camera coordinate system relative to a structured scene coordinate system;
s3.3, in the structured scene, the line characteristics are more obvious relative to the point characteristics and more important to closed loop detection, so that a two-norm error model of the overall improved point-line characteristics is established, the weight occupied by the line characteristic error model is increased in the overall error model, and the two-norm error model is expressed as follows:
Figure BDA0002245596790000044
wherein E is a two-norm error, and is set to deltal＝2δp，δl∈(0.2～1)。
Further, in step S4, estimating a camera pose by minimizing the error model, and generating three-dimensional map points of the structured scene; the method comprises the following specific steps:
minimizing a dotted line characteristic error model, namely solving an optimal solution for a target function in the error model;
the objective function of the dotted line characteristic error model is expressed as follows:
(JTJ+λI)Δx＝-JTE
Figure BDA0002245596790000041
wherein J is a Jacobian matrix of the objective function with respect to variables, the variables comprising: camera pose Tcw，kAnd three-dimensional map points P of a structured scenew，iΔ x is Tcw，k、Pw，iThe minimum quantity of (2) is abbreviated, wherein lambda is a dynamic adjustment parameter, and in order to enable the objective function to obtain an optimal solution, lambda is generally 0.5;
and performing approximate incremental linear solution on the target function by adopting a Levenberg-Marquardt method to obtain three-dimensional map points of the camera pose and the structured scene.
Further, in step S5, in the video frame, a key frame set is generated by a decision according to the tracking condition of the point and line features of the structured scene and the states of camera positioning and mapping; the method comprises the following specific steps:
in the synchronous positioning and mapping (SLAM) process of an RGB-D depth camera, the precondition for carrying out closed-loop detection is that key frames with moderate quantity and quality meeting expectations are selected from video frames as the reference of the closed-loop detection;
s5.1, judging whether the current frame is a key frame according to the selection and judgment conditions of the key frame; the conditions include:
(1) the interval between the current frame and the last selected key frame exceeds N1A frame;
(2) the working state for generating the positioning and mapping of the depth camera is stable, or N exists2The image frames of the frame are discarded;
(3) the image frame obtained by the current camera is constructed to be larger than MlA spatial straight line; the spatial straight line is a spatial straight line corresponding to LSD line characteristics obtained by optimizing a point-line characteristic error model based on a structural scene improvement;
(4) the similarity between the ORB point characteristics in the image frame obtained by the current camera and the previous key frame does not exceed S; the similarity is the proportion of the same number of spatial points in the current frame image as the spatial points in the previous key frame, wherein the spatial points correspond to the ORB point features obtained based on the optimization of the improved dotted line feature error model and account for all the spatial points;
s5.2, selecting the image frames which simultaneously meet the four conditions in the step S5.1 as key frames;
and S5.3, traversing all the video frames, and selecting all the key frames according to the S5.1-S5.2 to obtain a key frame set.
Further, N1，N2，MlThe S value can be adjusted according to the actual condition; the values based on experience are: n is a radical of1＝20，N2＝20，Ml＝50，S＝90％。
Further, in step S6, a keyframe set is used to establish a bag-of-words model based on the improved point-line characteristics, and the three-dimensional map points are subjected to closed-loop detection; the method comprises the following specific steps:
s6.1, generating words of the key frame set based on a k-means algorithm;
s6.2, in the closed-loop detection of the structured scene, the line features have higher discrimination compared with the point features, so that when words of a key frame set are used and a bag-of-words model based on the improved point-line features is established, the weight of the line features is increased;
judging whether the point characteristics corresponding to the words of the key frame set belong to the line characteristics of the key frame image or not, and giving weight to the words according to whether the words belong to the line characteristics or not;
s6.3, establishing a bag-of-words model based on the improved point-line characteristics by constructing point characteristic words and weights thereof, and applying the bag-of-words model to visual SLAM closed-loop detection of a structured scene;
and S6.4, when the same number of point features in the current frame image as the number of point feature words in the bag-of-words model based on the key frame set exceeds a threshold value, determining that a closed loop is detected.
Further, the step S6.1 generates words of the key frame set based on a k-means algorithm; the method comprises the following steps:
determining n by taking random ORB point characteristics in key frame images as initial clustering centerswordAn initial clustering center, 0 < nword＜nMAX，nMAXThe upper limit of the number of clustering centers is usually 100;
when the (i + 1) th clustering center is determined, the farther the distance between the current ith clustering center and the point is determined as the (i + 1) th clustering center, the higher the probability;
respectively calculating the explicit distance between the ORB point characteristics in the key frame and each clustering center;
selecting the clustering center closest to the point characteristic as the clustering center of the point characteristic, and connecting the point characteristic with the corresponding clustering center by using an octree data structure; all the obtained clustering centers are words of the key frame set;
further, in the step S6.2, it is determined whether a word of the key frame set belongs to a line feature of the key frame image, and a weight is given to the word according to whether the word belongs to the line feature; the method comprises the following specific steps:
η＝ω·TFi×IDFi
Figure BDA0002245596790000051
Figure BDA0002245596790000052
wherein η represents bag-of-words model describing dotted line features, ω represents weights of dotted line features, IDFiRepresents a word WwordMiddle ORB feature quantity featureiThe ratio of the number of features feature relative to all words in the line features ORB; TFiRepresents the word WwordThe frequency of presentation in any video frame, feature' representing the total number of words appearing in the video frame image, featureiIs the word WwordA number of times of presentation in the video frame image;
if the coordinates of the words in the image are located in the coordinates of any line feature in the image, the words belong to the line feature of the key frame image, and the weight omega is 1;
if the coordinates of the word in the image are outside the coordinates of any line feature in the image, the word does not belong to the line feature of the key frame image, and the weight omega is 0.1.
Further, in step S7, after the closed-loop condition is detected, the camera pose and the three-dimensional map points of the structured scene are optimized through the point-line characteristic error model, so as to improve the effect of positioning and mapping at the same time; the method comprises the following specific steps:
s7.1, after the closed loop is detected, the point line feature set { p ] of the current frame is collectedref，lrefPoint-line feature set { p } of key frame with highest similarity obtained by closed-loop detectionkey，lkeyCorresponding to the line features, and respectively matching the corresponding ORB point features with the LSD line features; wherein p isrefIs the point feature of the current frame, lrefIs a line feature of the current frame, pkeyIs a point feature of the key frame,/keyLine features for the keyframes;
s7.2, traversing a set K { ∑ (p, l) } of the current frame and the key frame, fusing all matched ORB point features and LSD line features, minimizing reprojection errors through a cost function, optimizing the camera pose and all structured scene three-dimensional map points, and improving the effect of positioning and mapping at the same time;
the expression of the cost function is as follows:
Figure BDA0002245596790000061
wherein i represents the ith matched ORB point feature, j represents the jth matched LSD line feature, and N and M respectively represent the number of the matched ORB point feature and the LSD line feature; eref，EkeyA two-norm error model of the improved dotted line features of the current video frame and the keyframe, respectively.
Has the advantages that: compared with the prior art, the technical scheme of the invention has the following beneficial technical effects:
the invention provides a structured scene vision SLAM scheme expanded by improved dotted line characteristics by taking a depth camera working in an indoor structured scene as an example. The method solves the problem that the precision and the efficiency of the visual SLAM in the existing structured scene are not high for closed-loop detection, provides a new solution for closed-loop detection in a specific environment, and provides great convenience for the visual SLAM to work. For example, the robot can be helped to improve the position location and the three-dimensional map point reconstruction accuracy of the surrounding structural environment in the fields of indoor vehicle location, indoor sorting express delivery robot work, sweeping robot path planning and the like, and a new feasible scheme with higher accuracy is provided for the fields.
Drawings
FIG. 1 is a schematic flow diagram of the system of the present invention.
Detailed Description
The technical scheme of the invention is further explained in detail by combining the embodiment and the attached drawings; it is to be understood that this embodiment is provided for illustration only and not for the purpose of limiting the scope of the invention, which is to be given the full breadth of the appended claims and any and all equivalent modifications thereto by those skilled in the art after reading this disclosure.
S1, performing basic calibration on the RGB-D depth camera to obtain internal reference information; visual synchronous positioning and image building (visual SLAM) initialization are carried out on the structured scene through the depth camera;
s2, extracting ORB point features and LSD line features in the structured scene through the video frame obtained by the camera, and respectively corresponding to the space points and the space straight lines in the structured scene;
s3, establishing an error model based on the improved point characteristics and the line characteristics of the structured scene according to the space point and the space straight line corresponding to the ORB point characteristics and the LSD line characteristics;
s4, estimating the pose of the camera through the minimized error model and generating three-dimensional map points of the structured scene;
s5, according to the obtained camera pose and the three-dimensional map points of the structured scene, in the video frame, generating a key frame set through the tracking condition of the point and line characteristics of the structured scene and the camera positioning and map building state;
s6, establishing a bag-of-words model based on the improved point-line characteristics by using the key frame set, and carrying out closed-loop detection on the three-dimensional map points;
and S7, optimizing the camera pose and the three-dimensional map points of the structured scene through a point-line characteristic error model according to the closed loop detection result obtained in the step S6 after the closed loop condition is detected, and improving the effect of positioning and map building at the same time.
In step S1, performing basic calibration on the RGB-D depth camera to obtain internal reference information; initializing a visual SLAM for the structured scene through the camera; the method comprises the following specific steps:
s1.1, taking a black and white checkerboard in a structured scene as a calibration object, shooting photos in different directions for the calibration object by adjusting the directions of the calibration object and a camera, and extracting checkerboard angular points from the photos;
s1.2, setting the camera coordinate of a certain space point P in the real environment as [ X, Y, Z]TX, Y and Z are coordinates of the camera in a three-dimensional space stereo coordinate system respectively;
obtaining the coordinates of a corresponding imaging point P' in an imaging plane according to the similarity relation of triangles in a pinhole camera model
Figure BDA0002245596790000071
f is the focal length of the camera; converting the point P' to pixel coordinates U, V]Is provided with
Wherein K is a camera reference matrix;
s1.3, reasonably selecting a correction parameter k according to actual conditions1，k2，l1，l2，l3Correcting radial distortion and tangential distortion in the basic calibration calculation of the camera; wherein k is1，k2For radial distortion correction factor,/1，l2，l3Is a tangential distortion correction coefficient;
s1.4, before extraction frORB point characteristics of the frame image are selected, and the number N of the ORB point characteristics is selectedORBThe video frame exceeding a certain value is taken as an initial frame; and taking the selected initial frame as a reference, and carrying out synchronous positioning and image building initialization on the structured scene through the depth camera. In this embodiment, an initial frame, N, is selected from the first 20 frames of the imageORB≥50。
In step S2, extracting ORB point features in the structured scene from the video frame obtained by the camera, and corresponding to spatial points in the structured scene; the method comprises the following specific steps:
s2.1.1, extracting pixel points in the image to obtain ORB feature points, and expressing the direction of the ORB feature points by a gray centroid method, wherein the direction of the feature points is the direction of a connecting line between the geometric center of the adjacent a x a image block and the gray centroid of the point;
s2.1.2, selecting q pixel pairs (j, k) in b x b image blocks adjacent to ORB feature points by greedy searchThe gray values of the pixel pairs are respectively Ij，IkComparing the brightness relation of two pixels in the pixel pair to obtain a q-dimensional vector descriptor composed of 1 and 0 to describe the relation between the feature point and the image, if Ij＞IkIf so, taking 1 as the corresponding bit of the descriptor, otherwise, taking 0, wherein the descriptor is the ORB feature descriptor in the structured scene; the ORB point characteristics are extracted by constructing an ORB characteristic descriptor for the video frame image;
s2.1.3, describing each ORB feature in the video frame image as its three-dimensional space point P in the structured scenewAnd correspondingly.
In this embodiment, a is 4, b is 20, and q is 256.
In step S2, extracting LSD line features in the structured scene from the video frame obtained by the camera, and corresponding to spatial straight lines in the structured scene; the method comprises the following specific steps:
the method for representing the spatial straight line in the structured scene is realized by extracting the LSD line characteristics of the video frame image;
s2.2.1, extracting LSD line feature in the image, and setting the line segment support domain of the LSD line feature as thetalSince lines are regular and dense in a structured scene, the line segment support domain is at the threshold θll′Merging the end points of the inner line segments l and l ', and taking the average value of the end point coordinates of the line segments l and l' as the coordinates of the merged end points;
s2.2.2, representing spatial straight lines in the structured scene corresponding to LSD line features using Pliicker coordinates;
setting the homogeneous coordinates of the two end points of the space straight line corresponding to the LSD line feature obtained in step S2.2.1 as:
P1(x1，y1，z1，1)，P2(x2，y2，z2，1)
the Pliicker matrix T is expressed as follows:
then pair with the LSD line featureSpatial straight line L in structured scenewExpressed using Pliicker coordinates as:
Figure BDA0002245596790000082
wherein n and v are space straight lines LwTwo end point coordinate vectors in the Pliicker coordinates, the values of which are calculated from the Pliicker matrix T.
In the step S3, an error model based on the improved point feature and the line feature of the structured scene is established according to the spatial point and the spatial straight line corresponding to the ORB point feature and the LSD line feature; the method comprises the following specific steps:
s3.1, establishing a space point P corresponding to the ORB point characteristicswThe error model of (2);
spatial point P in structured scene according to camera motion equationwSpatial point P converted to camera coordinatesc；
Obtaining a pixel coordinate system projection point P 'according to the pinhole imaging model'uvCalculating the actual coordinates PuvAnd P'uvAn error of (2);
the spatial point P corresponding to the ORB point featurewThe error model of (a) is expressed as:
Figure BDA0002245596790000095
wherein the content of the first and second substances,
Figure BDA0002245596790000096
representing the ith spatial point Pw，iProjection on pixel coordinates and its actual coordinates P on pixel planeuv，k，iThe error between; k represents the K frame image frame under the structured scene, K is the camera internal parameter, Tcw，kA pose transformation matrix of a camera coordinate system relative to a structured scene coordinate system;
s3.2, establishing a space straight line L corresponding to the LSD line characteristicswThe error model of (2);
projection L 'of space straight line corresponding to LSD line feature on pixel plane'uvActual coordinate L of the pixel plane with the spatial straight lineuvError between, constructing an error model:
Figure BDA0002245596790000091
wherein p iss，peIs a space line LwN, v are spatial straight lines LwTwo endpoint coordinate vectors in Pliicker coordinates;
then the spatial straight line L corresponding to the LSD line featurewThe error model of (a) is expressed as:
Figure BDA0002245596790000092
wherein the content of the first and second substances,
Figure BDA0002245596790000093
represents the jth spatial straight line Lw，jProjection on pixel coordinate and its actual coordinate L on pixel planeuv，k，jThe error between; k represents the K frame image frame under the structured scene, K is the camera internal parameter, Tcw，kA pose transformation matrix of a camera coordinate system relative to a structured scene coordinate system;
s3.3, in the structured scene, the line characteristics are more obvious relative to the point characteristics and more important to closed loop detection, so that a two-norm error model of the overall improved point-line characteristics is established, the weight occupied by the line characteristic error model is increased in the overall error model, and the two-norm error model is expressed as follows:
Figure BDA0002245596790000094
wherein E is a two-norm error, and is set to deltal＝2δp，δl∈(0.2～1)。
In step S4, estimating a camera pose by minimizing the error model, and generating three-dimensional map points of the structured scene; the method comprises the following specific steps:
minimizing a dotted line characteristic error model, namely solving an optimal solution for a target function in the error model;
the objective function of the dotted line characteristic error model is expressed as follows:
(JTJ+λI)Δx＝-JTE
Figure BDA0002245596790000101
wherein J is a Jacobian matrix of the objective function with respect to variables, the variables comprising: camera pose Tcw，kAnd three-dimensional map points P of a structured scenew，iΔ x is Tcw，k、Pw，iThe minimum quantity of (2) is abbreviated, wherein lambda is a dynamic adjustment parameter, and in order to enable the objective function to obtain an optimal solution, lambda is generally 0.5;
and performing approximate incremental linear solution on the target function by adopting a Levenberg-Marquardt method to obtain three-dimensional map points of the camera pose and the structured scene.
In step S5, in a video frame, a key frame set is generated by a decision based on tracking conditions of point and line features of a structured scene and states of camera positioning and mapping; the method comprises the following specific steps:
in the synchronous positioning and mapping (SLAM) process of an RGB-D depth camera, the precondition for carrying out closed-loop detection is that key frames with moderate quantity and quality meeting expectations are selected from video frames as the reference of the closed-loop detection;
s5.1, judging whether the current frame is a key frame according to the selection and judgment conditions of the key frame; the conditions include:
(1) the interval between the current frame and the last selected key frame exceeds N1A frame;
(2) the working state for generating the positioning and mapping of the depth camera is stable, or N exists2The image frames of the frame are discarded;
(3) the image frame obtained by the current camera is constructed to be larger than MlA spatial straight line; the space straight line is a space corresponding to LSD line characteristics obtained by optimizing a point-line characteristic error model based on a structural scene improvementA straight line between;
(4) the similarity between the ORB point characteristics in the image frame obtained by the current camera and the previous key frame does not exceed S; the similarity is the proportion of the same number of spatial points in the current frame image as the spatial points in the previous key frame, wherein the spatial points correspond to the ORB point features obtained based on the optimization of the improved dotted line feature error model and account for all the spatial points;
s5.2, selecting the image frames which simultaneously meet the four conditions in the step S5.1 as key frames;
and S5.3, traversing all the video frames, and selecting all the key frames according to the S5.1-S5.2 to obtain a key frame set.
In this example, N1＝20，N2＝20，Ml＝50，S＝90％。
In the step S6, a keyframe set is used to establish a bag-of-words model based on the improved point-line characteristics, and the closed-loop detection is performed on the three-dimensional map points; the method comprises the following specific steps:
s6.1, generating words of the key frame set based on a k-means algorithm;
s6.2, in the closed-loop detection of the structured scene, the line features have higher discrimination compared with the point features, so that when words of a key frame set are used and a bag-of-words model based on the improved point-line features is established, the weight of the line features is increased;
judging whether the point characteristics corresponding to the words of the key frame set belong to the line characteristics of the key frame image or not, and giving weight to the words according to whether the words belong to the line characteristics or not;
s6.3, establishing a bag-of-words model based on the improved point-line characteristics by constructing point characteristic words and weights thereof, and applying the bag-of-words model to visual SLAM closed-loop detection of a structured scene;
and S6.4, when the same number of point features in the current frame image as the number of point feature words in the bag-of-words model based on the key frame set exceeds a threshold value, determining that a closed loop is detected.
S6.1, generating words of the key frame set based on a k-means algorithm; the method comprises the following steps:
by random ORB point features in key frame imagesDetermining n as an initial cluster centerwordAn initial clustering center, 0 < nword＜nMAX，nMAXAs the upper limit of the number of cluster centers, n in this embodimentMAXTaking 100;
when the (i + 1) th clustering center is determined, the farther the distance between the current ith clustering center and the point is determined as the (i + 1) th clustering center, the higher the probability;
respectively calculating the explicit distance between the ORB point characteristics in the key frame and each clustering center;
selecting the clustering center closest to the point characteristic as the clustering center of the point characteristic, and connecting the point characteristic with the corresponding clustering center by using an octree data structure; all the obtained clustering centers are words of the key frame set;
in the step S6.2, judging whether the words of the key frame set belong to the line characteristics of the key frame image, and giving weight to the words according to whether the words belong to the line characteristics; the method comprises the following specific steps:
η＝ω·TFi×IDFi
Figure BDA0002245596790000111
Figure BDA0002245596790000112
η represents a bag-of-words model describing the dotted line characteristics, which is used for detecting whether a closed loop occurs based on the dotted line characteristics in the loop detection, ω represents the weight of the dotted line characteristics, IDFiRepresents the word WwordMiddle ORB feature quantity featureiThe ratio of the number of features feature relative to all words in the line features ORB; TFiRepresents the word WwordThe frequency of presentation in any video frame, feature' representing the total number of words appearing in the video frame image, featureiIs the word WwordA number of times of presentation in the video frame image;
if the coordinates of the words in the image are located in the coordinates of any line feature in the image, the words belong to the line feature of the key frame image, and the weight omega is 1;
if the coordinates of the word in the image are outside the coordinates of any line feature in the image, the word does not belong to the line feature of the key frame image, and the weight omega is 0.1.
In the step S7, after a closed-loop condition is detected, the camera pose and the three-dimensional map points of the structured scene are optimized through a dotted line characteristic error model, so as to improve the effect of positioning and map building at the same time; the method comprises the following specific steps:
s7.1, after the closed loop is detected, the point line feature set { p ] of the current frame is collectedref，lrefPoint-line feature set { p } of key frame with highest similarity obtained by closed-loop detectionkey，lkeyCorresponding to the line features, and respectively matching the corresponding ORB point features with the LSD line features; wherein p isrefIs the point feature of the current frame, lrefIs a line feature of the current frame, pkeyIs a point feature of the key frame,/keyLine features for the keyframes;
s7.2, traversing a set K { ∑ (p, l) } of the current frame and the key frame, fusing all matched ORB point features and LSD line features, minimizing reprojection errors through a cost function, optimizing the camera pose and all structured scene three-dimensional map points, and improving the effect of positioning and mapping at the same time;
the expression of the cost function is as follows:
Figure BDA0002245596790000121
wherein i represents the ith matched ORB point feature, j represents the jth matched LSD line feature, and N and M respectively represent the number of the matched ORB point feature and the LSD line feature; eref，EkeyA two-norm error model of the improved dotted line features of the current video frame and the keyframe, respectively.

Claims (10)

1. A structured scene vision SLAM method based on improved point-line characteristics is characterized in that: the method comprises the following steps:
s1, performing basic calibration on the RGB-D depth camera to obtain internal reference information; initializing a visual SLAM for the structured scene through the camera;
s2, extracting ORB point features and LSD line features in the structured scene through the video frame obtained by the camera, and respectively corresponding to the space points and the space straight lines in the structured scene;
s3, establishing an error model based on the improved point characteristics and the line characteristics of the structured scene according to the space point and the space straight line corresponding to the ORB point characteristics and the LSD line characteristics;
s4, estimating the pose of the camera through the minimized error model and generating three-dimensional map points of the structured scene;
s5, according to the obtained camera pose and the three-dimensional map points of the structured scene, in the video frame, generating a key frame set through the tracking condition of the point and line characteristics of the structured scene and the camera positioning and map building state;
s6, establishing a bag-of-words model based on the improved point-line characteristics by using the key frame set, and carrying out closed-loop detection on the three-dimensional map points;
and S7, optimizing the camera pose and the three-dimensional map points of the structured scene through a point-line characteristic error model according to the closed loop detection result obtained in the step S6 after the closed loop condition is detected, and improving the effect of positioning and map building at the same time.
2. The structured scene vision SLAM method based on the improved dotted line feature of claim 1, wherein: in step S1, performing basic calibration on the RGB-D depth camera to obtain internal reference information; initializing a visual SLAM for the structured scene through the camera; the method comprises the following specific steps:
s1.1, taking a black and white checkerboard in a structured scene as a calibration object, shooting photos in different directions for the calibration object by adjusting the directions of the calibration object and a camera, and extracting checkerboard angular points from the photos;
s1.2, setting the camera coordinate of a certain space point P in the real environment as [ X, Y, Z]TX, Y and Z are coordinates of the camera in a three-dimensional space stereo coordinate system respectively;
obtaining the coordinates of a corresponding imaging point P' in an imaging plane according to the similarity relation of triangles in a pinhole camera model
Figure FDA0002245596780000011
f is the focal length of the camera; converting the point P' to pixel coordinates U, V]Is provided with
Wherein K is a camera reference matrix;
s1.3, selecting a correction parameter k1，k2，l1，l2，l3Correcting radial distortion and tangential distortion in the basic calibration calculation of the camera; wherein k is1，k2For radial distortion correction factor,/1，l2，l3Is a tangential distortion correction coefficient;
s1.4, before extraction frORB point characteristics of the frame image are selected, and the number N of the ORB point characteristics is selectedORBAnd taking the video frame exceeding a certain value as an initial frame, and taking the selected initial frame as a reference to synchronously position and initialize the image in the structured scene through the depth camera.
3. The structured scene vision SLAM method based on the improved dotted line feature of claim 1, wherein: in step S2, extracting ORB point features in the structured scene from the video frame obtained by the camera, and corresponding to spatial points in the structured scene; the method comprises the following specific steps:
s2.1.1, extracting pixel points in the image to obtain ORB feature points, and expressing the direction of the ORB feature points by a gray centroid method, wherein the direction of the feature points is the direction of a connecting line between the geometric center of the adjacent a x a image block and the gray centroid of the point;
s2.1.2, selecting q pixel pairs (j, k) by greedy search in b x b image blocks adjacent to ORB feature points, wherein the gray values of the pixel pairs are Ij，IkComparing the luminance of two pixels in a pixel pairObtaining a q-dimensional vector descriptor consisting of 1 and 0 to describe the relationship between the characteristic point and the image if Ij＞IkIf so, taking 1 as the corresponding bit of the descriptor, otherwise, taking 0, wherein the descriptor is the ORB feature descriptor in the structured scene; the ORB point characteristics are extracted by constructing an ORB characteristic descriptor for the video frame image;
s2.1.3, describing each ORB feature in the video frame image as its three-dimensional space point P in the structured scenewAnd correspondingly.
4. The structured scene vision SLAM method based on the improved dotted line feature as claimed in claim 3, wherein: in step S2, extracting LSD line features in the structured scene from the video frame obtained by the camera, and corresponding to spatial straight lines in the structured scene; the method comprises the following specific steps:
s2.2.1, extracting LSD line feature in the image, and setting the line segment support domain of the LSD line feature as thetalFor a line segment support field at a threshold value thetall′Merging the end points of the inner line segments l and l ', and taking the average value of the end point coordinates of the line segments l and l' as the coordinates of the merged end points;
s2.2.2, representing spatial straight lines in the structured scene corresponding to LSD line features using Pliicker coordinates;
setting the homogeneous coordinates of the two end points of the space straight line corresponding to the LSD line feature obtained in step S2.2.1 as:
P1(x1，y1，z1，1)，P2(x2，y2，z2，1)
the Pliicker matrix T is expressed as follows:
Figure FDA0002245596780000021
the spatial straight line L in the structured scene corresponding to the LSD line featurewExpressed using Pliicker coordinates as:
Figure FDA0002245596780000022
wherein n and v are space straight lines LwTwo end point coordinate vectors in the Pliicker coordinates, the values of which are calculated from the Pliicker matrix T.
5. The structured scene vision SLAM method based on the improved dotted line feature as claimed in claim 4, wherein: in the step S3, an error model based on the improved point feature and the line feature of the structured scene is established according to the spatial point and the spatial straight line corresponding to the ORB point feature and the LSD line feature; the method comprises the following specific steps:
s3.1, establishing a space point P corresponding to the ORB point characteristicswThe error model of (2);
spatial point P in structured scene according to camera motion equationwSpatial point P converted to camera coordinatesc；
Obtaining a pixel coordinate system projection point P 'according to the pinhole imaging model'uvCalculating the actual coordinates PuvAnd P'uvAn error of (2);
the spatial point P corresponding to the ORB point featurewThe error model of (a) is expressed as:
Figure FDA0002245596780000031
wherein the content of the first and second substances,representing the ith spatial point Pw，iProjection on pixel coordinates and its actual coordinates P on pixel planeuv，k，iThe error between; k represents the K frame image frame under the structured scene, K is the camera internal parameter, Tcw，kA pose transformation matrix of a camera coordinate system relative to a structured scene coordinate system;
s3.2, establishing a space straight line L corresponding to the LSD line characteristicswThe error model of (2);
projection L 'of space straight line corresponding to LSD line feature on pixel plane'uvIn the image of space straight lineActual coordinates L of the pixel planeuvError between, constructing an error model:
Figure FDA0002245596780000033
wherein p iss，peIs a space line LwN, v are spatial straight lines LwTwo endpoint coordinate vectors in Pliicker coordinates;
then the spatial straight line L corresponding to the LSD line featurewThe error model of (a) is expressed as:
Figure FDA0002245596780000034
wherein the content of the first and second substances,
Figure FDA0002245596780000035
represents the jth spatial straight line Lw，jProjection on pixel coordinate and its actual coordinate L on pixel planeuv，k，jThe error between; k represents the K frame image frame under the structured scene, K is the camera internal parameter, Tcw，kA pose transformation matrix of a camera coordinate system relative to a structured scene coordinate system;
s3.3, establishing a two-norm error model of the overall improved point-line characteristic, and increasing the weight of the line characteristic error model in the overall error model, wherein the weight is expressed as follows:
Figure FDA0002245596780000036
wherein E is a two-norm error, and is set to deltal＝2δp，δl∈(0.2～1)。
6. The structured scene vision SLAM method based on the improved dotted line feature of claim 1, wherein: in step S4, estimating a camera pose by minimizing the error model, and generating three-dimensional map points of the structured scene; the method comprises the following specific steps:
minimizing a dotted line characteristic error model, namely solving an optimal solution for a target function in the error model;
the objective function of the dotted line characteristic error model is expressed as follows:
(JTJ+λI)Δx＝-JTE
Figure FDA0002245596780000041
wherein J is a Jacobian matrix of the objective function with respect to variables, the variables comprising: camera pose Tcw，kAnd three-dimensional map points P of a structured scenew，iΔ x is Tcw，k、Pw，iIn short, λ is a dynamic adjustment parameter;
and performing approximate incremental linear solution on the target function by adopting a Levenberg-Marquardt method to obtain three-dimensional map points of the camera pose and the structured scene.
7. The structured scene vision SLAM method based on the improved dotted line feature of claim 1, wherein: in step S5, in a video frame, a key frame set is generated by a decision based on tracking conditions of point and line features of a structured scene and states of camera positioning and mapping; the method comprises the following specific steps:
s5.1, judging whether the current frame is a key frame according to the selection and judgment conditions of the key frame; the conditions include:
(1) the interval between the current frame and the last selected key frame exceeds N1A frame;
(2) the working state for generating the positioning and mapping of the depth camera is stable, or N exists2The image frames of the frame are discarded;
(3) the image frame obtained by the current camera is constructed to be larger than MlA spatial straight line; the spatial straight line is a spatial straight line corresponding to LSD line characteristics obtained by optimizing a point-line characteristic error model based on a structural scene improvement;
(4) the similarity between the ORB point characteristics in the image frame obtained by the current camera and the previous key frame does not exceed S; the similarity is the proportion of the same number of spatial points in the current frame image as the spatial points in the previous key frame, wherein the spatial points correspond to the ORB point features obtained based on the optimization of the improved dotted line feature error model and account for all the spatial points;
s5.2, selecting the image frames which simultaneously meet the four conditions in the step S5.1 as key frames;
and S5.3, traversing all the video frames, and selecting all the key frames according to the S5.1-S5.2 to obtain a key frame set.
8. The structured scene vision SLAM method based on the improved dotted line feature of claim 1, wherein: in the step S6, a keyframe set is used to establish a bag-of-words model based on the improved point-line characteristics, and the closed-loop detection is performed on the three-dimensional map points; the method comprises the following specific steps:
s6.1, generating words of the key frame set based on a k-means algorithm;
s6.2, judging whether the point characteristics corresponding to the words of the key frame set belong to the line characteristics of the key frame image or not, and giving weight to the words according to whether the words belong to the line characteristics or not;
s6.3, establishing a bag-of-words model based on the improved point-line characteristics by constructing point characteristic words and weights thereof, and applying the bag-of-words model to visual SLAM closed-loop detection of a structured scene;
and S6.4, when the same number of point features in the current frame image as the number of point feature words in the bag-of-words model based on the key frame set exceeds a threshold value, determining that a closed loop is detected.
9. The structured scene vision SLAM method based on the improved dotted line feature of claim 8, wherein: s6.1, generating words of the key frame set based on a k-means algorithm; the method comprises the following steps:
determining n by taking random ORB point characteristics in key frame images as initial clustering centerswordAn initial clustering center, 0 < nword＜nMAX，nMAXAs the number of cluster centersAn upper limit;
when the (i + 1) th clustering center is determined, the farther the distance between the current ith clustering center and the point is determined as the (i + 1) th clustering center, the higher the probability;
respectively calculating the explicit distance between the ORB point characteristics in the key frame and each clustering center;
selecting the clustering center closest to the point characteristic as the clustering center of the point characteristic, and connecting the point characteristic with the corresponding clustering center by using an octree data structure; all the obtained clustering centers are words of the key frame set;
in the step S6.2, judging whether the words of the key frame set belong to the line characteristics of the key frame image, and giving weight to the words according to whether the words belong to the line characteristics; the method comprises the following specific steps:
η＝ω·TFi×IDFi
Figure FDA0002245596780000051
Figure FDA0002245596780000052
wherein η represents bag-of-words model describing dotted line features, ω represents weights of dotted line features, IDFiRepresents a word WwordMiddle ORB feature quantity featureiThe ratio of the number of features feature relative to all words in the line features ORB; TFiRepresents the word WwordThe frequency of presentation in any video frame, feature' representing the total number of words appearing in the video frame image, featureiIs the word WwordA number of times of presentation in the video frame image;
if the word belongs to the line feature of the key frame image, the weight omega is 1;
if the word does not belong to the line feature of the keyframe image, the weight ω is 0.1.
10. The structured scene vision SLAM method based on the improved dotted line feature as claimed in any one of claims 1 to 9, wherein: in the step S7, after a closed-loop condition is detected, the camera pose and the three-dimensional map points of the structured scene are optimized through a dotted line characteristic error model, so as to improve the effect of positioning and map building at the same time; the method comprises the following specific steps:
s7.1, after the closed loop is detected, the point line feature set { p ] of the current frame is collectedref，lrefPoint-line feature set { p } of key frame with highest similarity obtained by closed-loop detectionkey，lkeyCorresponding to the line features, and respectively matching the corresponding ORB point features with the LSD line features; wherein p isrefIs the point feature of the current frame, lrefIs a line feature of the current frame, pkeyIs a point feature of the key frame,/keyLine features for the keyframes;
s7.2, traversing a set K { ∑ (p, l) } of the current frame and the key frame, fusing all matched ORB point features and LSD line features, minimizing reprojection errors through a cost function, optimizing the camera pose and all structured scene three-dimensional map points, and improving the effect of positioning and mapping at the same time;
the expression of the cost function is as follows:
Figure FDA0002245596780000053
wherein i represents the ith matched ORB point feature, j represents the jth matched LSD line feature, and N and M respectively represent the number of the matched ORB point feature and the LSD line feature; eref，EkeyA two-norm error model of the improved dotted line features of the current video frame and the keyframe, respectively.
CN201911015560.3A 2019-10-24 2019-10-24 Structured scene vision SLAM method based on improved point-line characteristics Active CN110853100B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911015560.3A CN110853100B (en) 2019-10-24 2019-10-24 Structured scene vision SLAM method based on improved point-line characteristics

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911015560.3A CN110853100B (en) 2019-10-24 2019-10-24 Structured scene vision SLAM method based on improved point-line characteristics

Publications (2)

Publication Number Publication Date
CN110853100A true CN110853100A (en) 2020-02-28
CN110853100B CN110853100B (en) 2022-11-18

Family

ID=69597052

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911015560.3A Active CN110853100B (en) 2019-10-24 2019-10-24 Structured scene vision SLAM method based on improved point-line characteristics

Country Status (1)

Country Link
CN (1) CN110853100B (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111444853A (en) * 2020-03-27 2020-07-24 长安大学 Loop detection method of visual S L AM
CN112087728A (en) * 2020-07-21 2020-12-15 华为技术有限公司 Method and device for acquiring Wi-Fi fingerprint spatial distribution and electronic equipment
CN112307917A (en) * 2020-10-21 2021-02-02 国网上海市电力公司 Indoor positioning method integrating visual odometer and IMU
CN112330735A (en) * 2020-10-26 2021-02-05 武汉中海庭数据技术有限公司 Method and system for evaluating confidence of measurement accuracy of relative position of vehicle body
CN112381890A (en) * 2020-11-27 2021-02-19 上海工程技术大学 RGB-D vision SLAM method based on dotted line characteristics
CN112734839A (en) * 2020-12-31 2021-04-30 浙江大学 Monocular vision SLAM initialization method for improving robustness
CN113052977A (en) * 2021-03-30 2021-06-29 联想(北京)有限公司 Processing method and device
CN113298014A (en) * 2021-06-09 2021-08-24 安徽工程大学 Closed loop detection method, storage medium and equipment based on reverse index key frame selection strategy
CN114370879A (en) * 2022-01-14 2022-04-19 东南大学 AUV robust VBHIAKF-SLAM navigation method based on underwater environment characteristics
CN114529807A (en) * 2022-02-24 2022-05-24 广东工业大学 Loop detection method, device, equipment and storage medium for point, line and surface feature fusion
CN114750147A (en) * 2022-03-10 2022-07-15 深圳甲壳虫智能有限公司 Robot space pose determining method and device and robot
CN114862949A (en) * 2022-04-02 2022-08-05 华南理工大学 Structured scene vision SLAM method based on point, line and surface characteristics
CN117893693A (en) * 2024-03-15 2024-04-16 南昌航空大学 Dense SLAM three-dimensional scene reconstruction method and device

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108648240A (en) * 2018-05-11 2018-10-12 东南大学 Based on a non-overlapping visual field camera posture scaling method for cloud characteristics map registration
CN109165680A (en) * 2018-08-01 2019-01-08 东南大学 Single target object dictionary model refinement method under the indoor scene of view-based access control model SLAM

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108648240A (en) * 2018-05-11 2018-10-12 东南大学 Based on a non-overlapping visual field camera posture scaling method for cloud characteristics map registration
CN109165680A (en) * 2018-08-01 2019-01-08 东南大学 Single target object dictionary model refinement method under the indoor scene of view-based access control model SLAM

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
贾松敏等: "RTM框架下基于点线特征的视觉SLAM算法", 《机器人》 *

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111444853A (en) * 2020-03-27 2020-07-24 长安大学 Loop detection method of visual S L AM
CN111444853B (en) * 2020-03-27 2023-04-07 长安大学 Loop detection method of visual SLAM
CN112087728A (en) * 2020-07-21 2020-12-15 华为技术有限公司 Method and device for acquiring Wi-Fi fingerprint spatial distribution and electronic equipment
CN112087728B (en) * 2020-07-21 2022-01-14 华为技术有限公司 Method and device for acquiring Wi-Fi fingerprint spatial distribution and electronic equipment
CN112307917A (en) * 2020-10-21 2021-02-02 国网上海市电力公司 Indoor positioning method integrating visual odometer and IMU
CN112330735A (en) * 2020-10-26 2021-02-05 武汉中海庭数据技术有限公司 Method and system for evaluating confidence of measurement accuracy of relative position of vehicle body
CN112330735B (en) * 2020-10-26 2022-06-17 武汉中海庭数据技术有限公司 Method and system for evaluating confidence of measurement accuracy of relative position of vehicle body
CN112381890A (en) * 2020-11-27 2021-02-19 上海工程技术大学 RGB-D vision SLAM method based on dotted line characteristics
CN112381890B (en) * 2020-11-27 2022-08-02 上海工程技术大学 RGB-D vision SLAM method based on dotted line characteristics
CN112734839A (en) * 2020-12-31 2021-04-30 浙江大学 Monocular vision SLAM initialization method for improving robustness
CN113052977A (en) * 2021-03-30 2021-06-29 联想(北京)有限公司 Processing method and device
CN113298014A (en) * 2021-06-09 2021-08-24 安徽工程大学 Closed loop detection method, storage medium and equipment based on reverse index key frame selection strategy
CN114370879A (en) * 2022-01-14 2022-04-19 东南大学 AUV robust VBHIAKF-SLAM navigation method based on underwater environment characteristics
CN114370879B (en) * 2022-01-14 2023-03-10 东南大学 AUV robust VBHIAKF-SLAM navigation method based on underwater environment characteristics
CN114529807A (en) * 2022-02-24 2022-05-24 广东工业大学 Loop detection method, device, equipment and storage medium for point, line and surface feature fusion
CN114529807B (en) * 2022-02-24 2022-08-16 广东工业大学 Loop detection method, device, equipment and storage medium for point, line and surface feature fusion
CN114750147A (en) * 2022-03-10 2022-07-15 深圳甲壳虫智能有限公司 Robot space pose determining method and device and robot
CN114750147B (en) * 2022-03-10 2023-11-24 深圳甲壳虫智能有限公司 Space pose determining method and device of robot and robot
CN114862949A (en) * 2022-04-02 2022-08-05 华南理工大学 Structured scene vision SLAM method based on point, line and surface characteristics
CN114862949B (en) * 2022-04-02 2024-05-17 华南理工大学 Structured scene visual SLAM method based on dot-line surface characteristics
CN117893693A (en) * 2024-03-15 2024-04-16 南昌航空大学 Dense SLAM three-dimensional scene reconstruction method and device
CN117893693B (en) * 2024-03-15 2024-05-28 南昌航空大学 Dense SLAM three-dimensional scene reconstruction method and device

Also Published As

Publication number Publication date
CN110853100B (en) 2022-11-18

Similar Documents

Publication Publication Date Title
CN110853100B (en) Structured scene vision SLAM method based on improved point-line characteristics
CN109345588B (en) Tag-based six-degree-of-freedom attitude estimation method
CN110853075B (en) Visual tracking positioning method based on dense point cloud and synthetic view
CN110568447A (en) Visual positioning method, device and computer readable medium
CN111861882A (en) Image processing method, image processing apparatus, three-dimensional object modeling method, three-dimensional object modeling apparatus, image processing apparatus, and medium
CN113674416B (en) Three-dimensional map construction method and device, electronic equipment and storage medium
CN110009732B (en) GMS feature matching-based three-dimensional reconstruction method for complex large-scale scene
CN112752028B (en) Pose determination method, device and equipment of mobile platform and storage medium
Taylor et al. Fusion of multimodal visual cues for model-based object tracking
CN111797688A (en) Visual SLAM method based on optical flow and semantic segmentation
CN111127522A (en) Monocular camera-based depth optical flow prediction method, device, equipment and medium
Koide et al. General, single-shot, target-less, and automatic lidar-camera extrinsic calibration toolbox
CN110570474A (en) Pose estimation method and system of depth camera
Bu et al. Semi-direct tracking and mapping with RGB-D camera for MAV
CN113393524A (en) Target pose estimation method combining deep learning and contour point cloud reconstruction
Redzuwan et al. Affine versus projective transformation for SIFT and RANSAC image matching methods
Wang et al. Feature-based RGB-D camera pose optimization for real-time 3D reconstruction
Kim et al. Indoor RGB-D compass from a single line and plane
Xiang et al. Vilivo: Virtual lidar-visual odometry for an autonomous vehicle with a multi-camera system
Aranda et al. Omnidirectional visual homing using the 1D trifocal tensor
CN111882663A (en) Visual SLAM closed-loop detection method achieved by fusing semantic information
Zhang et al. Point-plane SLAM based on line-based plane segmentation approach
Laskar et al. Robust loop closures for scene reconstruction by combining odometry and visual correspondences
CN111915632B (en) Machine learning-based method for constructing truth database of lean texture target object
CN114972491A (en) Visual SLAM method, electronic device, storage medium and product

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant