CN114694013B - Distributed multi-machine cooperative vision SLAM method and system - Google Patents
Distributed multi-machine cooperative vision SLAM method and system Download PDFInfo
- Publication number
- CN114694013B CN114694013B CN202210373004.9A CN202210373004A CN114694013B CN 114694013 B CN114694013 B CN 114694013B CN 202210373004 A CN202210373004 A CN 202210373004A CN 114694013 B CN114694013 B CN 114694013B
- Authority
- CN
- China
- Prior art keywords
- algorithm
- closed
- robots
- loop
- optimization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Manipulator (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
Abstract
The invention discloses a distributed multi-machine cooperative vision SLAM method and a system, which comprises the following steps: running a visual SLAM program on the robot, acquiring an image through a camera, and extracting a NetVLAD global descriptor based on the image; judging by using an Euclidean distance based on the NetVLAD global descriptor, and if the requirement of a communication range is met, performing communication between robots and determining a closed loop between the robots; based on the robots determining the closed loop among the robots, the DGS1 algorithm is utilized to perform distributed graph optimization, and the cooperative positioning among a plurality of robots is determined. The invention effectively eliminates error closed loops, adopts distributed pose graph optimization to realize multi-machine cooperative positioning, enhances positioning capability, has robustness to illumination, and has small data transmission quantity and high optimization efficiency.
Description
Technical Field
The invention belongs to the technical field of synchronous positioning and map construction of a multi-machine cooperative mobile robot, and particularly relates to a distributed multi-machine cooperative vision SLAM method and system.
Background
Synchronous positioning and mapping (SLAM for short) refers to a carrier carrying a specific sensor, positioning itself in an unknown environment during movement, and building a map of the environment. SLAM can effectively solve the positioning problem under the GPS signal loss, and is widely applied to the fields of unmanned vehicles, robots and the like. The existing visual SLAM method based on a single machine has a mature solution, but with the expansion of the working environment, the single machine SLAM has low efficiency, the estimation time is increased, the estimation precision is reduced, and a good effect is often difficult to achieve. To solve this problem, the SLAM scheme of multi-machine collaboration has attracted a great deal of attention in both academic and industrial fields. The multi-machine cooperative vision SLAM generally refers to a group of mobile robots, carries a camera to acquire environment information in an unknown environment, estimates and optimizes self positioning information through data exchange, and establishes a map of the environment. Data association among robots is usually realized by identifying and determining a closed loop among multiple robots and calculating a relative pose transformation matrix by using similar scenes.
At present, the basic framework of multi-machine cooperative vision SLAM includes two parts, front-end inter-machine closed loop detection and back-end optimization. The existing multi-machine cooperative SLAM framework is mainly divided into a centralized framework and a distributed framework, the distributed framework is concerned due to good privacy and loose computing resource requirements, and the distributed framework is also adopted in the invention. The front end of the conventional multi-machine cooperative SLAM mainly processes image input through a bag-of-words model and is applied to scene identification to find an inter-machine closed loop. Because scene identification based on vision is often influenced by illumination brightness change, the invention adopts a global image descriptor NetVLAD based on a convolution neural network to carry out closed-loop detection between machines, thereby effectively reducing the influence of light on scene identification and enhancing the robustness of the system.
By calculating the Euclidean distance between the descriptors, whether the robot passes through a similar scene or not is judged to form a closed loop among multiple robots, and the method can realize the scene identification function by light-weight data transmission quantity. And performing geometric verification by using a PNP algorithm, and preliminarily screening a closed-loop candidate set. Afterwards, a Pairwise consistency detection method (PCM algorithm for short) is utilized, potential error closed loops can be further eliminated, and error optimization is avoided to influence system precision. The currently widely used multi-machine cooperative SLAM back-end optimization scheme is a Distributed Gauss-Seidel algorithm (DGS algorithm for short). However, the algorithm has high accuracy requirement on the rotation vector estimated in the first step, and inaccurate estimation can cause the convergence speed to become slow. Therefore, the invention is improved on the basis of the method, and the estimation precision and the estimation efficiency are improved on the basis of not increasing the data transmission quantity.
Disclosure of Invention
The invention aims to provide a distributed multi-machine cooperative vision SLAM method and system, which can effectively eliminate error closed loops, realize multi-machine cooperative positioning by adopting distributed pose graph optimization, enhance positioning capability, have robustness to illumination, have small data transmission quantity and high optimization efficiency.
In order to achieve the above object, the present invention provides a distributed multi-machine cooperative vision SLAM method, which comprises the following steps:
running a visual SLAM program on the robot, acquiring an image through a camera, and extracting a NetVLAD global descriptor based on the image;
judging by using an Euclidean distance based on the NetVLAD global descriptor, and if the requirement of a communication range is met, performing communication between robots and determining a closed loop between the robots;
based on the robots determining the closed loop among the robots, distributed graph optimization is performed by using a DGS1 algorithm, and the cooperative positioning among a plurality of robots is determined.
Optionally, the process of extracting the NetVLAD global descriptor based on the image includes: before the system runs, acquiring a NetVLAD test data set based on a scene training related image used by a robot; after the system is operated, when the communication range between the robots is not within the satisfying range, the visual SLAM program is loaded based on the images acquired by the camera, and each robot extracts the NetVLAD global descriptor of the images according to the trained NetVlad test data set.
Optionally, the process of determining the closed loop between the robots includes: when the communication range between the robots is within the satisfying range, calculating the NetVLAD global descriptor based on the NetVLAD global descriptor of each robot extracted image, judging the Euclidean distance between different image frame descriptors, determining the closed loop of the robots and acquiring a closed loop candidate set of the robots.
Optionally, judging that the condition that the optimization interval is too close is met based on the robot closed-loop candidate set includes: based on the robot closed-loop candidate set, image frames are obtained, whether the optimization interval between two adjacent closed loops is too close or not is judged based on the number of the image frames, and whether the optimization of the closed-loop images among multiple robots is selected or not is determined.
Optionally, geometric verification is performed based on a PNP algorithm to extract a closed-loop candidate set and estimate relative pose transformation among multiple machines: calculating an image frame by utilizing a solvePlanpanac function in OpenCV, acquiring a relative pose transformation matrix between two robots generating closed loop matching, and returning to an interior point; when the number of the interior points is large enough, judging the interior points as closed-loop candidate frames; and setting an internal point quantity threshold value to screen closed-loop candidate frames, acquiring visual characteristics, and determining relative pose change among robots.
Optionally, the determining an optimized closed-loop set by removing the error closed loop based on the closed-loop candidate set through a PCM algorithm includes: and performing pairwise consistency detection on the closed-loop candidate set extracted based on geometric verification by using a PCM algorithm, and rejecting a wrong closed loop by setting a PCM outlier threshold to obtain a closed-loop optimization candidate set.
Optionally, performing distributed graph optimization by using a DGS1 algorithm, and determining the co-location among the plurality of robots includes: the method comprises the steps of mutually transmitting pose information under image frames generating closed loops among robots, optimizing a closed loop candidate set, performing distributed pose graph optimization through a DGS1 algorithm, performing rotary variable estimation on the screened closed loop candidate set, optimizing a translation variable, solving a pose optimization problem by utilizing an SOR algorithm based on the rotary variable and the translation variable, repeatedly exchanging poses of the closed loops among the robots until an optimal track is formed, obtaining an optimized rotary component and translation component, and determining robot positioning.
In another aspect, to achieve the above object, the present invention provides a distributed multi-machine cooperative vision SLAM system, including:
a data acquisition module: running a visual SLAM program on the robot, acquiring an image through a camera, and extracting a NetVLAD global descriptor based on the image;
a closed loop identification module: based on the NetVLAD global descriptor, if the requirement of a communication range is met, communication is carried out among the robots, descriptor information is exchanged, and the Euclidean distance is utilized to judge whether different robots pass through the same scene, so that closed loops among the robots are determined;
a positioning module: based on the robots determining the closed loop among the robots, the DGS1 algorithm is utilized to perform distributed graph optimization, and the cooperative positioning among a plurality of robots is determined.
The invention has the technical effects that: the invention discloses a distributed multi-machine cooperative vision SLAM method and a distributed multi-machine cooperative vision SLAM system, which solve the problem of autonomous positioning of a robot in a large-scale operation environment, firstly, the front end described by the invention realizes efficient inter-machine closed loop detection by utilizing a NetVLAD global descriptor, solves the problem of data association among multiple machines, and reduces the limitation of the multi-machine system on bandwidth by less data exchange amount; secondly, the method for eliminating the error closed loop can realize accurate closed loop extraction on the premise of not increasing data exchange quantity so as to avoid hidden danger caused by the error closed loop; finally, the invention optimizes the data exchanged by multiple robots by an optimized DGS1 method, and realizes the improvement of the self-positioning precision and the optimization efficiency of each robot on the premise of protecting the privacy of data transmission.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, are included to provide a further understanding of the application, and the description of the exemplary embodiments of the application are intended to be illustrative of the application and are not intended to limit the application. In the drawings:
FIG. 1 is a schematic flow chart of a distributed multi-machine cooperative vision SLAM method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a data association implemented by an inter-machine closed loop according to a first embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a multi-machine closed-loop detection process according to an embodiment of the present invention;
FIG. 4 is a schematic diagram illustrating a principle of a PCM outlier culling algorithm according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a distributed multi-machine cooperative vision SLAM system according to a second embodiment of the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than here.
Example one
As shown in fig. 1-4, the present embodiment provides a distributed multi-machine cooperative vision SLAM method, which includes the following steps:
running a visual SLAM program on the robot, acquiring an image through a camera, and extracting a NetVLAD global descriptor based on the image;
based on the NetVLAD global descriptor, if the requirement of a communication range is met, communication is carried out among the robots, descriptor information is exchanged, and the Euclidean distance is utilized to judge whether different robots pass through the same scene, so that closed loops among the robots are determined;
based on the robots determining the closed loop among the robots, distributed graph optimization is performed by using a DGS1 algorithm, and the cooperative positioning among a plurality of robots is determined.
According to a further optimization scheme, the process of extracting the NetVLAD global descriptor based on the image comprises the following steps: before the system runs, acquiring a NetVLAD test data set based on a scene training related image used by a robot; after the system is operated, when the communication range between the robots is not within the satisfying range, the camera-based image acquisition is carried out to load the visual SLAM program, and each robot extracts the NetVLAD global descriptor of the image according to the trained NetVlad test data set.
Further optimizing the scheme, the process of determining the closed loop between the robots comprises the following steps: when the communication range between the robots is within the satisfying range, calculating the NetVLAD global descriptor to judge the Euclidean distance based on the NetVLAD global descriptor of the image extracted by each robot, determining the closed loop of the robots and acquiring a closed loop candidate set of the robots.
Further optimizing the scheme, judging that the condition of meeting the optimization interval too close based on the robot closed-loop candidate set comprises the following steps: based on the robot closed-loop candidate set, image frames are obtained, whether the optimization interval between two adjacent closed loops is too close or not is judged based on the number of the image frames, and whether the optimization of the closed-loop images among multiple robots is selected or not is determined.
And further optimizing the scheme, performing geometric verification based on a PNP algorithm to extract a closed-loop candidate set and estimate relative pose transformation among multiple machines: calculating an image frame by utilizing a solvePlanpanac function in OpenCV, acquiring a relative pose transformation matrix between two robots generating closed loop matching, and returning to an interior point; when the number of the interior points is large enough, judging the interior points as closed-loop candidate frames; and setting an internal point quantity threshold value to screen closed-loop candidate frames, acquiring visual characteristics and determining relative pose transformation among robots.
Further optimizing the scheme, reject the wrong closed loop through PCM algorithm, on the basis of the candidate set of closed loop, confirm that optimizes the closed loop set, including: and performing pairwise consistency detection on the closed-loop candidate set extracted based on geometric verification by using a PCM algorithm, and rejecting a wrong closed loop by setting a PCM outlier threshold to obtain a closed-loop optimization candidate set.
Further optimizing the scheme, carrying out distributed graph optimization by utilizing a DGS1 algorithm, and determining the cooperative positioning among a plurality of robots comprises the following steps: the method comprises the steps of mutually transmitting pose information under image frames generating closed loops among robots, optimizing a closed loop candidate set, performing distributed pose graph optimization through a DGS1 algorithm, performing rotary variable estimation on the screened closed loop candidate set, optimizing a translation variable, solving a pose optimization problem by utilizing an SOR algorithm based on the rotary variable and the translation variable, repeatedly exchanging poses of the closed loops among the robots until an optimal track is formed, obtaining an optimized rotary component and translation component, and determining robot positioning.
Example two
As shown in fig. 5, the present embodiment provides a distributed multi-machine cooperative vision SLAM system, including:
a data acquisition module: running a visual SLAM program on the robot, acquiring an image through a camera, and extracting a NetVLAD global descriptor based on the image;
a closed loop identification module: based on the NetVLAD global descriptor, if the requirement of a communication range is met, communication is carried out among the robots, descriptor information is exchanged, and the Euclidean distance is utilized to judge whether different robots pass through the same scene, so that closed loops among the robots are determined;
a positioning module: based on the robots determining the closed loop among the robots, distributed graph optimization is performed by using a DGS1 algorithm, and the cooperative positioning among a plurality of robots is determined.
Step 1: visual SLAM front based on NetVlad global descriptor. The NetVlad test data set is first trained according to the application scenario and loaded before SLAM starts. When the robots do not meet, namely, the robots are not in a communication range, each robot independently runs a visual SLAM program, and the acquired images are subjected to NetVLAD global descriptor extraction.
The following is a specific process:
training a NetVlad test data set based on the use scene of the robot, importing the NetVlad test data set into a processor of each robot, and loading the NetVlad test data set before starting SLAM. For example, the NetVLAD test dataset is trained using random feature extraction from several sequences in the Oxford RobotCar dataset. When the robots do not meet, each robot independently runs the visual SLAM program by using the image information of the environment acquired by the camera. For example, the open source visual SLAM scheme ORB-SLAM2 is employed as the visual SLAM scheme for a single robot. And according to the trained NetVlad test data set, extracting and storing a global descriptor of the current frame image by each robot.
Step 2: the method is characterized in that the method realizes the inter-machine closed loop detection based on the NetVLAD global descriptor, and eliminates potential error closed loops by utilizing PNP geometric verification and a PCM algorithm. When the robots meet each other, namely the distance between the robots meets the communication range, all image descriptors after the last meeting are transmitted between the robots, and a closed-loop candidate set is determined according to the Euclidean norm of the calculation descriptors. And judging whether the conditions of too close meeting occur or not according to the number of image frames experienced between two adjacent closed-loop optimizations so as to determine whether the optimization is performed or not.
The robots which have met mutually transmit all image descriptors after the last meeting, calculate the Euclidean norm to preliminarily judge whether the similar scene is reached, and generate a closed-loop candidate set as shown in the formula. Wherein
A NetVLAD descriptor of the i-frame image of robot alpha,
NetVLAD descriptor of robot beta in j frame image, tau NetVLAD The distance threshold for artificial setting may be set to 0.15, for example, and the larger the threshold is, the looser the closed-loop condition is, and the stricter the closed-loop condition is. Note that if too close to the last inter-machine closed loop optimization, this optimization will be abandoned to avoid computational overload and ill-conditioned graph optimization. The number of image frames from the previous time of the closed loop optimization to the current closed loop is judged whether the meeting time is too close, for example, the minimum number of image frames is set to be 50 frames,
and for the image frame generating the inter-machine closed loop, extracting visual features and corresponding three-dimensional position estimation, calculating by using a solvepnp Ranac function in OpenCV to obtain a relative pose transformation matrix between two robots generating the closed loop matching, and returning to a group of interior points. If the number of inliers is large enough, the secondary closed loop may be considered to be relatively successful. The partial closed loop can be preliminarily screened by setting an inner point quantity threshold value. The relative pose transformation matrix between the two robots can be solved, so that the system allows the robots in the initial state to be randomly placed without measuring the position relation between the robots in advance.
Further eliminating potential error closed loops in the closed loop candidate set by utilizing a PCM algorithm, and determining closed loops finally used for optimizationAnd (4) a ring set for enhancing the robustness of the system, wherein the algorithm principle is shown in figure 3. The PCM is used to check the consistency of the closed loop between machines, and to measure whether the closed loop is correct or not. In the formula | · | non-conducting phosphor Σ Which represents the mahalanobis distance,
and a relative pose transformation matrix representing the robot alpha from a frame j to a frame i, other symbols have similar meanings, and gamma represents a PCM detection threshold and can be set to 0.01, for example. A higher threshold indicates a looser requirement for the closed loop, whereas a stricter one,
and step 3: from step 2, the robot numbers with good inter-robot closed loops can be obtained, and the robots with the inter-robot closed loops will all execute the following distributed graph optimization process. The robots mutually transmit pose information under a closed-loop image frame, and distributed pose image optimization is performed based on a DGS1 algorithm. The DGS algorithm and the multi-machine cooperative SLAM have the following models for alpha with the inter-machine closed loop constraint i And beta j And a relative pose transformation matrix:
the robots mutually transmit pose information under an image frame which generates a closed loop,
wherein
A rotation matrix, R, representing the robot α in i frames ò And t ò Is the measurement noise.
The first step of the DGS1 algorithm is performed to calculate the optimized rotation vector from the formula, where
Is rotational noise. The cost function is composed of two parts, wherein the first part is the closed loop constraint between machines < alpha generated by the current encounter i ,β j And > etc. The second part is virtual closed-loop constraint, for robots with more than one inter-robot closed-loop match, e.g. with constraint < alpha i ,β j ＞、＜α k ,χ l > can form single-machine relative pose transformation matrix constraint < alpha i ,α k > (ii). For the robot with only single inter-machine closed-loop matching, the second constraint can be formed by utilizing the closed-loop constraint in the last meeting process without additional calculation. The introduction of the virtual closed-loop constraint is beneficial to enhancing the global consistency of the estimation of the system, and further utilizes the historical closed-loop information to improve the precision of the first-step rotation estimation in the optimization of the distributed graph, thereby guiding faster convergence speed,
and carrying out a second step of the DGS1 algorithm, substituting the optimized rotation variable, and preliminarily optimizing the translation variable. And in the second step of the DGS algorithm, the rotation variables are initialized by utilizing the rotation estimation obtained in the first step, and the SOR algorithm is adopted to solve the full attitude optimization problem. The iterative convergence speed of the SOR is related to the selection of the initial optimization sequence, and from the aspect of accelerating the convergence of the algorithm, the SOR algorithm is improved, the steepest Descent (SD algorithm) and the SOR algorithm are combined, the SD algorithm is used for providing an effective search direction for the SOR algorithm, the convergence speed is accelerated, and the performance of the algorithm is improved. The SD algorithm may approach the optimal solution quickly, but the convergence speed near the optimal solution will be slower, complementary to the SOR algorithm, thus combining the SD algorithm with the SOR algorithmThe algorithm efficiency is improved. DGS1, on the basis of the rotation variable optimized in the first step, solving the translation variable subjected to preliminary optimization by adopting a steepest descent method, wherein the optimization function is a formula
In order to shift the noise in a horizontal direction,
and carrying out a third step of DGS1 algorithm, substituting the initial rotation estimation obtained in the first step and the initial translation estimation obtained in the second step, solving the full-attitude optimization problem by using an SOR algorithm, and repeatedly exchanging the attitudes related to the closed loop among the robots until the attitudes reach a consensus on the optimal trajectory estimation so as to improve the positioning accuracy of the robots and obtain the optimized rotation component and translation component.
The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (8)
1. A distributed multi-machine cooperative vision SLAM method is characterized by comprising the following steps:
running a visual SLAM program on the robot, acquiring an image through a camera, and extracting a NetVLAD global descriptor based on the image;
judging by using an Euclidean distance based on the NetVLAD global descriptor, and if the requirement of a communication range is met, performing communication between robots and determining a closed loop between the robots;
based on the robots determining the closed loop among the robots, carrying out distributed graph optimization by using a DGS1 algorithm, and determining the cooperative positioning among a plurality of robots;
image frames transmitted mutually between robots to generate closed loopDistributed pose graph optimization is carried out on the pose information based on a DGS1 algorithm; the multi-machine cooperative SLAM has the following model for alpha with closed-loop constraint between machines i And beta j And a relative pose transformation matrix:
pose information under image frames of closed loop generated by mutual transmission between robots
Wherein
Representing the rotation matrix of the robot alpha under the i frame, R ∈ And t ∈ Is the measurement noise;
the first step of the DGS1 algorithm is performed, and the optimized rotation vector is calculated by the following formula
carrying out a second step of the DGS1 algorithm, substituting the optimized rotation variable, and primarily optimizing a translation variable; initializing a rotation variable by using the rotation estimation obtained in the first step, and solving a full attitude optimization problem by adopting an SOR algorithm; the iterative convergence speed of the SOR is related to the selection of the initial optimization sequence, the SOR algorithm is improved from the aspect of accelerating algorithm convergence, the steepest descent SD algorithm and the SOR algorithm are combined, an effective search direction is provided for the SOR algorithm by using the steepest descent SD algorithm, the convergence speed is accelerated, and the algorithm performance is improved; the steepest descent SD algorithm is close to the optimal solution quickly, but the convergence speed near the optimal solution is reduced and is complementary with the SOR algorithm, so that the combination of the steepest descent SD algorithm and the SOR algorithm is favorable for improving the algorithm efficiency; DGS1, on the basis of the rotation variable optimized in the first step, adopting a steepest descent SD algorithm to solve the translation variable subjected to primary optimization, wherein the optimization function is as follows, wherein
In order to translate the noise in the direction of the noise,
and carrying out a third step of DGS1 algorithm, substituting the initial rotation estimation obtained in the first step and the initial translation estimation obtained in the second step, solving the full-attitude optimization problem by using an SOR algorithm, and repeatedly exchanging the attitudes related to the closed loop among the robots until the attitudes reach a consensus on the optimal trajectory estimation so as to improve the positioning accuracy of the robots and obtain the optimized rotation component and translation component.
2. The distributed multi-machine collaborative vision SLAM method of claim 1, wherein the process of extracting a NetVLAD global descriptor based on the image comprises: before the system runs, acquiring a NetVLAD test data set based on a scene training related image used by a robot; after the system is operated, when the communication range between the robots is not within the satisfying range, the camera is used for acquiring images to load a visual SLAM program, and each robot extracts a NetVLAD global descriptor of the images according to a trained NetVLAD test data set.
3. The distributed multi-machine collaborative vision SLAM method of claim 2, wherein the process of determining an inter-robot closed loop comprises: when the communication range between the robots is within a satisfying range, calculating a NetVLAD global descriptor of an image extracted by each robot to judge the Euclidean distance, determining the closed loop of the robots and acquiring a closed loop candidate set of the robots based on the NetVLAD global descriptor.
4. The distributed multi-machine cooperative vision SLAM method of claim 3, wherein determining that an optimization interval too close condition is satisfied based on the robot closed-loop candidate set comprises: based on the robot closed-loop candidate set, image frames are obtained, whether the optimization interval between two adjacent closed loops is too close or not is judged based on the number of the image frames, and whether the optimization of the closed-loop images among multiple robots is selected or not is determined.
5. The distributed multi-machine cooperative vision SLAM method of claim 4, wherein a PNP algorithm is used for geometric validation to extract a closed-loop candidate set and estimate the relative pose transformation among multiple machines: calculating an image frame by utilizing a solvePtRanac function in OpenCV, acquiring a relative pose transformation matrix between two robots generating closed-loop matching, and returning to an interior point; when the number of the inner points is large enough, judging the inner points as closed-loop candidate frames; and setting an internal point quantity threshold value to screen closed-loop candidate frames, acquiring visual characteristics and determining relative pose transformation among robots.
6. The distributed multi-machine cooperative vision SLAM method of claim 5, wherein determining an optimized closed-loop set by removing erroneous closed-loops based on a closed-loop candidate set through a PCM algorithm, comprises: and performing pairwise consistency detection on the closed-loop candidate set extracted based on geometric verification by using a PCM algorithm, and rejecting a wrong closed loop by setting a PCM outlier threshold to obtain a closed-loop optimization candidate set.
7. The distributed multi-machine cooperative vision SLAM method of claim 6, wherein distributed graph optimization using a DGS1 algorithm, determining a plurality of inter-robot co-locations comprises: the method comprises the steps of mutually transmitting pose information under image frames generating closed loops among robots, optimizing a closed loop candidate set, performing distributed pose graph optimization through a DGS1 algorithm, performing rotary variable estimation on the screened closed loop candidate set, optimizing a translation variable, solving a pose optimization problem by utilizing an SOR algorithm based on the rotary variable and the translation variable, repeatedly exchanging poses of the closed loops among the robots until an optimal track is formed, obtaining an optimized rotary component and translation component, and determining robot positioning.
8. A distributed multi-machine cooperative vision SLAM system, comprising:
a data acquisition module: running a visual SLAM program on the robot, acquiring an image through a camera, and extracting a NetVLAD global descriptor based on the image;
a closed loop identification module: based on the NetVLAD global descriptor, if the requirement of a communication range is met, communication is carried out among the robots, descriptor information is exchanged, and the Euclidean distance is utilized to judge whether different robots pass through the same scene, so that closed loops among the robots are determined;
a positioning module: based on the robots determining the closed loop among the robots, carrying out distributed graph optimization by using a DGS1 algorithm, and determining the cooperative positioning among a plurality of robots; the robots mutually transmit to generate position and attitude information under the image frame of a closed loop, and the position and attitude information is processed based on a DGS1 algorithmOptimizing a distributed pose graph; the multi-machine cooperative SLAM has the following model for alpha with closed-loop constraint between machines i And beta j There is a relative pose transformation matrix:
pose information under image frames of closed loop generated by mutual transmission between robots
Wherein
Representing the rotation matrix of the robot alpha under the i frame, R ∈ And t ∈ Is the measurement noise;
the first step of the DGS1 algorithm is performed to calculate the optimized rotation vector from the following formula
carrying out a second step of the DGS1 algorithm, substituting the optimized rotation variable, and primarily optimizing a translation variable; initializing a rotation variable by using the rotation estimation obtained in the first step, and solving a full attitude optimization problem by adopting an SOR algorithm; the iterative convergence speed of the SOR is related to the selection of the initial optimization sequence, the SOR algorithm is improved from the aspect of accelerating algorithm convergence, the steepest descent SD algorithm and the SOR algorithm are combined, an effective search direction is provided for the SOR algorithm by using the steepest descent SD algorithm, the convergence speed is accelerated, and the algorithm performance is improved; the steepest descent SD algorithm is close to the optimal solution quickly, but the convergence speed near the optimal solution is reduced and is complementary with the SOR algorithm, so that the combination of the steepest descent SD algorithm and the SOR algorithm is favorable for improving the algorithm efficiency; DGS1, on the basis of the rotation variable optimized in the first step, adopting a steepest descent SD algorithm to solve the translation variable subjected to primary optimization, wherein the optimization function is as follows, wherein
In order to shift the noise in a horizontal direction,
and carrying out a third step of DGS1 algorithm, substituting the initial rotation estimation obtained in the first step and the initial translation estimation obtained in the second step, solving the full-attitude optimization problem by using an SOR algorithm, and repeatedly exchanging the attitudes related to the closed loop among the robots until the attitudes reach a consensus on the optimal trajectory estimation so as to improve the positioning accuracy of the robots and obtain the optimized rotation component and translation component.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210373004.9A CN114694013B (en) | 2022-04-11 | 2022-04-11 | Distributed multi-machine cooperative vision SLAM method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210373004.9A CN114694013B (en) | 2022-04-11 | 2022-04-11 | Distributed multi-machine cooperative vision SLAM method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114694013A CN114694013A (en) | 2022-07-01 |
| CN114694013B true CN114694013B (en) | 2022-11-15 |
Family
ID=82142044
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210373004.9A Active CN114694013B (en) | 2022-04-11 | 2022-04-11 | Distributed multi-machine cooperative vision SLAM method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114694013B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109800692A (en) * | 2019-01-07 | 2019-05-24 | 重庆邮电大学 | A kind of vision SLAM winding detection method based on pre-training convolutional neural networks |
| CN110070615A (en) * | 2019-04-12 | 2019-07-30 | 北京理工大学 | A kind of panoramic vision SLAM method based on polyphaser collaboration |
| CN110781790A (en) * | 2019-10-19 | 2020-02-11 | 北京工业大学 | Visual SLAM closed loop detection method based on convolutional neural network and VLAD |
| CN111258313A (en) * | 2020-01-20 | 2020-06-09 | 深圳市普渡科技有限公司 | Multi-sensor fusion SLAM system and robot |
| CN111652934A (en) * | 2020-05-12 | 2020-09-11 | Oppo广东移动通信有限公司 | Positioning method, map construction method, device, equipment and storage medium |
| CN111812978A (en) * | 2020-06-12 | 2020-10-23 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Cooperative SLAM method and system for multiple unmanned aerial vehicles |
| CN112562081A (en) * | 2021-02-07 | 2021-03-26 | 之江实验室 | Visual map construction method for visual layered positioning |
| CN113808269A (en) * | 2021-09-23 | 2021-12-17 | 视辰信息科技(上海)有限公司 | Map generation method, positioning method, system and computer readable storage medium |
| WO2022045431A1 (en) * | 2020-08-31 | 2022-03-03 | 네이버랩스 주식회사 | Position estimation method and system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105856230B (en) * | 2016-05-06 | 2017-11-24 | 简燕梅 | A kind of ORB key frames closed loop detection SLAM methods for improving robot pose uniformity |
| CN107680133A (en) * | 2017-09-15 | 2018-02-09 | 重庆邮电大学 | A kind of mobile robot visual SLAM methods based on improvement closed loop detection algorithm |
| US20200401617A1 (en) * | 2019-06-21 | 2020-12-24 | White Raven Ltd | Visual positioning system |
-
2022
- 2022-04-11 CN CN202210373004.9A patent/CN114694013B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109800692A (en) * | 2019-01-07 | 2019-05-24 | 重庆邮电大学 | A kind of vision SLAM winding detection method based on pre-training convolutional neural networks |
| CN110070615A (en) * | 2019-04-12 | 2019-07-30 | 北京理工大学 | A kind of panoramic vision SLAM method based on polyphaser collaboration |
| CN110781790A (en) * | 2019-10-19 | 2020-02-11 | 北京工业大学 | Visual SLAM closed loop detection method based on convolutional neural network and VLAD |
| CN111258313A (en) * | 2020-01-20 | 2020-06-09 | 深圳市普渡科技有限公司 | Multi-sensor fusion SLAM system and robot |
| CN111652934A (en) * | 2020-05-12 | 2020-09-11 | Oppo广东移动通信有限公司 | Positioning method, map construction method, device, equipment and storage medium |
| CN111812978A (en) * | 2020-06-12 | 2020-10-23 | 哈尔滨工业大学(深圳)(哈尔滨工业大学深圳科技创新研究院) | Cooperative SLAM method and system for multiple unmanned aerial vehicles |
| WO2022045431A1 (en) * | 2020-08-31 | 2022-03-03 | 네이버랩스 주식회사 | Position estimation method and system |
| CN112562081A (en) * | 2021-02-07 | 2021-03-26 | 之江实验室 | Visual map construction method for visual layered positioning |
| CN113808269A (en) * | 2021-09-23 | 2021-12-17 | 视辰信息科技(上海)有限公司 | Map generation method, positioning method, system and computer readable storage medium |
Non-Patent Citations (5)
| Title |
|---|
| DiSCo-SLAM: Distributed Scan Context-Enabled Multi-Robot LiDAR SLAM With Two-Stage Global-Local Graph Optimization;Yewei Huang et al;《IEEE ROBOTICS AND AUTOMATION LETTERS》;20220105;第7卷(第2期);1150-1157 * |
| Distributed Trajectory Estimation with Privacy and Communication Constraints: a Two-Stage Distributed Gauss-Seidel Approach;Siddharth Choudhary et al;《2016 IEEE International Conference on Robotics and Automation (ICRA)》;20160521;5261-5268 * |
| DOOR-SLAM: Distributed, Online, and Outlier Resilient SLAM for Robotic Teams;Pierre-Yves Lajoie et al;《IEEE ROBOTICS AND AUTOMATION LETTERS》;20200430;第5卷(第2期);1656-1663 * |
| 基于最小回环检测的多车协同SLAM框架;李博洋 等;《电子学报》;20211130(第11期);2241-2250 * |
| 面向数据共享的多无人机协同SLAM方法;史殿习 等;《计算机学报》;20210531;第44卷(第5期);983-998 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114694013A (en) | 2022-07-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112734852B (en) | Robot mapping method and device and computing equipment | |
| Chang et al. | Kimera-multi: a system for distributed multi-robot metric-semantic simultaneous localization and mapping | |
| CN113674416B (en) | Three-dimensional map construction method and device, electronic equipment and storage medium | |
| CN109671119A (en) | A kind of indoor orientation method and device based on SLAM | |
| WO2020258820A1 (en) | Mobile side vision fusion positioning method and system, and electronic device | |
| WO2019196476A1 (en) | Laser sensor-based map generation | |
| Belter et al. | Improving accuracy of feature-based RGB-D SLAM by modeling spatial uncertainty of point features | |
| WO2021169049A1 (en) | Method for glass detection in real scene | |
| WO2022003740A1 (en) | Method for determining the confidence of a disparity map through a self-adaptive learning of a neural network, and sensor system thereof | |
| CN111882602A (en) | Visual odometer implementation method based on ORB feature points and GMS matching filter | |
| CN111812978B (en) | Cooperative SLAM method and system for multiple unmanned aerial vehicles | |
| CN112233149A (en) | Scene flow determination method and device, storage medium and electronic device | |
| CN113902862B (en) | Visual SLAM loop verification system based on consistency cluster | |
| CN104182747A (en) | Object detection and tracking method and device based on multiple stereo cameras | |
| CN114743273A (en) | Human skeleton behavior identification method and system based on multi-scale residual error map convolutional network | |
| Zhu et al. | Fusing panoptic segmentation and geometry information for robust visual slam in dynamic environments | |
| CN112258647B (en) | Map reconstruction method and device, computer readable medium and electronic equipment | |
| CN114694013B (en) | Distributed multi-machine cooperative vision SLAM method and system | |
| CN114707611B (en) | Mobile robot map construction method, storage medium and equipment based on graph neural network feature extraction and matching | |
| Bai et al. | An improved ransac algorithm based on adaptive threshold for indoor positioning | |
| CN113744236B (en) | Loop detection method, device, storage medium and computer program product | |
| CN115239902A (en) | Method, device and equipment for establishing surrounding map of mobile equipment and storage medium | |
| CN116977189A (en) | Synchronous positioning and mapping method, device and storage medium | |
| CN112270357A (en) | VIO vision system and method | |
| CN114202579B (en) | Dynamic scene-oriented real-time multi-body SLAM system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |