CN110487274B - SLAM method and system for weak texture scene, navigation vehicle and storage medium - Google Patents
SLAM method and system for weak texture scene, navigation vehicle and storage medium Download PDFInfo
- Publication number
- CN110487274B CN110487274B CN201910694313.4A CN201910694313A CN110487274B CN 110487274 B CN110487274 B CN 110487274B CN 201910694313 A CN201910694313 A CN 201910694313A CN 110487274 B CN110487274 B CN 110487274B
- Authority
- CN
- China
- Prior art keywords
- matrix
- characteristic point
- point
- value
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 230000009466 transformation Effects 0.000 claims abstract description 30
- 238000004364 calculation method Methods 0.000 claims abstract description 16
- 238000010276 construction Methods 0.000 claims abstract description 12
- 238000013136 deep learning model Methods 0.000 claims abstract description 10
- 239000011159 matrix material Substances 0.000 claims description 62
- 238000005457 optimization Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 6
- 238000007476 Maximum Likelihood Methods 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 claims description 4
- 238000000354 decomposition reaction Methods 0.000 claims description 2
- 238000005259 measurement Methods 0.000 claims description 2
- 238000000605 extraction Methods 0.000 description 6
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
Landscapes
- Engineering & Computer Science (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an SLAM method and system for a weak texture scene, a navigation vehicle and a storage medium, and relates to the field of instant positioning and map construction. The method comprises the following steps: reading an image sequence, and extracting and matching feature points in the image sequence according to a preset deep learning model; resolving the pose of the camera to obtain pose transformation parameters; carrying out map point calculation on the feature point pairs in the matching result; generating a three-dimensional point cloud map at the current moment; and repeating the steps until all images in the image sequence are read, generating a three-dimensional point cloud map at all times, and completing real-time positioning and map construction of the image sequence. The method is suitable for the weak texture scene, realizes the application of the SLAM method in the weak texture scene, and can improve the precision and accuracy of the map construction.
Description
Technical Field
The invention relates to the field of instant positioning and map construction, in particular to an SLAM method, a system, a navigation vehicle and a storage medium for weak texture scenes.
Background
The SLAM (Simultaneous Localization And Mapping) problem can be described as: whether there is a way to let a robot move while gradually tracing a complete map of an unknown environment by placing the robot at an unknown position in the environment. The existing monocular SLAM method based on vision mainly comprises the steps of feature point detection and extraction, camera pose estimation, camera pose optimization, local map construction and the like.
At present, research aiming at autonomous navigation and positioning of robots still mainly focuses on solving the problem of drawing of structured scenes, and the scenes often contain a large number of artificial objects, such as tables and chairs, floor tiles, railings, and objects with obvious angle features, point features and line features on the outer wall of a building, and are beneficial to detecting and extracting feature points.
However, in the field of extraterrestrial planet detection, the deep space mapping task is directed at a weak texture scene, and the weak texture scene usually does not have or only has few line features and lacks obvious point features, so that the current three-dimensional mapping method is not suitable for the weak texture scene, and the problems of too few extracted feature points, more mismatching and the like easily occur, so that the three-dimensional mapping cannot be completed.
Disclosure of Invention
The invention aims to solve the technical problem of providing an SLAM method, a system, a navigation vehicle and a storage medium which can be used for weak texture scenes aiming at the defects of the prior art.
The technical scheme for solving the technical problems is as follows:
a SLAM method for weak texture scenes, comprising:
s1, acquiring an image sequence acquired by the monocular camera;
s2, reading a first frame image and a second frame image in the image sequence, and extracting and matching feature points in the first frame image and the second frame image according to a preset deep learning model;
s3, camera pose calculation is carried out according to the matching result, and pose transformation parameters of the second frame image relative to the first frame image are obtained;
s4, carrying out map point calculation on the feature point pairs in the matching result according to the pose transformation parameters, and converting the two-dimensional feature point pairs into three-dimensional space points;
s5, generating a three-dimensional point cloud map of the current time according to the three-dimensional space points;
and S6, repeating the steps S2 to S5 until all images in the image sequence are read, generating a three-dimensional point cloud map at all times, and completing real-time positioning and map construction of the image sequence.
The invention has the beneficial effects that: the SLAM method provided by the invention is suitable for weak texture scenes, the images of the weak texture scenes are subjected to feature point extraction and matching through the deep learning model, more suitable feature detection and extraction rules can be abstracted from samples, then map point calculation is carried out, a three-dimensional point cloud map is constructed, the application of the SLAM method in the weak texture scenes is realized, feature points with better stability and robustness can be provided based on the features of deep learning, the real-time positioning and map construction of the weak texture scenes are realized, and the precision and accuracy of map construction can be improved.
Another technical solution of the present invention for solving the above technical problems is as follows:
a storage medium, wherein instructions are stored, and when the instructions are read by a computer, the instructions cause the computer to execute the SLAM method for the weak texture scene according to the above technical solution.
Another technical solution of the present invention for solving the above technical problems is as follows:
a SLAM navigation vehicle for weak texture scenes comprising:
a memory for storing a computer program;
and the processor is used for executing the computer program to realize the SLAM method for the weak texture scene according to the technical scheme.
Another technical solution of the present invention for solving the above technical problems is as follows:
a SLAM system for weak texture scenes comprising:
the acquisition unit is used for acquiring an image sequence acquired by the monocular camera;
the matching unit is used for reading a first frame image and a second frame image in the image sequence, and extracting and matching feature points in the first frame image and the second frame image according to a preset deep learning model;
the pose resolving unit is used for resolving the pose of the camera according to the matching result to obtain pose transformation parameters of the second frame image relative to the first frame image;
the map point resolving unit is used for resolving map points of the feature point pairs in the matching result according to the pose transformation parameters and converting the two-dimensional feature point pairs into three-dimensional space points;
and the modeling unit is used for generating a three-dimensional point cloud map at the current moment according to the three-dimensional space points.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
Fig. 1 is a schematic flowchart of an embodiment of a SLAM method for a weak texture scene according to the present invention;
fig. 2 is a schematic flow chart of the SLAM method for a weak texture scene according to another embodiment of the present invention;
fig. 3 is a schematic flow chart of the SLAM method for a weak texture scene according to another embodiment of the present invention;
fig. 4 is a schematic flowchart of an SLAM method for a weak texture scene according to another embodiment of the present invention;
fig. 5 is a structural framework diagram provided by an embodiment of the SLAM system for weak texture scenes according to the present invention.
Detailed Description
The principles and features of this invention are described below in conjunction with the following drawings, which are set forth to illustrate, but are not to be construed to limit the scope of the invention.
As shown in fig. 1, a schematic flow diagram provided by an embodiment of an SLAM method for a weak texture scene according to the present invention is that the SLAM method is applied to the weak texture scene, and when a monocular camera acquires an image sequence, the image is processed and mapped in real time, including:
s1, acquiring an image sequence acquired by the monocular camera;
it should be understood that the present invention is applicable to weak texture scene recognition, and therefore, the image sequence acquired by the monocular camera may be an image sequence including a weak texture scene, and the monocular camera may be placed on a navigation vehicle or a robot, for example, may acquire a weak texture scene such as extraterrestrial planet image or space image. Here, the weak texture scene refers to a scene that does not have or lacks line features, point features, and angle features.
S2, reading a first frame image and a second frame image in the image sequence, and extracting and matching feature points in the first frame image and the second frame image according to a preset deep learning model;
it should be understood that the deep learning model can be a convolutional neural network, which is pre-established and trained, the specific structure of the convolutional neural network can be set according to actual requirements, the training process can also be set according to actual requirements, and finally, the trained convolutional neural network can accurately and efficiently output feature point extraction matching results of the weak texture scene.
For example, the convolutional neural network may include an input layer, the input layer is sequentially connected with a convolutional layers, the number of convolutional layers, the size of convolutional cores, the step size and the filling may be set according to actual requirements, and then connected with a pooling layer, the pooling size, the step size and the filling of a pooling region may be set according to actual requirements, or may be set according to actual requirements to form a fully-connected layer, and finally, an image is output by the output layer.
S3, performing camera pose calculation according to the matching result to obtain pose transformation parameters of the second frame image relative to the first frame image;
it should be understood that the matching result includes a plurality of pairs of matched feature point pairs, the intrinsic matrix can be solved through the camera internal reference matrix and the feature point pairs, and the pose transformation parameters of the second frame image relative to the first frame image can be obtained by decomposing the intrinsic matrix.
S4, carrying out map point calculation on the feature point pairs in the matching result according to the pose transformation parameters, and converting the two-dimensional feature point pairs into three-dimensional space points;
it should be understood that the feature point pairs in the matching result are all two-dimensional points, depth information of the space points of the feature point pairs is obtained by calculating the depth values of the feature point pairs, and the two-dimensional feature point pairs can be converted into three-dimensional space points by adding the two-dimensional coordinate information to the depth information.
Generally, pose transformation parameters are R and t, R is a rotation matrix, and t is a translation matrix.
S5, generating a three-dimensional point cloud map of the current time according to the three-dimensional space points;
and S6, repeating the steps S2 to S5 until all images in the image sequence are read, generating a three-dimensional point cloud map at all times, and completing real-time positioning and map construction of the image sequence.
It should be understood that during the repeated execution of step S2, the remaining pictures in the picture sequence are read, for example, when steps S2 to S5 are executed for the second time after steps S2 to S5 are executed for the first time, the first frame picture and the second frame picture in the remaining pictures, that is, the third frame picture and the fourth frame picture of the entire picture sequence, are read.
The SLAM method provided by the embodiment is suitable for a weak texture scene, the images of the weak texture scene are subjected to feature point extraction and matching through a deep learning model, more appropriate feature detection and extraction rules can be abstracted from samples, then map point calculation is carried out, a three-dimensional point cloud map is constructed, the application of the SLAM method in the weak texture scene is realized, feature points with better stability and robustness can be provided based on deep learning features, the real-time positioning and map construction of the weak texture scene are realized, and the precision and accuracy of map construction can be improved.
Optionally, in some embodiments, as shown in fig. 2, in step S3, the method specifically includes:
s31, randomly extracting n characteristic point pairs from the matching result, wherein n is more than or equal to 8;
s32, acquiring a camera internal reference matrix of the monocular camera, calculating the normalized coordinates of n characteristic point pairs according to the camera internal reference matrix, and resolving according to the normalized coordinates of the n characteristic point pairs to obtain an essential matrix;
s33, repeating the steps S31 to S32 m times to obtain m intrinsic matrixes, and optimizing all the intrinsic matrixes according to a preset optimization algorithm to obtain an optimized target intrinsic matrix, wherein m is more than or equal to 1;
and S34, decomposing the target essential matrix to obtain pose transformation parameters.
Preferably, 8 pairs of feature points may be randomly extracted from the matching result.
It should be understood that the repeated optimization of the essential matrix is performed to make the solution of the essential matrix obtain more interior points in all the matching points, and the more interior points, the more reliable the solution of the essential matrix is proved.
It should be noted that the size of m affects the robustness and speed of the calculation, and the larger m is, the more reliable the calculation result is, but the corresponding calculation time is also long. Generally, it is desirable to calculate that the number of the outer points does not exceed 50% of the total number of the point pairs in the matching point set, and it can be satisfied by setting m to 500, and preferably, the value of m in this scheme may be 1000, so that the requirement of the number of the inner points can be satisfied, and the reliability of the calculation result can be ensured.
Optionally, in some embodiments, as shown in fig. 3, in step S33, the method specifically includes:
s331, repeating the steps S31 to S32 m times to obtain m essential matrixes, and respectively calculating the measurement value of each essential matrix;
s332, comparing the size of the metric value of each essential matrix, and taking the essential matrix with the minimum metric value as the optimized target essential matrix.
Optionally, in some embodiments, in step S331, the metric value of each essence matrix is calculated according to the following formula:
wherein C is a metric value, T is a preset threshold value, eiThe maximum likelihood error of the ith characteristic point pair is 1, 2, …, n.
Wherein e isiCan be calculated according to the following formula:
wherein,
and
representing the position of the matching point, x, solved by the essential matrixiAnd yiCoordinates of the ith pair of matching points are represented.
Optionally, in some embodiments, as shown in fig. 4, in step S4, the method specifically includes:
s41, acquiring any characteristic point pair in the matching result, wherein any characteristic point pair comprises a first characteristic point and a second characteristic point which are matched;
s42, sequentially calculating the depth value of the first characteristic point and the depth value of the second characteristic point according to the pose transformation parameters;
s43, calculating the depth average value of the depth values of the first characteristic points and the second characteristic points;
and S44, obtaining the three-dimensional coordinate of any characteristic point pair according to the two-dimensional coordinate of the first characteristic point, the two-dimensional coordinate of the second characteristic point and the depth average value, and generating a three-dimensional space point according to the three-dimensional coordinate.
It should be noted that the first feature point and the second feature point are homonymous points on the two images, and it is assumed that the point P is a spatial point corresponding to the homonymous point, and theoretically, a connecting line between the two points and the camera main point intersects the point P, but since there is noise disturbance in the observation, the two lines cannot intersect in practice, and therefore, the position of P can be solved by an averaging method.
Optionally, in some embodiments, in step S42, the depth value of the first feature point is calculated according to the following formula:
s1x2×Rx1+x2×t＝0
calculating the depth value of the second feature point according to the following formula:
s2x2＝s1Rx1+t
wherein s is1Is the depth value of the first feature point, s2Is the depth value, x, of the second feature point1Is the normalized coordinate, x, of the first feature point2And R and t are pose transformation parameters.
Optionally, in some embodiments, before step S5, the method further includes:
after each g frame of image is read, performing beam adjustment on the prior h frame of image of the current frame of image, optimizing the pose transformation parameters and three-dimensional space points of the h frame of image, wherein g is more than or equal to 1 and less than or equal to h.
For example, g may be 7 and h may be 15.
Preferably, after the first 15 frames of images are read, the first 15 frames of images may be subjected to global beam method adjustment to optimize the imaging quality, and then the last 15 frames of images may be subjected to local beam method adjustment every 7 frames.
It is understood that some or all of the alternative embodiments described above may be included in some embodiments.
In other embodiments of the present invention, a storage medium is further provided, where the storage medium stores instructions that, when read by a computer, cause the computer to execute the SLAM method for the weak texture scene according to the above technical solution.
In other embodiments of the present invention, there is also provided a SLAM navigation car for a weak texture scene, including:
a memory for storing a computer program;
and the processor is used for executing the computer program to realize the SLAM method for the weak texture scene according to the technical scheme.
As shown in fig. 5, a structural framework diagram is provided for an embodiment of the SLAM system for a weak texture scene according to the present invention, where the SLAM system is applicable to the weak texture scene, and includes:
the acquiring unit 1 is used for acquiring an image sequence acquired by a monocular camera;
the matching unit 2 is used for reading a first frame image and a second frame image in the image sequence, and extracting and matching feature points in the first frame image and the second frame image according to a preset deep learning model;
the pose calculating unit 3 is used for calculating the pose of the camera according to the matching result to obtain pose transformation parameters of the second frame image relative to the first frame image;
the map point resolving unit 4 is used for resolving map points of the feature point pairs in the matching result according to the pose transformation parameters and converting the two-dimensional feature point pairs into three-dimensional space points;
and the modeling unit 5 is used for generating a three-dimensional point cloud map of the current moment according to the three-dimensional space points.
Optionally, in some embodiments, the pose settlement unit specifically includes:
the characteristic point pair selection subunit is used for randomly extracting n characteristic point pairs from the matching result, wherein n is more than or equal to 8;
the intrinsic matrix resolving subunit is used for acquiring a camera internal parameter matrix of the monocular camera, calculating the normalized coordinates of the n characteristic point pairs according to the camera internal parameter matrix, and resolving according to the normalized coordinates of the n characteristic point pairs to obtain an intrinsic matrix;
the essence matrix optimization subunit is used for obtaining m essence matrices obtained by operating the essence matrix solving subunit m times, optimizing all the essence matrices according to a preset optimization algorithm to obtain an optimized target essence matrix, wherein m is more than or equal to 1;
and the essential matrix decomposition subunit is used for decomposing the target essential matrix to obtain pose transformation parameters.
Optionally, in some embodiments, the essence matrix optimization subunit is specifically configured to obtain m essence matrices obtained by operating the essence matrix solution subunit m times, and calculate a metric value of each essence matrix; and comparing the magnitude of the metric value of each essential matrix, and taking the essential matrix with the minimum metric value as the optimized target essential matrix.
Optionally, in some embodiments, the essence matrix optimization subunit is specifically configured to calculate the metric value of each essence matrix according to the following formula:
wherein C is a metric value, T is a preset threshold value, eiThe maximum likelihood error of the ith characteristic point pair is 1, 2, …, n.
Optionally, in some embodiments, the map point calculating unit 4 specifically includes:
a characteristic point pair obtaining subunit, configured to obtain any characteristic point pair in the matching result, where any characteristic point pair includes a first characteristic point and a second characteristic point that have been matched;
the depth value calculation operator unit is used for calculating the depth value of the first characteristic point and the depth value of the second characteristic point in sequence according to the pose transformation parameter;
the depth average value operator unit is used for calculating the depth average value of the depth values of the first characteristic points and the second characteristic points;
and the three-dimensional coordinate calculating subunit is used for obtaining the three-dimensional coordinate of any characteristic point pair according to the two-dimensional coordinate of the first characteristic point, the two-dimensional coordinate of the second characteristic point and the depth average value, and generating a three-dimensional space point according to the three-dimensional coordinate.
Optionally, in some embodiments, the depth value operator unit is specifically configured to calculate the depth value of the first feature point according to the following formula:
s1x2×Rx1+x2×t＝0
calculating the depth value of the second feature point according to the following formula:
s2x2＝s1Rx1+t
wherein s is1Is the depth value of the first feature point, s2Is the depth value, x, of the second feature point1Is the normalized coordinate, x, of the first feature point2And R and t are pose transformation parameters.
Optionally, in some embodiments, the method further comprises:
and the image optimization unit is used for performing beam adjustment on the previous h frame image of the current frame image after the matching unit 2 reads g frames of images, optimizing the pose transformation parameters and three-dimensional space points of the h frame image, and enabling g to be more than or equal to 1 and less than or equal to h.
It is understood that some or all of the alternative embodiments described above may be included in some embodiments.
It should be noted that the above embodiments are product embodiments corresponding to the previous method embodiments, and for the description of each optional implementation in the product embodiments, reference may be made to corresponding descriptions in the above method embodiments, and details are not described here again.
The reader should understand that in the description of this specification, reference to the description of the terms "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described method embodiments are merely illustrative, and for example, the division of steps into only one logical functional division may be implemented in practice in another way, for example, multiple steps may be combined or integrated into another step, or some features may be omitted, or not implemented.
The above method, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
While the invention has been described with reference to specific embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (5)
1. A SLAM method for weak texture scenes, comprising:
s1, acquiring an image sequence acquired by the monocular camera;
s2, reading a first frame image and a second frame image in the image sequence, and extracting and matching feature points in the first frame image and the second frame image according to a preset deep learning model;
s3, camera pose calculation is carried out according to the matching result, and pose transformation parameters of the second frame image relative to the first frame image are obtained;
s4, carrying out map point calculation on the feature point pairs in the matching result according to the pose transformation parameters, and converting the two-dimensional feature point pairs into three-dimensional space points;
s5, generating a three-dimensional point cloud map of the current time according to the three-dimensional space points;
s6, repeating the steps S2 to S5 until all images in the image sequence are read, generating a three-dimensional point cloud map at all moments, and completing real-time positioning and map construction of the image sequence;
in step S3, the method specifically includes:
s31, randomly extracting n characteristic point pairs from the matching result, wherein n is more than or equal to 8;
s32, acquiring a camera internal parameter matrix of the monocular camera, calculating the normalized coordinates of the n feature point pairs according to the camera internal parameter matrix, and resolving according to the normalized coordinates of the n feature point pairs to obtain an essential matrix;
s33, repeating the steps S31 to S32 m times to obtain m intrinsic matrixes, and optimizing all the intrinsic matrixes according to a preset optimization algorithm to obtain an optimized target intrinsic matrix, wherein m is more than or equal to 1;
s34, decomposing the target essential matrix to obtain pose transformation parameters;
in step S33, the method specifically includes:
s331, repeating the steps S31 to S32 m times to obtain m essential matrixes, and respectively calculating the measurement value of each essential matrix according to the following formula:
wherein C is a metric value, T is a preset threshold value, eiFor the maximum likelihood error of the ith characteristic point pair, i ═1，2，...，n；
S332, comparing the magnitude of the metric value of each essential matrix, and taking the essential matrix with the minimum metric value as the optimized target essential matrix;
in step S4, the method specifically includes:
s41, acquiring any characteristic point pair in the matching result, wherein the any characteristic point pair comprises a first characteristic point and a second characteristic point which are matched;
s42, sequentially calculating the depth value of the first characteristic point and the depth value of the second characteristic point according to the pose transformation parameter;
s43, calculating the depth average value of the depth values of the first characteristic points and the second characteristic points;
s44, obtaining a three-dimensional coordinate of any one characteristic point pair according to the two-dimensional coordinate of the first characteristic point, the two-dimensional coordinate of the second characteristic point and the depth average value, and generating a three-dimensional space point according to the three-dimensional coordinate;
in step S42, the depth value of the first feature point is calculated according to the following formula:
s1x2×Rx1+x2×t＝0
calculating the depth value of the second feature point according to the following formula:
s2x2＝s1Rx1+t
wherein s is1Is the depth value of the first feature point, s2Is the depth value, x, of the second feature point1Is the normalized coordinate, x, of the first feature point2And R and t are pose transformation parameters.
2. The SLAM method for weak texture scenes of claim 1, further comprising, before step S5:
after each g frame of image is read, performing beam adjustment on the prior h frame of image of the current frame of image, optimizing the pose transformation parameters and three-dimensional space points of the h frame of image, wherein g is more than or equal to 1 and less than or equal to h.
3. A storage medium having stored therein instructions which, when read by a computer, cause the computer to execute the SLAM method for weak texture scenes according to claim 1 or 2.
4. A SLAM navigation vehicle for weak texture scenes comprising:
a memory for storing a computer program;
a processor for executing the computer program for implementing the SLAM method for weak texture scenes as claimed in claim 1 or 2.
5. A SLAM system for weak texture scenes, comprising:
the acquisition unit is used for acquiring an image sequence acquired by the monocular camera;
the matching unit is used for reading a first frame image and a second frame image in the image sequence, and extracting and matching feature points in the first frame image and the second frame image according to a preset deep learning model;
the pose resolving unit is used for resolving the pose of the camera according to the matching result to obtain pose transformation parameters of the second frame image relative to the first frame image;
the map point resolving unit is used for resolving map points of the feature point pairs in the matching result according to the pose transformation parameters and converting the two-dimensional feature point pairs into three-dimensional space points;
the modeling unit is used for generating a three-dimensional point cloud map of the current moment according to the three-dimensional space points;
wherein, the position and posture resolving unit specifically comprises:
the characteristic point pair selection subunit is used for randomly extracting n characteristic point pairs from the matching result, wherein n is more than or equal to 8;
the intrinsic matrix resolving subunit is used for acquiring a camera internal parameter matrix of the monocular camera, calculating the normalized coordinates of the n characteristic point pairs according to the camera internal parameter matrix, and resolving according to the normalized coordinates of the n characteristic point pairs to obtain an intrinsic matrix;
the essence matrix optimization subunit is used for obtaining m essence matrices obtained by operating the essence matrix solving subunit m times, optimizing all the essence matrices according to a preset optimization algorithm to obtain an optimized target essence matrix, wherein m is more than or equal to 1;
the essential matrix decomposition subunit is used for decomposing the target essential matrix to obtain pose transformation parameters;
the essence matrix optimization subunit is specifically configured to obtain m essence matrices obtained by operating the essence matrix solution subunit m times, and calculate a metric value of each essence matrix; comparing the magnitude of the metric value of each essential matrix, and taking the essential matrix with the minimum metric value as the optimized target essential matrix;
the essence matrix optimization subunit is specifically configured to calculate a metric value of each essence matrix according to the following formula:
wherein C is a metric value, T is a preset threshold value, eiThe maximum likelihood error of the ith characteristic point pair is i ═ 1, 2.
The map point calculating unit specifically comprises:
a characteristic point pair obtaining subunit, configured to obtain any characteristic point pair in the matching result, where any characteristic point pair includes a first characteristic point and a second characteristic point that have been matched;
the depth value calculation operator unit is used for calculating the depth value of the first characteristic point and the depth value of the second characteristic point in sequence according to the pose transformation parameter;
the depth average value operator unit is used for calculating the depth average value of the depth values of the first characteristic points and the second characteristic points;
the three-dimensional coordinate calculating subunit is used for obtaining a three-dimensional coordinate of any characteristic point pair according to the two-dimensional coordinate of the first characteristic point, the two-dimensional coordinate of the second characteristic point and the depth average value, and generating a three-dimensional space point according to the three-dimensional coordinate;
the depth value calculation operator unit is specifically configured to calculate a depth value of the first feature point according to the following formula:
s1x2×Rx1+x2×t＝0
calculating the depth value of the second feature point according to the following formula:
s2x2＝s1Rx1+t
wherein s is1Is the depth value of the first feature point, s2Is the depth value, x, of the second feature point1Is the normalized coordinate, x, of the first feature point2And R and t are pose transformation parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910694313.4A CN110487274B (en) | 2019-07-30 | 2019-07-30 | SLAM method and system for weak texture scene, navigation vehicle and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910694313.4A CN110487274B (en) | 2019-07-30 | 2019-07-30 | SLAM method and system for weak texture scene, navigation vehicle and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110487274A CN110487274A (en) | 2019-11-22 |
| CN110487274B true CN110487274B (en) | 2021-01-29 |
Family
ID=68547806
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910694313.4A Active CN110487274B (en) | 2019-07-30 | 2019-07-30 | SLAM method and system for weak texture scene, navigation vehicle and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110487274B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111402332B (en) * | 2020-03-10 | 2023-08-18 | 兰剑智能科技股份有限公司 | AGV composite map building and navigation positioning method and system based on SLAM |
| CN112258575B (en) * | 2020-10-13 | 2022-12-02 | 浙江大学 | Method for quickly identifying object in synchronous positioning and map construction |
| CN112288816B (en) * | 2020-11-16 | 2024-05-17 | Oppo广东移动通信有限公司 | Pose optimization method, pose optimization device, storage medium and electronic equipment |
| CN112752028B (en) * | 2021-01-06 | 2022-11-11 | 南方科技大学 | Pose determination method, device and equipment of mobile platform and storage medium |
| CN112907671B (en) * | 2021-03-31 | 2022-08-02 | 深圳市慧鲤科技有限公司 | Point cloud data generation method and device, electronic equipment and storage medium |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103335634A (en) * | 2013-06-24 | 2013-10-02 | 南京航空航天大学 | Visual target plate device for calibrating component mounting attitude and calibration method |
| CN104019799A (en) * | 2014-05-23 | 2014-09-03 | 北京信息科技大学 | Relative orientation method by using optimization of local parameter to calculate basis matrix |
| CN105654492A (en) * | 2015-12-30 | 2016-06-08 | 哈尔滨工业大学 | Robust real-time three-dimensional (3D) reconstruction method based on consumer camera |
| CN106504321A (en) * | 2016-11-07 | 2017-03-15 | 达理 | Method using the method for photo or video reconstruction three-dimensional tooth mould and using RGBD image reconstructions three-dimensional tooth mould |
| CN108171249A (en) * | 2018-01-29 | 2018-06-15 | 北京工业大学 | A kind of local description learning method based on RGBD data |
| CN108369743A (en) * | 2015-08-28 | 2018-08-03 | 帝国科技及医学学院 | Use multi-directional camera map structuring space |
| CN108537845A (en) * | 2018-04-27 | 2018-09-14 | 腾讯科技（深圳）有限公司 | Pose determines method, apparatus and storage medium |
| CN108564616A (en) * | 2018-03-15 | 2018-09-21 | 中国科学院自动化研究所 | Method for reconstructing three-dimensional scene in the rooms RGB-D of fast robust |
| CN108665540A (en) * | 2018-03-16 | 2018-10-16 | 浙江工业大学 | Robot localization based on binocular vision feature and IMU information and map structuring system |
| CN109341703A (en) * | 2018-09-18 | 2019-02-15 | 北京航空航天大学 | A kind of complete period uses the vision SLAM algorithm of CNNs feature detection |
| CN109509211A (en) * | 2018-09-28 | 2019-03-22 | 北京大学 | Positioning simultaneously and the feature point extraction and matching process and system built in diagram technology |
| CN109978911A (en) * | 2019-02-22 | 2019-07-05 | 青岛小鸟看看科技有限公司 | A kind of characteristics of image point-tracking method and camera |
| CN110047142A (en) * | 2019-03-19 | 2019-07-23 | 中国科学院深圳先进技术研究院 | No-manned plane three-dimensional map constructing method, device, computer equipment and storage medium |
Family Cites Families (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106489170B (en) * | 2013-08-12 | 2020-03-31 | 苹果公司 | Vision-based inertial navigation |
| WO2017172778A1 (en) * | 2016-03-28 | 2017-10-05 | Sri International | Collaborative navigation and mapping |
| CN108076338B (en) * | 2016-11-14 | 2022-04-08 | 北京三星通信技术研究有限公司 | Image visual processing method, device and equipment |
| WO2018125938A1 (en) * | 2016-12-30 | 2018-07-05 | DeepMap Inc. | Enrichment of point cloud data for high-definition maps for autonomous vehicles |
| CN107747941B (en) * | 2017-09-29 | 2020-05-15 | 歌尔股份有限公司 | Binocular vision positioning method, device and system |
| US10436590B2 (en) * | 2017-11-10 | 2019-10-08 | Ankobot (Shanghai) Smart Technologies Co., Ltd. | Localization system and method, and robot using the same |
| CN108229416B (en) * | 2018-01-17 | 2021-09-10 | 苏州科技大学 | Robot SLAM method based on semantic segmentation technology |
| CN108776989B (en) * | 2018-06-08 | 2022-06-24 | 北京航空航天大学 | Low-texture planar scene reconstruction method based on sparse SLAM framework |
| CN109307508B (en) * | 2018-08-29 | 2022-04-08 | 中国科学院合肥物质科学研究院 | Panoramic inertial navigation SLAM method based on multiple key frames |
| CN109859266B (en) * | 2019-01-28 | 2022-11-25 | 西安理工大学 | Pre-transformation-based visual simultaneous positioning and drawing method under large visual angle change |
-
2019
- 2019-07-30 CN CN201910694313.4A patent/CN110487274B/en active Active
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103335634A (en) * | 2013-06-24 | 2013-10-02 | 南京航空航天大学 | Visual target plate device for calibrating component mounting attitude and calibration method |
| CN104019799A (en) * | 2014-05-23 | 2014-09-03 | 北京信息科技大学 | Relative orientation method by using optimization of local parameter to calculate basis matrix |
| CN108369743A (en) * | 2015-08-28 | 2018-08-03 | 帝国科技及医学学院 | Use multi-directional camera map structuring space |
| CN105654492A (en) * | 2015-12-30 | 2016-06-08 | 哈尔滨工业大学 | Robust real-time three-dimensional (3D) reconstruction method based on consumer camera |
| CN106504321A (en) * | 2016-11-07 | 2017-03-15 | 达理 | Method using the method for photo or video reconstruction three-dimensional tooth mould and using RGBD image reconstructions three-dimensional tooth mould |
| CN108171249A (en) * | 2018-01-29 | 2018-06-15 | 北京工业大学 | A kind of local description learning method based on RGBD data |
| CN108564616A (en) * | 2018-03-15 | 2018-09-21 | 中国科学院自动化研究所 | Method for reconstructing three-dimensional scene in the rooms RGB-D of fast robust |
| CN108665540A (en) * | 2018-03-16 | 2018-10-16 | 浙江工业大学 | Robot localization based on binocular vision feature and IMU information and map structuring system |
| CN108537845A (en) * | 2018-04-27 | 2018-09-14 | 腾讯科技（深圳）有限公司 | Pose determines method, apparatus and storage medium |
| CN109341703A (en) * | 2018-09-18 | 2019-02-15 | 北京航空航天大学 | A kind of complete period uses the vision SLAM algorithm of CNNs feature detection |
| CN109509211A (en) * | 2018-09-28 | 2019-03-22 | 北京大学 | Positioning simultaneously and the feature point extraction and matching process and system built in diagram technology |
| CN109978911A (en) * | 2019-02-22 | 2019-07-05 | 青岛小鸟看看科技有限公司 | A kind of characteristics of image point-tracking method and camera |
| CN110047142A (en) * | 2019-03-19 | 2019-07-23 | 中国科学院深圳先进技术研究院 | No-manned plane three-dimensional map constructing method, device, computer equipment and storage medium |
Non-Patent Citations (5)
| Title |
|---|
| "Improving monocular visual SLAM in dynamic environments: an optical-flow-based approach";Cheng, Jiyu等;《ADVANCED ROBOTICS》;20190618;第33卷(第12期);576-589 * |
| "The comparison between FTF-VO and MF-VO for high accuracy mobile robot localization";Qin Man等;《IEEE International Conference on Intelligent Robots and Systems 》;20181231;48-51 * |
| "基于单目相机的即时定位与稠密地图重建";薛唐立;《万方数据库》;20181218;全文 * |
| "基于改进HOG特征的建筑物识别方法";杨松等;《计算机工程与应用》;20181231;第54卷(第7期);196-200 * |
| "视觉 SLAM 中三维点云配准的 Cayley 方法";张鸿燕等;《中国民航大学学报》;20171031;第35卷(第5期);47-51 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110487274A (en) | 2019-11-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110487274B (en) | SLAM method and system for weak texture scene, navigation vehicle and storage medium | |
| CN107958482B (en) | Three-dimensional scene model construction method and device | |
| CN111780764B (en) | Visual positioning method and device based on visual map | |
| US10225473B2 (en) | Threshold determination in a RANSAC algorithm | |
| TWI569229B (en) | Method for registering data | |
| CN106940704A (en) | A kind of localization method and device based on grating map | |
| CN106705849B (en) | Calibrating Technique For The Light-strip Sensors | |
| CN113850865A (en) | Human body posture positioning method and system based on binocular vision and storage medium | |
| CN111144349A (en) | Indoor visual relocation method and system | |
| EP3185212B1 (en) | Dynamic particle filter parameterization | |
| CN114004883A (en) | Visual perception method and device for curling ball, computer equipment and storage medium | |
| CN113223078A (en) | Matching method and device of mark points, computer equipment and storage medium | |
| CN115512055A (en) | Method and device for performing indoor structure three-dimensional reconstruction based on two-dimensional video and computer equipment | |
| CN117237431A (en) | Training method and device of depth estimation model, electronic equipment and storage medium | |
| JP2007025863A (en) | Photographing system, photographing method, and image processing program | |
| CN111198563B (en) | Terrain identification method and system for dynamic motion of foot type robot | |
| CN113670316A (en) | Path planning method and system based on double radars, storage medium and electronic equipment | |
| CN113112547A (en) | Robot, repositioning method thereof, positioning device and storage medium | |
| JP2006195790A (en) | Lens distortion estimation apparatus, lens distortion estimation method, and lens distortion estimation program | |
| JP3614709B2 (en) | Landscape image index method, apparatus, and recording medium recording landscape image index program | |
| US20220068018A1 (en) | Method for 3d reconstruction of an object | |
| CN115880428A (en) | Animal detection data processing method, device and equipment based on three-dimensional technology | |
| AU2017300877B2 (en) | Method and device for aiding the navigation of a vehicle | |
| CN111586299B (en) | Image processing method and related equipment | |
| CN116109522B (en) | Contour correction method, device, medium and equipment based on graph neural network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |