CN113920196A - Visual positioning method and device and computer equipment - Google Patents
Visual positioning method and device and computer equipment Download PDFInfo
- Publication number
- CN113920196A CN113920196A CN202111191550.2A CN202111191550A CN113920196A CN 113920196 A CN113920196 A CN 113920196A CN 202111191550 A CN202111191550 A CN 202111191550A CN 113920196 A CN113920196 A CN 113920196A
- Authority
- CN
- China
- Prior art keywords
- pixel
- determining
- coordinates
- feature points
- coordinate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000000007 visual effect Effects 0.000 title claims abstract description 39
- 238000005070 sampling Methods 0.000 claims abstract description 143
- 230000005484 gravity Effects 0.000 claims abstract description 23
- 238000012937 correction Methods 0.000 claims description 37
- 238000010606 normalization Methods 0.000 claims description 26
- 238000004590 computer program Methods 0.000 claims description 12
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000003384 imaging method Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 6
- 238000000605 extraction Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000004807 localization Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013507 mapping Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/97—Determining parameters from multiple pictures
-
- G06T3/047—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
Abstract
The embodiment of the invention discloses a visual positioning method, a visual positioning device and computer equipment, wherein the method comprises the following steps: acquiring a camera image; acquiring feature points corresponding to an image, and determining an initial sampling template corresponding to the feature points, wherein the initial sampling template comprises a first preset number of pixel point pairs; determining a normalized coordinate and a pixel gravity center coordinate corresponding to the feature point; determining an error angle according to the normalized coordinates and the pixel barycentric coordinates; adjusting the initial sampling template according to the error angle to obtain a modified sampling template, and determining descriptors corresponding to the feature points according to the modified sampling template; and matching the feature points corresponding to any other frame of camera image according to the descriptors to determine the pose of the camera. The change of the feature point descriptors caused by the distortion of the camera can be reduced, the matching accuracy of the feature points is improved, more accurate feature point descriptors are obtained, the corresponding feature points are accurately matched through the descriptors, so that the pose of the camera is determined, and the positioning accuracy and robustness of the whole system are improved.
Description
Technical Field
The invention relates to the field of computer software, in particular to a visual positioning method, a visual positioning device and computer equipment.
Background
A Visual synchronous positioning and mapping (VSLAM) system collects images of an environment by using a camera, and estimates the motion condition of the camera in the environment by using the relative change of image pixels or area features at different moments, namely, calculates the relative pose between the images. Compared with a common camera, the fisheye camera can provide a wider field angle, can obtain more information in the environment, and improves the robustness of the VSLAM algorithm.
However, images acquired by a fisheye camera have obvious distortion, and the VSLAM algorithm based on the feature point method generally needs to use feature point descriptors to match feature points. Due to the distortion, the same feature point generates different descriptors at different imaging positions, thereby affecting the accuracy of feature point matching. Thereby having a serious influence on the positioning accuracy and robustness of the whole VSLAM system.
Disclosure of Invention
In view of the defects of the prior art, the application provides a visual positioning method, a visual positioning device and computer equipment, which are used for accurately matching descriptors with image feature points acquired by a monocular fisheye camera and improving the positioning accuracy and robustness of a VSLAM system.
In a first aspect, the present application provides a visual positioning method, the method comprising:
acquiring a camera image;
acquiring feature points corresponding to the image, and determining an initial sampling template corresponding to the feature points, wherein the initial sampling template comprises a first preset number of pixel point pairs;
determining a normalized coordinate and a pixel gravity center coordinate corresponding to the feature point;
determining an error angle according to the normalized coordinates and the pixel barycentric coordinates;
adjusting the initial sampling template according to the error angle to obtain a modified sampling template;
determining descriptors corresponding to the feature points according to the modified sampling template;
and matching the feature points corresponding to any other frame of camera image according to the descriptor to determine the pose of the camera.
Optionally, the determining an initial sampling template corresponding to the feature point includes:
randomly sampling on a pixel plane by taking the characteristic point as a center to obtain the first preset number of pixel point pairs;
and taking the first preset number of pixel point pairs as an initial sampling template of the feature points.
Optionally, the determining the normalized coordinates and the pixel barycentric coordinates corresponding to the feature points includes:
the pixel coordinates of the feature points are back projected onto a normalization plane of a preset camera coordinate system, and the normalization coordinates of the feature points on the normalization plane are obtained;
and selecting a second preset number of coordinate points in a preset area by taking the normalized coordinates as a center, and determining the gravity center coordinates of the pixels according to the coordinate points.
Optionally, the obtaining the normalized coordinates corresponding to the feature points on the normalized plane includes:
determining a coordinate conversion relation between the pixel coordinate of the feature point and the normalized coordinate;
and determining the normalized coordinates according to camera internal parameters, camera distortion coefficients and the coordinate conversion relation.
Optionally, the determining the coordinates of the center of gravity of the pixel according to the coordinate point includes:
projecting the coordinate point to a pixel coordinate system to obtain a pixel coordinate corresponding to the coordinate point;
and determining the gravity center coordinates of the pixels according to the pixel coordinates and the pixel values corresponding to the pixel coordinates.
Optionally, the adjusting the initial sampling template according to the error angle to obtain a modified sampling template includes:
back projecting the initial sampling template to a normalization plane of a preset camera coordinate system, and determining an initial projection sampling template corresponding to the initial sampling template on the normalization plane;
and rotating the initial projection sampling template clockwise around the origin by the angle of the error angle by taking the normalized coordinates as the origin to obtain the correction sampling template.
Optionally, the determining the descriptor corresponding to the feature point according to the modified sampling template includes:
acquiring a first preset number of correction sampling point pairs according to the correction sampling template;
projecting the correction sampling point pairs to a pixel coordinate system, and determining the pixel value of each correction sampling point in the correction sampling point pairs;
and determining descriptors corresponding to the feature points according to the pixel value size relationship of each correction sampling point in the correction sampling point pair.
In a second aspect, the present application provides a visual positioning apparatus, the apparatus comprising:
the image acquisition module is used for acquiring a camera image;
the characteristic point determining module is used for acquiring characteristic points corresponding to the image and determining an initial sampling template corresponding to the characteristic points, wherein the initial sampling template comprises a first preset number of pixel point pairs;
the coordinate determination module is used for determining the normalized coordinates and the pixel gravity center coordinates corresponding to the feature points;
an error angle determination module for determining an error angle according to the normalized coordinates and the pixel barycentric coordinates;
the adjusting module is used for adjusting the initial sampling template according to the error angle to obtain a corrected sampling template;
a descriptor determining module, configured to determine, according to the modified sampling template, a descriptor corresponding to the feature point;
and the pose determining module is used for matching the feature points corresponding to any other frame of camera image according to the descriptors so as to determine the pose of the camera.
In a third aspect, the present application provides a computer device comprising a memory and a processor, the memory storing a computer program, the computer program executing the visual positioning method of any one of the above when the processor runs.
In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program which, when run on a processor, performs a visual positioning method according to any of the above.
The application provides a visual positioning method, a visual positioning device and computer equipment, wherein the method comprises the following steps: acquiring feature points corresponding to a camera image, and determining an initial sampling template corresponding to the feature points; determining a normalized coordinate and a pixel gravity center coordinate corresponding to the feature point; determining an error angle according to the normalized coordinates and the pixel barycentric coordinates; adjusting the initial sampling template according to the error angle to obtain a corrected sampling template, and determining descriptors corresponding to the feature points according to the corrected sampling template; and matching the feature points corresponding to any other frame of camera image according to the descriptor to determine the pose of the camera. By the method, the change of the feature point descriptors caused by the distortion of the fisheye camera can be reduced, more accurate feature point descriptors can be obtained, the accuracy of feature point matching is improved, the pose of the camera is accurately positioned by matching the feature points through the descriptors, accurate visual positioning is realized, and the accuracy and the robustness of the positioning of the whole VSLAM system are improved.
Drawings
In order to more clearly illustrate the technical solution of the present invention, the drawings required to be used in the embodiments will be briefly described below, and it should be understood that the following drawings only illustrate some embodiments of the present invention, and therefore should not be considered as limiting the scope of the present invention. Like components are numbered similarly in the various figures.
FIG. 1 shows a flow diagram of a visual localization method;
FIG. 2 shows a schematic diagram of a back projection model of a visual localization method;
FIG. 3 shows a schematic view of a projection model of a visual localization method;
FIG. 4 shows a block diagram of a visual positioning apparatus;
fig. 5 shows a schematic structural diagram of a computer device.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments.
The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
Hereinafter, the terms "including", "having", and their derivatives, which may be used in various embodiments of the present invention, are only intended to indicate specific features, numbers, steps, operations, elements, components, or combinations of the foregoing, and should not be construed as first excluding the existence of, or adding to, one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing.
Furthermore, the terms "first," "second," "third," and the like are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which various embodiments of the present invention belong. The terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning that is consistent with their contextual meaning in the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein in various embodiments of the present invention.
Example 1
The embodiment of the application provides a visual positioning method, and as shown in fig. 1, the method for generating the descriptor of the image feature point includes:
s101, acquiring a camera image;
specifically, a camera of the camera is used for collecting and shooting a real environment to obtain a camera image, and image processing software of the computer equipment acquires a plurality of frames of images collected by the camera so as to facilitate subsequent calculation processing. In practical application, images acquired by the monocular fisheye camera are distorted, and different descriptors can be generated by the same feature point at different imaging positions, so that the accuracy of feature point matching is influenced.
S102, acquiring feature points corresponding to each frame of image, and determining an initial sampling template corresponding to the feature points, wherein the initial sampling template comprises a first preset number of pixel point pairs;
specifically, feature points in each frame of image are extracted by using a feature point extraction algorithm, optional image feature point extraction algorithms include a FAST feature point extraction algorithm, a harris corner detection method, a SIFT algorithm, a SURF algorithm, and the like, and in order to increase the running speed of the algorithms, the FAST feature point extraction algorithm is preferred in this embodiment. The feature point refers to a point where the image gray value changes drastically or a point where the curvature is large on the edge of the image (i.e., the intersection of two edges). The characteristic points of the image can reflect the essential characteristics of the image, and can identify the target object in the image. The purpose of feature point extraction is to match the same feature points between different images to solve the positional relationship between the different images. And taking the feature point as a center, sampling around the feature point to obtain a first preset number of pixel point pairs, wherein the first preset number of pixel point pairs are initial sampling templates of the feature point, and the pixel point pairs are feature descriptors for describing the feature point.
S103, acquiring a normalized coordinate and a pixel gravity center coordinate corresponding to the feature point;
specifically, the normalized coordinates are coordinates of the feature points on a normalized plane, and the pixel barycentric coordinates are barycentric coordinates of sampling points of the feature points on the normalized plane. And projecting the characteristic points in the pixel coordinate system to the normalized coordinate system to obtain normalized coordinates, and calculating pixel gravity center coordinates corresponding to the characteristic points, so as to facilitate the subsequent calculation of an error angle.
S104, determining an error angle according to the normalized coordinates and the pixel barycentric coordinates;
specifically, the error angle is calculated according to the normalized coordinates of the feature points on the normalized plane and the pixel barycentric coordinates of the feature points on the normalized plane, so that subsequent correction according to the error angle is facilitated.
And S105, adjusting the initial sampling template according to the error angle to obtain a corrected sampling template.
And after the error angle is determined, adjusting the angle of the error angle of the initial sampling template according to the error angle, and obtaining a corrected sampling template after adjustment.
And S106, determining descriptors corresponding to the feature points according to the modified sampling template.
And determining to obtain a descriptor corresponding to the feature point according to the obtained modified sampling template, wherein the feature point descriptor is the accurate descriptor of the feature point in the embodiment. In particular, feature point descriptors are a simplified representation of an image, containing only the most important information about the image, sufficient to identify the image.
And S107, matching the feature points corresponding to any other frame of camera image according to the descriptor to determine the pose of the camera.
Specifically, the feature point descriptors do not change with changes in various changing conditions, the descriptors describe pixel information in the vicinity of the feature points, and feature points having similar appearances in the image have similar descriptors. For example, a camera shoots a house, the feature point in an initial frame picture shot by the camera is an upper right corner point a of the house, the upper right corner point a of the house in a next frame picture after the camera moves and rotates and is shot again is similar to a descriptor that the feature point in the initial frame picture is the upper right corner point a of the house, and the pose shot by the current camera can be positioned through the similar descriptor. Therefore, when the camera moves and rotates and the visual angle changes, the same object in the images of different visual angles is matched by using the feature point descriptors which can still keep unchanged, the pose of the camera is determined, and accurate visual positioning is realized.
And calculating and determining the pose change, namely the moving and rotating conditions of the camera according to the pose of the initial shooting position of the camera and the pose of the current shooting position. And determining the motion track of the camera according to the pose change of the camera, and realizing an accurate image establishing function according to the motion track and the feature points of the camera.
In practical application, due to distortion of the monocular fisheye camera, different descriptors can be generated at different imaging positions by the same feature point, so that the accuracy of feature point matching is influenced. In this embodiment, after the error angle is calculated by the normalized coordinates of the feature points and the coordinates of the center of gravity of the pixel, the angle of the error angle is adjusted for the initial sampling template, and the feature point descriptor corresponding to the corrected sampling template obtained after adjustment is the accurate descriptor of the feature points in this embodiment. The change of the feature point descriptors caused by the distortion of the fisheye camera is solved, the descriptors are accurately matched for the feature points, and more accurate feature point descriptors are obtained, so that the accuracy of feature point matching is improved, the accurate visual positioning and mapping functions are realized, and the accuracy and the robustness of the positioning of the whole VSLAM system are improved.
In a specific embodiment, the obtaining of the initial sampling template corresponding to the feature point includes:
randomly sampling on a pixel plane by taking the characteristic point as a center to obtain the first preset number of pixel point pairs;
and taking the first preset number of pixel point pairs as an initial sampling template of the feature points.
Specifically, a pixel plane where a feature point is located is randomly sampled on an image by using gaussian distribution, a certain number of pixel points are obtained by taking the feature point as a center through random sampling, and then the pixel points are grouped in pairs to obtain a first preset number of pixel point pairs, wherein the value setting of the first preset number includes but is not limited to 128, 256 and 512. In this embodiment, 256 pairs of pixel points are taken as an example for description, and 256 pairs of pixel points are obtained by performing random sampling around a feature point with the feature point as a center, where the 256 pairs of pixel points are initial sampling templates corresponding to the feature point. When other feature points need to be calculated, the other feature points are compared with the pixel points in the initial sampling template, and 256 pairs of pixel points around the other feature points are obtained.
The number of the pixel point pairs corresponds to the number of bits of the descriptor, each pixel point pair corresponds to a value, and the finally generated descriptor is a 256-bit binary string.
In a specific embodiment, the obtaining of the normalized coordinates and the pixel barycentric coordinates corresponding to the feature points includes:
the pixel coordinates of the feature points are back projected onto a normalization plane of a preset camera coordinate system, and the normalization coordinates of the feature points on the normalization plane are obtained;
and selecting a second preset number of coordinate points in a preset rectangular area by taking the normalized coordinates as a center, and acquiring the gravity center coordinates of the pixels according to the coordinate points.
Specifically, as shown in fig. 2, the pre-established camera coordinate system is a camera coordinate system established with the camera position corresponding to the first frame image as an origin, the optical center of the camera is used as the origin, the Z axis coincides with the optical axis of the camera and is perpendicular to the image imaging plane, and the shooting direction is the positive direction of the Z axis; the X, Y axis is parallel to the X, Y axis of the image coordinate axes.
The pixel coordinates of the image are established as follows: the image processing method comprises the steps that a plurality of pixel points are arranged on a pixel plane of an image, the upper left side of each pixel point is used as an original point, pixel coordinates are used for representing the positions of the pixel points on the image, a u axis is parallel to the positive direction of an image plane and is horizontally arranged to the right, a v axis is perpendicular to the u axis and is arranged downwards, and a unit under a pixel coordinate system is a pixel. And presetting a normalization plane of the camera coordinate system as a coordinate plane parallel to the XOY plane, wherein the Z-axis coordinate value is 1.
The plane of the pixel coordinate system and the imaging plane are on the same plane, and the respective origin and measurement unit are different.
In a specific embodiment, the obtaining the normalized coordinates corresponding to the feature points on the normalized plane includes:
determining a coordinate conversion relation between the pixel coordinate and the normalized coordinate;
and determining the normalized coordinates according to camera internal parameters, camera distortion coefficients and the coordinate conversion relation.
For example, if the coordinates of the feature point P are (u, v), the feature point P (u, v) is reflected and projected onto the normalized plane S1, and the coordinates P are (x, y, 1.0), which is calculated as follows:
formula (1):
wherein, PdThe relational expression is expressed according to the coordinates of the pixel p (u, v) after the pinhole camera images the model under the imaging plane. PdThe coordinate under the imaging plane is (x)d，yd)。
u and v correspond to the u-th column, v-th row element in pixel coordinates. k is a radical of1、k2、k3、k4Is a camera distortion coefficient, cx、cy、fx、fyFor the internal reference of the camera, the internal reference of the camera is generally considered to be fixed after being shipped from a factory and cannot be changed in the using process.
Formula (2):
r is the projection length, rdIs the projected length after distortion.
Formula (3):
and solving an expression of r by adopting a Newton iteration method, and substituting the expression into the formula to obtain a normalized coordinate.
Specifically, a second preset number of coordinate points are selected in the preset area with the normalized coordinate as the center, for example, the preset area may be a rectangular area with an arbitrary size with the feature point as the center, and the second preset number may be the number of m × n coordinate points selected in the arbitrary rectangular area, in this embodiment, 20 × 20 coordinate points of the second preset number are equally spaced in a rectangle with the preset area of 0.1 × 0.1 with P ═ (x, y, 1.0) as the center on the normalized plane point.
In a specific embodiment, the obtaining the coordinates of the center of gravity of the pixel according to the coordinate point includes:
projecting the coordinate point to a pixel coordinate system to obtain a pixel coordinate corresponding to the coordinate point;
and determining the gravity center coordinates of the pixels according to the pixel coordinates and the pixel values corresponding to the pixel coordinates.
As shown in fig. 3, an arbitrary point P is setiProjecting to a camera pixel plane to obtain a pixel value, wherein the projection method comprises the following steps:
formula (4):
formula (5):
wherein the formula (5) corresponds to that in FIG. 3
And (4) an angle. The formula (4) and the formula (5) are obtained by combining the formula (2):
formula (6):
and calculating the corresponding pixel values of the 20 multiplied by 20 coordinate points after the coordinate points are projected to the camera imaging plane according to the calculation process. Obtain the coordinates of the center of gravity of
IiIs the pixel value corresponding to the coordinate point.
In a specific embodiment, the adjusting the initial sampling template according to the error angle to obtain a modified sampling template includes:
back projecting the initial sampling template to a normalization plane of a preset camera coordinate system, and determining an initial projection sampling template corresponding to the initial sampling template on the normalization plane;
and rotating the initial projection sampling template clockwise around the origin by the angle of the error angle by taking the normalized coordinates as the origin to obtain the correction sampling template.
Specifically, the error angle is calculated by using (x) the center of gravity Cc，yc1.0) and P, the formula is as follows:
formula (7): theta-atan 2 (y)c-y,xc-y)
As shown in fig. 2, D1 is the region corresponding to the normalized plane S1 before projection, and S0 is the region corresponding to the projection onto the pixel plane. And in the projection process, the characteristic points P and the initial sampling template D0 area of the characteristic points are back-projected to a normalized plane of a preset camera coordinate system together to obtain an initial projection sampling template D1. And rotating the initial projection sampling template D1 clockwise around the origin by the angle theta of the error angle by taking the feature point normalized coordinates as the origin, wherein the expression of the angle theta of the error angle is formula (7), so that a modified sampling template can be obtained, and the descriptor can be calculated according to the modified sampling template.
In a specific embodiment, the obtaining the descriptor corresponding to the feature point according to the modified sampling template includes:
acquiring a first preset number of correction sampling point pairs according to the correction sampling template;
projecting the correction sampling point pairs to the pixel coordinate system to obtain the pixel values of the correction sampling points in the correction sampling point pairs;
and determining descriptors corresponding to the feature points according to the pixel value size relationship of each correction sampling point in the correction sampling point pair.
Specifically, if there are 256 pixel point pairs in the initial projection sampling template D1, for any one pair of points (p)i，pj) The calculation method is as follows: p is to bei，pjProjected onto a pixel plane to obtain a corresponding pixel value, if piCorresponding pixel value greater than pjIf not, the corresponding bit data is set to be 1, otherwise, the bit data is set to be 0, each pixel point pair obtains a one-bit binary digit, 256 pixel point pairs obtain corresponding 256-bit binary data, and the 256-bit binary data is the descriptor of the image feature point.
Example 2
The embodiment of the present application provides a visual positioning apparatus 400, as shown in fig. 4, the visual positioning apparatus 400 includes:
an image acquisition module 401, configured to acquire a camera image;
a feature point determining module 402, configured to obtain feature points corresponding to each frame of image, and determine an initial sampling template corresponding to the feature points, where the initial sampling template includes a first preset number of pixel point pairs;
a coordinate determining module 403, configured to determine a normalized coordinate and a pixel barycentric coordinate corresponding to the feature point;
an error angle determination module 404, configured to determine an error angle according to the normalized coordinates and the pixel barycentric coordinates;
an adjusting module 405, configured to adjust the initial sampling template according to the error angle to obtain a modified sampling template;
a descriptor determining module 406, configured to determine, according to the modified sampling template, a descriptor corresponding to the feature point;
and the pose determining module 407 is used for matching the feature points corresponding to any other frame of camera image according to the descriptor to determine the pose of the camera.
Optionally, the feature point obtaining module 402 is further configured to obtain the first preset number of pixel point pairs by taking the feature point as a center and randomly sampling on a pixel plane; and taking the first preset number of pixel point pairs as an initial sampling template of the feature points.
A coordinate obtaining module 403, configured to back-project the pixel coordinates of the feature points onto a normalization plane of a preset camera coordinate system, so as to obtain the normalization coordinates corresponding to the feature points on the normalization plane; and selecting a second preset number of coordinate points in a preset area by taking the normalized coordinates as a center, and determining the gravity center coordinates of the pixels according to the coordinate points.
The adjusting module 405 is further configured to back-project the initial sampling template to a normalization plane of a preset camera coordinate system, and determine an initial projection sampling template corresponding to the initial sampling template on the normalization plane; and rotating the initial projection sampling template clockwise around the origin by the angle of the error angle by taking the normalized coordinates as the origin to obtain the correction sampling template.
The descriptor determining module 406 is further configured to obtain a first preset number of modified sampling point pairs according to the modified sampling template; projecting the correction sampling point pairs to the pixel coordinate system, and determining the pixel value of each correction sampling point in the correction sampling point pairs; and determining descriptors corresponding to the feature points according to the pixel value size relationship of each correction sampling point in the correction sampling point pair.
For specific implementation steps, reference may be made to the description related to the visual positioning method provided in embodiment 1, and details are not repeated here to avoid repetition.
The visual positioning device 400 of the embodiment can reduce the change of the feature point descriptors caused by the distortion of the fisheye camera, obtain more accurate feature point descriptors, improve the accuracy of feature point matching, accurately position the pose of the camera by matching the feature points through the descriptors, realize accurate visual positioning, and further improve the accuracy and robustness of the positioning of the whole VSLAM system.
Example 3
The embodiment of the present application provides a computer device, including a memory and a processor, where the memory stores a computer program, and the computer program executes the visual positioning method of any one of the above embodiments 1 when the processor runs.
Furthermore, as shown in fig. 5, the present disclosure provides a computer device 500, which includes a memory 501 and a processor 502, where the memory 501 stores a computer program, and the computer program, when running on the processor 502, executes the visual positioning method described in method embodiment 1 above.
A processor 502 for acquiring camera images;
acquiring feature points corresponding to the image, and determining an initial sampling template corresponding to the feature points, wherein the initial sampling template comprises a first preset number of pixel point pairs;
determining a normalized coordinate and a pixel gravity center coordinate corresponding to the feature point;
determining an error angle according to the normalized coordinates and the pixel barycentric coordinates;
adjusting the initial sampling template according to the error angle to obtain a corrected sampling template, and determining descriptors corresponding to the feature points according to the corrected sampling template;
and matching the feature points corresponding to any other frame of camera image according to the descriptor to determine the pose of the camera.
The processor 502 is further configured to randomly sample on a pixel plane by taking the feature point as a center to obtain the first preset number of pixel point pairs;
and taking the first preset number of pixel point pairs as an initial sampling template of the feature points.
The processor 502 is further configured to back-project the feature points onto a normalization plane of a preset camera coordinate system, so as to obtain the normalization coordinates corresponding to the feature points on the normalization plane;
and selecting a second preset number of coordinate points in a preset area by taking the normalized coordinates as a center, and determining the gravity center coordinates of the pixels according to the coordinate points.
The processor 502 is further configured to determine a coordinate transformation relationship between the pixel coordinate and the normalized coordinate;
and determining the normalized coordinates according to camera internal parameters, camera distortion coefficients and the coordinate conversion relation.
The processor 502 is further configured to project the coordinate point to a pixel coordinate system, so as to obtain a pixel coordinate corresponding to the coordinate point;
and determining the gravity center coordinates of the pixels according to the pixel coordinates and the pixel values corresponding to the pixel coordinates.
The processor 502 is further configured to back-project the initial sampling template to a normalization plane of a preset camera coordinate system, and determine an initial projection sampling template corresponding to the initial sampling template on the normalization plane;
and rotating the initial projection sampling template clockwise around the origin by the angle of the error angle by taking the normalized coordinates as the origin to obtain the correction sampling template.
The processor 502 is further configured to obtain a first preset number of modified sampling point pairs according to a modified sampling template;
projecting the correction sampling point pairs to the pixel coordinate system, and determining the pixel value of each correction sampling point in the correction sampling point pairs;
and determining descriptors corresponding to the feature points according to the pixel value size relationship of each correction sampling point in the correction sampling point pair.
The specific implementation steps may refer to the description related to the image feature point descriptor generation method provided in embodiment 1, and are not described herein again to avoid repetition.
The computer equipment provided by the embodiment can accurately acquire the descriptors of the image feature points of the monocular fisheye camera, reduce the change of the descriptor of the feature points caused by the distortion of the fisheye camera, acquire more accurate descriptor of the feature points, improve the accuracy of feature point matching, accurately position the pose of the camera by matching the feature points through the descriptors, realize accurate visual positioning, and further improve the accuracy and robustness of the positioning of the whole VSLAM system.
Example 4
An embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program, when running on a processor, executes the visual positioning method according to any one of the above embodiments 1.
For specific implementation steps, reference may be made to the description related to the visual positioning method provided in embodiment 1, and details are not repeated here to avoid repetition.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative and, for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In addition, each functional module or unit in each embodiment of the present invention may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
The functions, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention or a part of the technical solution that contributes to the prior art in essence can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a smart phone, a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and all the changes or substitutions should be covered within the scope of the present invention.
Claims (10)
1. A visual positioning method, characterized in that the method comprises:
acquiring a camera image;
acquiring feature points corresponding to the image, and determining an initial sampling template corresponding to the feature points, wherein the initial sampling template comprises a first preset number of pixel point pairs;
determining a normalized coordinate and a pixel gravity center coordinate corresponding to the feature point;
determining an error angle according to the normalized coordinates and the pixel barycentric coordinates;
adjusting the initial sampling template according to the error angle to obtain a modified sampling template;
determining descriptors corresponding to the feature points according to the modified sampling template;
and matching the feature points corresponding to any other frame of camera image according to the descriptor to determine the pose of the camera.
2. The method of claim 1, wherein the determining the initial sampling template corresponding to the feature point comprises:
randomly sampling on a pixel plane by taking the characteristic point as a center to obtain the first preset number of pixel point pairs;
and taking the first preset number of pixel point pairs as an initial sampling template of the feature points.
3. The method of claim 1, wherein determining the normalized coordinates and pixel centroid coordinates corresponding to the feature points comprises:
back projecting the feature points to a normalization plane of a preset camera coordinate system to obtain the normalization coordinates corresponding to the feature points on the normalization plane;
and selecting a second preset number of coordinate points in a preset area by taking the normalized coordinates as a center, and determining the gravity center coordinates of the pixels according to the coordinate points.
4. The method of claim 3, wherein the obtaining the normalized coordinates of the feature points corresponding to the normalized plane comprises:
determining a coordinate conversion relation between the pixel coordinate of the feature point and the normalized coordinate;
and determining the normalized coordinates according to camera internal parameters, camera distortion coefficients and the coordinate conversion relation.
5. The method of claim 3, wherein said determining the pixel barycentric coordinates from the coordinate points comprises:
projecting the coordinate point to a pixel coordinate system to obtain a pixel coordinate corresponding to the coordinate point;
and determining the gravity center coordinates of the pixels according to the pixel coordinates and the pixel values corresponding to the pixel coordinates.
6. The method of claim 1, wherein said adjusting the initial sampling template according to the error angle to obtain a modified sampling template comprises:
back projecting the initial sampling template to a normalization plane of a preset camera coordinate system, and determining an initial projection sampling template corresponding to the initial sampling template on the normalization plane;
and rotating the initial projection sampling template clockwise around the origin by the angle of the error angle by taking the normalized coordinates as the origin to obtain the correction sampling template.
7. The method of claim 1, wherein said determining descriptors corresponding to said feature points according to said modified sampling template comprises:
acquiring a first preset number of correction sampling point pairs according to the correction sampling template;
projecting the correction sampling point pairs to a pixel coordinate system, and determining the pixel value of each correction sampling point in the correction sampling point pairs;
and determining descriptors corresponding to the feature points according to the pixel value size relationship of each correction sampling point in the correction sampling point pair.
8. A visual positioning device, the device comprising:
the image acquisition module is used for acquiring a camera image;
the characteristic point determining module is used for acquiring characteristic points corresponding to the image and determining an initial sampling template corresponding to the characteristic points, wherein the initial sampling template comprises a first preset number of pixel point pairs;
the coordinate determination module is used for determining the normalized coordinates and the pixel gravity center coordinates corresponding to the feature points;
an error angle determination module for determining an error angle according to the normalized coordinates and the pixel barycentric coordinates;
the adjusting module is used for adjusting the initial sampling template according to the error angle to obtain a corrected sampling template;
a descriptor determining module, configured to determine, according to the modified sampling template, a descriptor corresponding to the feature point;
and the pose determining module is used for matching the feature points corresponding to any other frame of camera image according to the descriptors so as to determine the pose of the camera.
9. A computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, performs the visual positioning method of any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that it stores a computer program which, when run on a processor, performs the visual positioning method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111191550.2A CN113920196A (en) | 2021-10-13 | 2021-10-13 | Visual positioning method and device and computer equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111191550.2A CN113920196A (en) | 2021-10-13 | 2021-10-13 | Visual positioning method and device and computer equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113920196A true CN113920196A (en) | 2022-01-11 |
Family
ID=79240156
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111191550.2A Pending CN113920196A (en) | 2021-10-13 | 2021-10-13 | Visual positioning method and device and computer equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113920196A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115311359A (en) * | 2022-07-18 | 2022-11-08 | 北京城市网邻信息技术有限公司 | Camera pose correction method and device, electronic equipment and storage medium |
-
2021
- 2021-10-13 CN CN202111191550.2A patent/CN113920196A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115311359A (en) * | 2022-07-18 | 2022-11-08 | 北京城市网邻信息技术有限公司 | Camera pose correction method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111145238B (en) | Three-dimensional reconstruction method and device for monocular endoscopic image and terminal equipment | |
| CN107705333B (en) | Space positioning method and device based on binocular camera | |
| CN111354042B (en) | Feature extraction method and device of robot visual image, robot and medium | |
| CN108765328B (en) | High-precision multi-feature plane template and distortion optimization and calibration method thereof | |
| WO2021174939A1 (en) | Facial image acquisition method and system | |
| CN106683070B (en) | Height measuring method and device based on depth camera | |
| US20150093042A1 (en) | Parameter calibration method and apparatus | |
| JP4951498B2 (en) | Face image recognition device, face image recognition method, face image recognition program, and recording medium recording the program | |
| CN106530358A (en) | Method for calibrating PTZ camera by using only two scene images | |
| CN109919971B (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| TW201118791A (en) | System and method for obtaining camera parameters from a plurality of images, and computer program products thereof | |
| JP5833507B2 (en) | Image processing device | |
| US11922658B2 (en) | Pose tracking method, pose tracking device and electronic device | |
| CN108345821B (en) | Face tracking method and device | |
| CN112085790A (en) | Point-line combined multi-camera visual SLAM method, equipment and storage medium | |
| CN111798373A (en) | Rapid unmanned aerial vehicle image stitching method based on local plane hypothesis and six-degree-of-freedom pose optimization | |
| JP2017123087A (en) | Program, device and method for calculating normal vector of planar object reflected in continuous photographic images | |
| CN115564842A (en) | Parameter calibration method, device, equipment and storage medium for binocular fisheye camera | |
| CN110807833A (en) | Mesh topology obtaining method and device, electronic equipment and storage medium | |
| CN111627073B (en) | Calibration method, calibration device and storage medium based on man-machine interaction | |
| CN111950554A (en) | Identification card identification method, device, equipment and storage medium | |
| CN113870350A (en) | Image sample annotation method and device, terminal equipment and image annotation system | |
| CN113920196A (en) | Visual positioning method and device and computer equipment | |
| JP6086491B2 (en) | Image processing apparatus and database construction apparatus thereof | |
| KR100944293B1 (en) | Mechanism for reconstructing full 3D model using single-axis turntable images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |