CN107403451B - Self-adaptive binary characteristic monocular vision odometer method, computer and robot - Google Patents
Self-adaptive binary characteristic monocular vision odometer method, computer and robot Download PDFInfo
- Publication number
- CN107403451B CN107403451B CN201710459019.6A CN201710459019A CN107403451B CN 107403451 B CN107403451 B CN 107403451B CN 201710459019 A CN201710459019 A CN 201710459019A CN 107403451 B CN107403451 B CN 107403451B
- Authority
- CN
- China
- Prior art keywords
- feature
- image
- point
- frame
- search
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 97
- 238000001514 detection method Methods 0.000 claims abstract description 30
- 238000005286 illumination Methods 0.000 claims abstract description 13
- 230000008859 change Effects 0.000 claims abstract description 4
- 238000010276 construction Methods 0.000 claims abstract description 4
- 230000008569 process Effects 0.000 claims description 39
- 230000003044 adaptive effect Effects 0.000 claims description 36
- 230000000007 visual effect Effects 0.000 claims description 31
- 238000012360 testing method Methods 0.000 claims description 10
- 238000001914 filtration Methods 0.000 claims description 7
- 238000006073 displacement reaction Methods 0.000 claims description 6
- 230000009467 reduction Effects 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 4
- 230000005764 inhibitory process Effects 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 4
- 230000004044 response Effects 0.000 claims description 4
- 238000010845 search algorithm Methods 0.000 claims description 3
- 230000002159 abnormal effect Effects 0.000 abstract description 7
- 238000004364 calculation method Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 7
- 238000003066 decision tree Methods 0.000 description 2
- 230000008520 organization Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/77—Determining position or orientation of objects or cameras using statistical methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of computer vision, and discloses a self-adaptive binary feature monocular vision odometer method, a computer and a robot, wherein a feature detection threshold value is self-adaptively adjusted according to illumination change conditions in the environment, and a constant feature number is stably detected under different illumination conditions; meanwhile, the detected features are organized in a quadtree construction mode, and the features are randomly selected by a method combining depth-first search and breadth-first search, so that the final output features are uniformly distributed on the whole image. The final output characteristics of the invention are uniformly distributed on the whole image, thereby reducing the influence of the characteristic abnormal value on the dynamic target on the pose estimation precision.
Description
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to a self-adaptive binary feature monocular vision odometer method, a computer and a robot.
Background
In computer vision, Visual Odometer (VO) refers to estimating the position and orientation of a mobile robot using input video frames of a single or multiple cameras. The method is not only one of core technologies of automatic navigation of the mobile robot, but also has great application prospect in the fields of emerging wearable equipment calculation, 3D reconstruction, virtual reality, augmented reality, automatic driving and the like. The visual odometry methods can be broadly classified into the following three methods in terms of the kind and number of cameras used: the first type is a visual odometry method based on a binocular camera, and has the outstanding advantages that two images of different viewpoints at a certain position can be obtained at each moment, the depth information of the camera at the moment can be directly calculated through the known parallax relation between the two images, and therefore the actual position of a map point in a scene can be known more easily; there are some disadvantages in that the installation configuration of the binocular cameras is inconvenient, and the switching between the large-scale outdoor environment and the small-scale indoor environment is inconvenient because the parallax between the binocular cameras is fixed. The second type is a visual odometry method based on an RGB-D camera, which can obtain an image of a scene and is integrated with other sensors to directly obtain a dense depth map of the scene, and the representative camera of this type is a Kinect depth camera, which has the disadvantages of high hardware cost and short effective depth distance detected by the sensors, so that the applicable environment is limited. The third category is a monocular camera-based visual odometry method, which can perform positioning and mapping using only a single camera. However, the monocular camera can only obtain one image of the scene at each moment, and the depth information of the map points in the scene needs to be obtained through calculation according to the correlation correspondence between the multiple images, so the acquisition of the depth information is a difficult point in the monocular visual odometer. Because the depth information in monocular vision is calculated through the corresponding relation between the images of different viewpoints, a scaling factor exists between the depth value of the calculated map point and the actual depth value, and the uncertainty of the depth enables the monocular vision odometry method to be freely switched between a large-scale outdoor environment and a small-scale indoor environment. Therefore, the monocular vision odometer method has the advantages of low hardware cost, wide applicable environment range and the like. When the environmental condition in the camera motion scene is good, the existing visual odometer method estimates the motion pose track of the camera, so that better real-time performance and robustness can be achieved. However, the existing visual odometry method has some disadvantages as follows: when the ambient illumination in a camera motion scene is too bright or too dark, the existing visual odometry method cannot extract enough features on video frames acquired under the illumination condition, so that the estimation of the pose track of the camera is interrupted, or the accuracy of pose estimation is greatly reduced due to too much abnormal point proportion; when a dynamic moving target exists in a camera moving scene, pose estimation errors can be caused by features detected on the dynamic target; when the frame rate of the video frames collected by the camera and the video frame rate processed by the visual mileage method are not synchronous, a frame skipping phenomenon occurs, and therefore the track estimation of the visual mileage method is easily interrupted.
In summary, the problems of the prior art are as follows: the existing visual odometer method cannot extract enough features, so that the estimation of the pose track of a camera is interrupted or the accuracy of the pose estimation is greatly reduced due to excessive proportion of abnormal points; when a dynamic moving target exists in a camera moving scene, pose estimation errors can be caused by features detected on the dynamic target; when the frame rate of the video frames collected by the camera and the video frame rate processed by the visual mileage method are not synchronous, a frame skipping phenomenon occurs, and the track estimation of the visual mileage method is interrupted.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a self-adaptive binary characteristic monocular vision odometer method, a computer and a robot.
The invention is realized in such a way that a self-adaptive binary characteristic monocular vision odometry method adaptively adjusts a characteristic detection threshold value according to the illumination change condition in the environment, and stably detects constant characteristic number under different illumination conditions; meanwhile, the detected features are organized in a quadtree construction mode, and the features are randomly selected by a method combining depth-first search and breadth-first search, so that the final output features are uniformly distributed on the whole image.
Further, the adaptive binary feature monocular vision odometry method comprises the following steps:
step one, creating two processes which are respectively marked as a Process I and a Process II; the Process I is used for collecting video frames of the camera, and the Process II is used for estimating the pose of the camera by the visual odometer;
step two, opening up a shared memory area for data communication between the Process I and the Process II; recording the shared memory area as a frame sequence pool, and maintaining a first-in first-out video frame queue on the frame sequence pool;
step three, opening a camera in the Process I, and collecting video frames; if the resolution of the video frame is too large, reducing the resolution of the video frame according to the proportional relation, and pressing the video frame into a frame sequence pool;
step four, the Process II takes out the video frame images from the shared memory pool in sequence;
calculating histogram statistics of the gray level image, wherein the width of each histogram bar is 16 pixels; dividing the total number of the images by the number of pixels of each histogram bar to obtain the statistical probability of each histogram bar;
step six, setting different corner detection thresholds according to different image entropy sizes, and detecting different numbers of feature points;
step seven, filtering the image by using a Gaussian average operator, filtering noise, dividing the whole image into small image blocks, and respectively performing corner feature detection on the small image blocks by using an AGAST corner detection algorithm by using a threshold value which is self-adaptively set according to the environment condition;
step eight, obtaining an AGAST corner point with maximum response of the maximum value in the local image block by using a non-maximum value inhibition algorithm;
step nine, if the non-maximum value is inhibited, the number of the characteristic points is larger than the number of the specified characteristic points to be selected, and the characteristic point sets with the specified number are randomly and uniformly selected;
step ten, calculating the direction of the acquired AGAST corner feature;
step eleven, generating a binary feature descriptor through an rBRIEF algorithm to obtain self-adaptive uniform binary features detected on the image;
step twelve, searching matched adaptive uniform binary feature corresponding point sets on the current frame and the previous frame through the Hamming distance and direction distance difference between the adaptive uniform binary features;
and step thirteen, calculating the pose matrix of the camera through the searched matching feature point corresponding sets between the front frame and the rear frame.
Further, the formula for calculating the resolution reduction of the video frame in the third step is as follows:
where (w, h) is the size of the original image and (w ', h') represents the size of the image after resolution reduction.
Further, the Process II in the fourth step sequentially fetches the video frame images from the shared memory pool; the image is a colour image, converted into a grey-scale image, denoted as IgrayFor image IgrayPerforming an adaptive uniform binary feature detection algorithm, detecting IgrayAdaptive uniform binary feature.
Further, I is calculated in the fifth stepgrayEntropy of image of (I)gray) Calculating histogram statistics of the gray level image, wherein the width of each histogram bar is 16 pixels; dividing the total number of the images by the number of pixels of each histogram bar to obtain the statistical probability p of each histogram bariThe image entropy can be calculated according to the following formula:
further, the method for randomly and uniformly selecting the designated number of feature points in the ninth step specifically comprises the following steps:
organizing all feature points on an image into a quadtree, randomly and uniformly selecting the feature points by using a search algorithm combining breadth-first search and depth-first search, and recursively and equally dividing the image into four blocks, namely an upper left block, a lower left block, an upper right block and a lower right block, according to a length-width middle axis until no feature point exists in the image block; utilizing BFS algorithm to randomly search and select nodes on the quadtree, if a certain node is a leaf node and the node has only one characteristic point, selecting the characteristic point; otherwise, continuing to randomly search other nodes; if a node is not a leaf node, selecting a characteristic point in an image area represented by the node, and randomly finding a leaf node containing the characteristic point which is not selected previously; and iterating until a specified number of feature points is selected.
Further, the direction θ of the AGAST corner feature calculated in the step ten is calculated by the following formula:
wherein M isijRepresenting the geometric moment of the image block in which the feature is located, in terms of Mij＝∑x，yxiyjI (x, y) is calculated; x is the number ofcAnd ycRepresenting the centroid of the image block in which the feature is located, in xc＝M10/M00And formula yc＝M01/M00Calculating to obtain; u'ijRepresenting the central moment of the image block in which the feature is located, from the geometric moment M of the image blockijAnd centroid xc、ycAnd (4) calculating.
Further, the twelfth step specifically includes:
firstly, setting the size of a search window, and searching a matching point in a neighborhood window where each feature point is located;
then, search query feature point piAll test feature point sets { p 'in neighborhood window'i}, calculating piAnd { p'iRecording the minimum Hamming distance and the next-smallest Hamming distance between all the feature point descriptors in the graph, and the corresponding test feature points; when the minimum Hamming distance is less than a specified threshold tmAnd the ratio between the smallest hamming distance and the next smallest hamming distance is smaller than a certain ratio rmThen the corresponding test feature point is the feature point matched with the query feature point;
finally, a Delaunay triangular mesh generation algorithm is used for generating triangular meshes on the matching points of the current frame, and triangles with areas obviously larger than most of the meshes in the triangular meshes and vertexes far away from most of the meshes on the triangles are removed; calculating the displacement value of each triangular pixel vertex in the triangular mesh relative to the matching point on the previous frame, and rejecting the matching point when the displacement value is greater than a certain threshold value; and finally finding a matching corresponding point set between the front frame and the back frame.
Another object of the present invention is to provide a computer for applying the adaptive binary feature monocular vision odometry method.
Another object of the present invention is to provide a robot applying the adaptive binary feature monocular vision odometry method.
The invention has the advantages and positive effects that: the feature detection threshold can be adaptively adjusted according to the illumination change condition in the environment, and the constant feature number can be stably detected under different illumination conditions. As can be seen from (b) of fig. 11, in the case that (a) the scene illumination of the motion trajectory of the image camera changes dramatically, the adaptive uniform binary feature proposed by the method can stably detect a specified number of features compared to the ORB feature, and the real-time performance of the two features is kept on the same order of magnitude. In addition, the detected features are organized in a quadtree construction mode, and the features are randomly selected by a method of combining depth-first search and breadth-first search, so that the final output features are uniformly distributed on the whole image, and the influence of the feature abnormal value on the dynamic target on the pose estimation precision is reduced. As can be seen from fig. 10, the fitting accuracy between the camera motion trajectory estimated by the monocular visual odometry method based on the adaptive uniform binary feature proposed by the method and the actual camera motion trajectory is higher than the camera motion trajectory estimated by the monocular visual odometry method based on the ORB feature.
Drawings
Fig. 1 is a flowchart of an adaptive binary feature monocular vision odometry method provided by an embodiment of the present invention.
Fig. 2 is a flowchart of an implementation of the adaptive binary feature monocular vision odometry method according to the embodiment of the present invention.
Fig. 3 is a schematic diagram of a memory organization form of a frame sequence pool according to an embodiment of the present invention.
Fig. 4 is a flowchart of an adaptive uniform binary feature detection algorithm according to an embodiment of the present invention.
Fig. 5 is a schematic diagram of the relationship between image contrast and image entropy provided by the embodiment of the present invention.
Fig. 6 is a schematic diagram of the detection result of the AGAST corner after image deblocking (a) and image deblocking (b) according to an embodiment of the present invention.
Fig. 7 is a diagram illustrating the result of randomly and uniformly selecting feature points according to an embodiment of the present invention.
Fig. 8 is a schematic diagram of a self-adaptive uniform binary feature detection result according to an embodiment of the present invention.
Fig. 9 is a schematic diagram of a feature point set corresponding to a search match between two frames according to an embodiment of the present invention.
FIG. 10 is a schematic diagram illustrating comparison of pose trajectory calculation results of a monocular vision odometry method based on adaptive uniform binary characteristics and ORB characteristics according to an embodiment of the present invention;
wherein (a) represents an actual trajectory of camera motion; (b) the solid line in the middle represents the monocular visual odometry method based on the adaptive uniform binary characteristic, and the dotted line represents the monocular visual odometry method based on the ORB characteristic. From (b), it can be seen that the accuracy of fitting the actual camera motion trajectory in the graph (a) of the camera motion trajectory calculated by the monocular visual odometry method based on the adaptive uniform binary feature is better than that of the monocular visual odometry method based on the ORB feature.
Fig. 11 is a graph comparing the performance of the adaptive uniform binary feature and the ORB feature provided by the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The following detailed description of the principles of the invention is provided in connection with the accompanying drawings.
As shown in fig. 1, the adaptive binary feature monocular vision odometry method provided by the embodiment of the present invention includes the following steps:
s101: creating two processes which are respectively marked as a Process I and a Process II; the Process I is used for collecting video frames of the camera, and the Process II is used for estimating the pose of the camera by the visual odometer;
s102: and opening up a shared memory area for data communication between the Process I and the Process II. Recording the shared memory area as a frame sequence pool, and maintaining a first-in first-out video frame queue on the frame sequence pool;
s103: and opening a camera in the Process I, and acquiring a video frame. If the resolution of the video frame is too large, reducing the resolution of the video frame according to a certain proportional relation, and then pressing the video frame into a frame sequence pool;
s104: the Process II sequentially takes out the video frame images from the shared memory pool;
s105: calculating histogram statistics of the gray level image, wherein the width of each histogram bar is 16 pixels; dividing the total number of the images by the number of pixels of each histogram bar to obtain the statistical probability of each histogram bar;
s106: setting different corner detection thresholds according to different image entropy sizes so as to detect different numbers of feature points;
s107: filtering the image by using a Gaussian average operator, filtering noise, dividing the whole image into small image blocks, and respectively performing angular point feature detection on the small image blocks by using an AGAST angular point detection algorithm by using a threshold value which is self-adaptively set according to the environment condition;
s108: obtaining an AGAST corner point with maximum response of the maximum value in the local image block by using a non-maximum value inhibition algorithm;
s109: if the number of the characteristic points is larger than the number of the designated characteristic points to be selected after the non-maximum value is restrained, randomly and uniformly selecting the characteristic point set with the designated number;
s110: calculating the direction of the obtained AGAST corner feature;
s111: generating a binary feature descriptor through an rBRIEF algorithm to obtain self-adaptive uniform binary features detected on the image;
s112: searching matched adaptive uniform binary feature corresponding point sets on the current frame and the previous frame through the Hamming distance and the direction distance difference between the adaptive uniform binary features;
s113: and calculating a pose matrix of the camera through the searched matching feature point corresponding sets between the previous frame and the next frame.
The application of the principles of the present invention will now be described in further detail with reference to the accompanying drawings.
As shown in fig. 2, the adaptive binary feature monocular vision odometry method provided by the embodiment of the present invention specifically includes the following steps:
step 1: two processes are created, denoted Process I and Process II respectively. The Process I is used for collecting video frames of the camera, and the Process II is used for estimating the pose of the camera by the visual odometer.
Step 2: and opening up a shared memory area for data communication between the Process I and the Process II. Note that the shared memory area is a frame sequence pool, on which a first-in first-out video frame queue is maintained, and the memory organization form of the frame sequence pool is as shown in fig. 3.
The information head of the frame sequence pool reserves some information such as flag bits, the head and tail positions of the queue, the size of the image and the like. b represents whether an image exists in the queue, s represents the position of the head of the queue, e represents the position of the tail of the queue, w represents the width of the image, h represents the height of the image, and c represents the number of channels of the image.
And step 3: and opening a camera in the Process I, and acquiring a video frame. If the resolution of the video frame is too large, the resolution of the video frame is reduced according to a certain proportional relation, and then the video frame is pressed into a frame sequence pool. The calculation formula of the video frame resolution reduction is as follows:
where (w, h) is the size of the original image and (w ', h') represents the size of the image after resolution reduction.
And 4, step 4: and the Process II sequentially fetches the video frame images from the shared memory pool. If the image is a color image, it is converted to a grayscale image, denoted asIgray. First, for image IgrayPerforming an adaptive uniform binary feature detection algorithm, detecting IgrayThe above adaptive uniform binary feature, as in step 5 to step 11, and the flow chart of the adaptive uniform binary feature detection algorithm is as in fig. 4. Then, a matching corresponding feature point set between the current frame and the previous frame is searched, as in step 12. And finally, calculating the pose of the camera through the corresponding point sets matched between the front frame and the rear frame, as in step 13.
And 5: calculation of IgrayEntropy of image of (I)gray). First, histogram statistics of the gray-scale image are calculated, and the width of each histogram bar is 16 pixels. Then, the number of pixels of each histogram bar is divided by the total number of the images to obtain the statistical probability p of each histogram bari. Then, the image entropy can be calculated according to the following formula:
step 6: and setting different corner detection thresholds according to different image entropy sizes so as to detect different numbers of feature points. The image with larger entropy shows that the contrast of the image is higher, the illumination condition of the environment is reflected to be good, and a higher corner detection threshold value is set at the moment. The image with smaller entropy shows that the contrast of the image is smaller, and reflects that the illumination of the environment is too bright or too dark, or the image is too dark or too bright caused by the exposure of the camera, and a lower corner detection threshold is set. Since 16 histogram bins are divided, the possible interval of entropy values is [0, 4 ]. The relationship between image entropy and contrast is shown in fig. 5.
The image entropy of a video frame acquired by a camera is calculated in an actual scene environment, statistics shows that the contrast of the image is very small when the entropy value is [0, 2.7], the corner detection threshold at the moment is set at 10, the contrast of the image is good when the entropy value is (2.7, 3.0], the corner detection threshold at the moment is set at 30, and the corner detection threshold at the moment is set at 40 when the entropy value is (3.0, 4).
And 7: using 3 x 3 gaussian averaging operator for image IgrayAnd filtering to remove noise. And (3) dividing the whole image into small image blocks, and performing corner feature detection on the small image blocks respectively through an AGAST corner detection algorithm by utilizing a threshold value which is self-adaptively set according to the environment condition in the step (6).
When the AGAST corners are detected in the whole image, the detected corners may be gathered in the image, and the corners detected when the image is not blocked are mainly concentrated on the left and right trees, as shown in fig. 6 (a). When the whole image is divided into small image blocks and the angular point detection is respectively carried out on the image blocks, each area can be enabled to have a proper number of angular point distributions, and the influence of the angular points on the dynamic target in the image and other abnormal value points can be counteracted to a certain extent.
The AGAST corner detection algorithm uses a binary decision tree to judge and select a final corner, each small block of the image after being blocked has a binary decision tree, and accordingly, the number of detected characteristic points on the whole image is more. Meanwhile, after the segmentation, a feature is detected on each region, as shown in fig. 6 (b).
The size of the image block is recommended to be neither too small, which would result in too many image blocks for image segmentation and thus in an increased computational effort, nor too large, which would not have the effect of segmenting the image block. The image block size selected by the experimental test of the invention on the image is 50 pixels, because the time of feature calculation is closest to the time of feature calculation on the whole image, and the feature points are basically distributed on the whole image area.
And 8: and obtaining the AGAST corner point with maximum response of the maximum value in the local image block by using a non-maximum value inhibition algorithm.
And step 9: if the number of the feature points is larger than the number of the feature points to be selected in step 8 after the non-maximum value is suppressed, the feature point sets with the designated number are randomly and uniformly selected, and the random selection result is as shown in fig. 7. The method for randomly and uniformly selecting the designated number of feature points comprises the following steps:
all feature points on the image are organized into a quadtree, and feature points are randomly and uniformly selected by using a Search algorithm combining a Breadth First Search (BFS) and a Depth First Search (DFS). Firstly, an image is divided into four blocks of an upper left block, a lower left block, an upper right block and a lower right block according to the length and width middle axis in a recursion mode until no characteristic point exists in the image block; and then randomly searching and selecting nodes on the quadtree by using a BFS algorithm, and if a certain node is a leaf node and the node has only one characteristic point, selecting the characteristic point. Otherwise, the BFS algorithm is continuously used for randomly searching other nodes. If a node is not a leaf node but requires the selection of a feature point within the image region represented by the node, then the DFS algorithm is used to randomly find a leaf node containing previously unselected feature points. The above process is iterated until a specified number of feature points is selected.
Step 10: calculating the direction theta of the AGAST corner feature obtained in the step 9, wherein a calculation formula is as follows:
wherein M isijRepresenting the geometric moment of the image block in which the feature is located, according to the formula Mij＝∑x，yxiyjI (x, y) is calculated. x is the number ofcAnd ycRepresenting the centroid of the image block in which the feature is located, according to formula xc＝M10/M00And formula yc＝M01/M00And (4) calculating.
Step 11: generating a binary feature descriptor by an rBRIEF algorithm to obtain IgrayThe adaptive uniform binary feature detected above, as in fig. 8.
Step 12: and searching matched adaptive uniform binary feature corresponding point sets on the current frame and the previous frame through the Hamming distance and direction distance difference between the adaptive uniform binary features, as shown in FIG. 9.
First, the size of the search window is set, and each feature point will find a matching point in the neighborhood window in which it is located.
Then, search query feature point piAll test feature point sets { p 'in neighborhood window'i}. Calculating piAnd { p'iThe hamming distances between all feature point descriptors in the pixel are recorded, the smallest hamming distance and the next smallest hamming distance, and their corresponding test feature points. When the minimum Hamming distance is less than a specified threshold tmAnd the ratio between the smallest hamming distance and the next smallest hamming distance is smaller than a certain ratio rmThen the corresponding test feature point is the feature point that matches the query feature point. If p is not found by the Hamming distanceiThen the direction of the feature is searched once again according to the same method. Wherein, the threshold value tmAnd the ratio rmAccording to image Entropy Encopy (I)gray) And (4) setting. When the image entropy is small, the threshold tmAnd the ratio rmThe setting is large. When the image entropy is large, the threshold tmAnd the ratio rmThe setting is small.
And finally, generating triangular meshes at the matching points of the current frame by using a Delaunay triangular mesh generation algorithm, and eliminating triangles with areas obviously larger than most meshes in the triangular meshes and vertexes far away from most meshes on the triangles. And then calculating the displacement value of each triangular pixel vertex in the triangular mesh relative to the matching point on the previous frame, and rejecting the matching point when the displacement value is greater than a certain threshold value. And finally finding a matching corresponding point set between the front frame and the back frame.
Step 13: and (4) calculating a pose matrix of the camera through the corresponding set of the matched feature points between the front frame and the rear frame searched in the step (12). The calculation of the camera pose is divided into two cases: when a system is just initialized, calculating the pose of a camera by adopting an eight-point algorithm rejected based on RANSAC abnormal values through matching feature point corresponding sets between front and back frames, and triangulating and reconstructing 3D scene structure points corresponding to the feature points; after the system is initialized, finding out the incidence relation between the reconstructed 3D scene structure point and the feature point on the current frame, calculating the pose of the camera by adopting a PnP algorithm rejected based on an algebraic abnormal value, and triangularly reconstructing the 3D scene structure point corresponding to the feature point.
Comparing the pose calculation results of the monocular vision odometry method based on the self-adaptive uniform binary characteristic with the pose calculation results of the monocular vision odometry method based on the ORB characteristic, as shown in FIG. 10.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (9)
1. A self-adaptive binary feature monocular vision odometry method is characterized in that a feature detection threshold value is self-adaptively adjusted according to illumination change conditions in the environment, and a constant feature number is stably detected under different illumination conditions; meanwhile, the detected features are organized in a quadtree construction mode, and the features are randomly selected by a method combining depth-first search and breadth-first search, so that the final output features are uniformly distributed on the whole image;
the self-adaptive binary feature monocular vision odometry method comprises the following steps of:
step one, creating two processes which are respectively marked as a Process I and a Process II; the Process I is used for collecting video frames of the camera, and the Process II is used for estimating the pose of the camera by the visual odometer;
step two, opening up a shared memory area for data communication between the Process I and the Process II; recording the shared memory area as a frame sequence pool, and maintaining a first-in first-out video frame queue on the frame sequence pool;
step three, opening a camera in the Process I, and collecting video frames; if the resolution of the video frame is too large, reducing the resolution of the video frame according to the proportional relation, and pressing the video frame into a frame sequence pool;
step four, the process II takes out the video frame images from the shared memory pool in sequence;
calculating histogram statistics of the gray level image, wherein the width of each histogram bar is 16 pixels; dividing the total number of the images by the number of pixels of each histogram bar to obtain the statistical probability of each histogram bar;
step six, setting different corner detection thresholds according to different image entropy sizes, and detecting different numbers of feature points;
step seven, filtering the image by using a Gaussian average operator, filtering noise, dividing the whole image into small image blocks, and respectively performing corner feature detection on the small image blocks by using an AGAST corner detection algorithm by using a threshold value which is self-adaptively set according to the environment condition;
step eight, obtaining an AGAST corner point with maximum response of the maximum value in the local image block by using a non-maximum value inhibition algorithm;
step nine, if the non-maximum value is inhibited, the number of the characteristic points is larger than the number of the specified characteristic points to be selected, and the characteristic point sets with the specified number are randomly and uniformly selected;
step ten, calculating the direction of the acquired AGAST corner feature;
step eleven, generating a binary feature descriptor through an rBRIEF algorithm to obtain self-adaptive uniform binary features detected on the image;
step twelve, searching matched adaptive uniform binary feature corresponding point sets on the current frame and the previous frame through the Hamming distance and direction distance difference between the adaptive uniform binary features;
and step thirteen, calculating the pose matrix of the camera through the searched matching feature point corresponding sets between the front frame and the rear frame.
2. The adaptive binary feature monocular visual odometry method of claim 1, wherein the formula for the resolution reduction of the video frame in step three is:
where (w, h) is the size of the original image and (w ', h') represents the size of the image after resolution reduction.
3. The adaptive binary feature monocular visual odometry method of claim 1, wherein the process ii in step four fetches video frame images in order from a shared memory pool; the image is a colour image, converted into a grey-scale image, denoted as IgrayFor image IgrayPerforming an adaptive uniform binary feature detection algorithm, detecting IgrayAdaptive uniform binary feature.
4. The adaptive binary feature monocular visual odometry method of claim 1, where in step five, I is calculatedgrayEntropy of image of (I)gray) Calculating histogram statistics of the gray level image, wherein the width of each histogram bar is 16 pixels; dividing the total number of the images by the number of pixels of each histogram bar to obtain the statistical probability p of each histogram bariThe image entropy can be calculated according to the following formula:
5. the adaptive binary feature monocular visual odometry method of claim 1, wherein the method of randomly and uniformly selecting a specified number of feature points in step nine specifically comprises:
organizing all feature points on an image into a quadtree, randomly and uniformly selecting the feature points by using a search algorithm combining breadth-first search and depth-first search, and recursively and equally dividing the image into four blocks, namely an upper left block, a lower left block, an upper right block and a lower right block, according to a length-width middle axis until no feature point exists in the image block; utilizing BFS algorithm to randomly search and select nodes on the quadtree, if a certain node is a leaf node and the node has only one characteristic point, selecting the characteristic point; otherwise, continuing to randomly search other nodes; if a node is not a leaf node, selecting a characteristic point in an image area represented by the node, and randomly finding a leaf node containing the characteristic point which is not selected previously; and iterating until a specified number of feature points is selected.
6. The adaptive binary-feature monocular visual odometry method of claim 1, wherein the direction θ of the AGAST corner feature computed in step ten is computed as follows:
wherein M isijRepresenting the geometric moment of the image block in which the feature is located, in terms of Mij＝∑x，yxiyjI (x, y) is calculated; x is the number ofcAnd ycRepresenting the centroid of the image block in which the feature is located, in xc＝M10/M00And formula yc＝M01/M00And (4) calculating.
7. The adaptive binary feature monocular visual odometry method of claim 1, wherein the step twelve specifically comprises:
firstly, setting the size of a search window, and searching a matching point in a neighborhood window where each feature point is located;
then, search query feature point piAll test feature point sets { p 'in neighborhood window'iCalculate px and { p'iRecording the minimum Hamming distance and the next-smallest Hamming distance between all the feature point descriptors in the graph, and the corresponding test feature points; when the minimum Hamming distance is less than a specified threshold tmAnd the ratio of the smallest Hamming distance to the next smallest Hamming distance is less than a certain ratioRate rmThen the corresponding test feature point is the feature point matched with the query feature point;
finally, a Delaunay triangular mesh generation algorithm is used for generating triangular meshes on the matching points of the current frame, and triangles with areas obviously larger than most of the meshes in the triangular meshes and vertexes far away from most of the meshes on the triangles are removed; calculating the displacement value of each triangular pixel vertex in the triangular mesh relative to the matching point on the previous frame, and rejecting the matching point when the displacement value is greater than a certain threshold value; and finally finding a matching corresponding point set between the front frame and the back frame.
8. A computer using the adaptive binary feature monocular visual odometry method of any one of claims 1 to 7.
9. A robot applying the adaptive binary feature monocular vision odometry method of any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710459019.6A CN107403451B (en) | 2017-06-16 | 2017-06-16 | Self-adaptive binary characteristic monocular vision odometer method, computer and robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710459019.6A CN107403451B (en) | 2017-06-16 | 2017-06-16 | Self-adaptive binary characteristic monocular vision odometer method, computer and robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107403451A CN107403451A (en) | 2017-11-28 |
| CN107403451B true CN107403451B (en) | 2020-11-10 |
Family
ID=60404767
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710459019.6A Active CN107403451B (en) | 2017-06-16 | 2017-06-16 | Self-adaptive binary characteristic monocular vision odometer method, computer and robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107403451B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108010081B (en) * | 2017-12-01 | 2021-12-17 | 中山大学 | RGB-D visual odometer method based on Census transformation and local graph optimization |
| CN108182438B (en) * | 2018-01-17 | 2020-09-25 | 清华大学 | Deep reinforcement learning-based image binary feature learning method and device |
| CN109146972B (en) * | 2018-08-21 | 2022-04-12 | 南京师范大学镇江创新发展研究院 | Visual navigation method based on rapid feature point extraction and gridding triangle constraint |
| CN109827591B (en) * | 2019-01-23 | 2022-05-17 | 西安微电子技术研究所 | Odometer signal detection circuit and method suitable for multiple input signals |
| CN110033499B (en) * | 2019-04-21 | 2021-01-15 | 南京林业大学 | Landsat OLI image-based water body mapping method |
| CN114066824B (en) * | 2021-10-28 | 2024-05-14 | 华南理工大学 | Binocular vision odometer method with dynamic target detection function |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101937078A (en) * | 2009-06-30 | 2011-01-05 | 深圳市气象局 | Nowcasting method and system of thunder cloud cluster based on boundary recognition and tracer technique |
| CN102982561A (en) * | 2012-12-31 | 2013-03-20 | 哈尔滨工业大学 | Method for detecting binary robust scale invariable feature of color of color image |
| CN103112015A (en) * | 2013-01-29 | 2013-05-22 | 山东电力集团公司电力科学研究院 | Operating object position and posture recognition method applicable to industrial robot |
| CN103258202A (en) * | 2013-05-02 | 2013-08-21 | 电子科技大学 | Method for extracting textural features of robust |
| CN105426841A (en) * | 2015-11-19 | 2016-03-23 | 北京旷视科技有限公司 | Human face detection based monitor camera self-positioning method and apparatus |
| CN105975971A (en) * | 2016-04-22 | 2016-09-28 | 安徽大学 | Low-memory image feature description algorithm |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3038017A1 (en) * | 2014-12-23 | 2016-06-29 | Thomson Licensing | Method of generating descriptor for interest point in image and apparatus implementing the same |
-
2017
- 2017-06-16 CN CN201710459019.6A patent/CN107403451B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101937078A (en) * | 2009-06-30 | 2011-01-05 | 深圳市气象局 | Nowcasting method and system of thunder cloud cluster based on boundary recognition and tracer technique |
| CN102982561A (en) * | 2012-12-31 | 2013-03-20 | 哈尔滨工业大学 | Method for detecting binary robust scale invariable feature of color of color image |
| CN103112015A (en) * | 2013-01-29 | 2013-05-22 | 山东电力集团公司电力科学研究院 | Operating object position and posture recognition method applicable to industrial robot |
| CN103258202A (en) * | 2013-05-02 | 2013-08-21 | 电子科技大学 | Method for extracting textural features of robust |
| CN105426841A (en) * | 2015-11-19 | 2016-03-23 | 北京旷视科技有限公司 | Human face detection based monitor camera self-positioning method and apparatus |
| CN105975971A (en) * | 2016-04-22 | 2016-09-28 | 安徽大学 | Low-memory image feature description algorithm |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107403451A (en) | 2017-11-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107403451B (en) | Self-adaptive binary characteristic monocular vision odometer method, computer and robot | |
| CN110264416B (en) | Sparse point cloud segmentation method and device | |
| CN110378997B (en) | ORB-SLAM 2-based dynamic scene mapping and positioning method | |
| CN112102409B (en) | Target detection method, device, equipment and storage medium | |
| CN109086724B (en) | Accelerated human face detection method and storage medium | |
| CN111724439A (en) | Visual positioning method and device in dynamic scene | |
| Lee et al. | An intelligent depth-based obstacle detection system for visually-impaired aid applications | |
| CN110599522B (en) | Method for detecting and removing dynamic target in video sequence | |
| CN112364865B (en) | Method for detecting small moving target in complex scene | |
| CN101860729A (en) | Target tracking method for omnidirectional vision | |
| CN110570474B (en) | Pose estimation method and system of depth camera | |
| CN112465021B (en) | Pose track estimation method based on image frame interpolation method | |
| CN110610150A (en) | Tracking method, device, computing equipment and medium of target moving object | |
| CN105160649A (en) | Multi-target tracking method and system based on kernel function unsupervised clustering | |
| CN111144213A (en) | Object detection method and related equipment | |
| CN111274964B (en) | Detection method for analyzing water surface pollutants based on visual saliency of unmanned aerial vehicle | |
| CN113223045A (en) | Vision and IMU sensor fusion positioning system based on dynamic object semantic segmentation | |
| CN116879870B (en) | Dynamic obstacle removing method suitable for low-wire-harness 3D laser radar | |
| JP2020149641A (en) | Object tracking device and object tracking method | |
| CN104182968A (en) | Method for segmenting fuzzy moving targets by wide-baseline multi-array optical detection system | |
| CN114022531A (en) | Image processing method, electronic device, and storage medium | |
| CN117036404A (en) | Monocular thermal imaging simultaneous positioning and mapping method and system | |
| CN114511803A (en) | Target occlusion detection method for visual tracking task | |
| Yang et al. | Contrast limited adaptive histogram equalization for an advanced stereo visual slam system | |
| CN116843754A (en) | Visual positioning method and system based on multi-feature fusion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |