CN117011704A - Feature extraction method based on dotted line feature fusion and self-adaptive threshold - Google Patents

Abstract

The invention discloses a feature extraction method based on point-line feature fusion and adaptive threshold, which belongs to the field of automatic driving. In order to solve the positioning accuracy difference of autonomous navigation of unmanned vehicles based on visual SLAM in low texture and short-term fast motion scenes. , weak robustness and other issues, the present invention provides a feature point extraction method based on point-line feature fusion and adaptive threshold. First, the image is input, and the image is set to a fixed resolution. Secondly, the image is processed in parallel through two threads. Line feature extraction and ORB point uniformity feature extraction with adaptive threshold are performed on the image. Finally, feature fusion is performed to adapt to low texture, Feature extraction in low light scenes. The present invention is of great significance for feature extraction in low-texture complex scenes and accurate positioning and mapping in fast-moving scenes.

Description

A feature extraction method based on point-line feature fusion and adaptive thresholding

Technical field

The invention belongs to the field of automatic driving, and in particular relates to a feature extraction method based on point-line feature fusion and adaptive threshold.

Background technique

Visual SLAM means that mobile devices use image information to not only calculate poses, but also construct environment maps at the same time. It is widely used in many fields, such as three-dimensional reconstruction, robots, autonomous driving, etc. However, because the actual application scenarios of unmanned vehicles are changeable and complex, especially in environments such as rapid movement in a short period of time and low texture, motion blur and too little overlapping area between two frames will occur, resulting in poor positioning accuracy. Low, poor stability and other issues. This also makes how to solve the problems of poor positioning accuracy and weak robustness in autonomous navigation of unmanned vehicles based on visual SLAM in low-texture and short-term fast-moving scenes, which has become one of the research hotspots in recent years.

In recent years, with the continuous efforts, innovation and research of scientific researchers, visual SLAM has been gradually improved regardless of theory or technology, but this is all based on ideal conditions, such as clear and rich textures. However, few actual environments reach the ideal environmental state. Currently, many widely used visual SLAM technologies are based on point features. The positioning accuracy of such systems in low-texture scenes is relatively poor, mainly because they cannot extract enough point features to meet the needs of pose estimation. Therefore, how to improve the positioning accuracy of visual SLAM in low-texture scenes has become one of the focuses of scholars.

Contents of the invention

In view of this, the purpose of the present invention is to design a feature extraction method based on point-line feature fusion and adaptive thresholding to solve the above background problem of being unable to extract enough feature points to meet the needs of pose estimation in low-texture scenes. Problems in technology.

The present invention is a point-line feature point fusion extraction method, which is implemented through the following scheme:

Step 1: Image input, prepare for feature extraction: collect surrounding images through binocular cameras in real time, collect images through a fixed number of frames, and set the image resolution to 752*480.

Step 2: Use the feature point extraction algorithm in ORBSLAM3 to extract ORB feature points based on the adaptive threshold;

Further, in step 2, using the feature point extraction algorithm in ORBSLAM3 to extract ORB feature points based on the adaptive threshold also includes the following steps:

Step a: Divide the input image into four areas, and define different FAST corner detection thresholds based on the different degrees of confusion of the sub-image grayscale values corresponding to each area;

Step b: Define image I ₁ as the feature point image to be extracted. The height of I ₁ is h _i and the width is w _i . In order to improve the uniformity of ORB features, I ₁ is divided into a height of h _i /2 and a width of Four sub-images of w _i /2;

Step c: Use the coefficient of variation to describe the degree of dispersion of the gray value of all pixels in each sub-image. The coefficient of variation s is the ratio of the standard deviation of a set of data to the average value. The calculation formula is:

where g _i represents the gray value of a certain pixel in the sub-image, represents the average gray value of the sub-image, t represents the number of pixels, the larger the variation coefficient, the higher the degree of gray confusion. Define g _s = s*30, and use the following formula to define the initial corner detection threshold g _th :

Step d: After setting the adaptive detection threshold according to the variation coefficient s, in order to ensure scale invariance, construct an eight-layer image pyramid for the sub-image, then extract feature points from each layer of the pyramid, and define the ORB feature points that need to be extracted from I ₁ The total number is x _t , and the scaling factor γ _s of the image pyramid, then the number of feature points x _ti that need to be extracted in the i-th layer of each sub-image can be expressed as:

where n is the number of pyramid layers, inv(γ _s ) represents the reciprocal of the scaling factor;

Step e: After calculating the number of feature points that need to be extracted for each layer of the pyramid, use a square grid with a side length of 30 pixels to divide the pyramid into regions, and perform FAST corner point extraction in each grid. If the number of corner points extracted in the grid is is 0, then reduce the detection threshold of the area to g _th /2, and extract again. If the corner points are still not extracted, abandon the grid to ensure the number of feature points, and repeat the above operation until all the corner points in the grid are Corner point extraction is completed;

Step f: After extracting all corner points, use the quadtree to manage the corner points. Define the root node of the quadtree as the entire sub-image. Divide the sub-image into 4 areas as child nodes of the root. If the corner point in the child node If the number is 2 (and above), continue to divide the quadtree into the sub-node. If the number of corner points is 1, keep the sub-node and not continue to divide it. If the number of corner points is 0, delete the sub-node until the pyramid After the number of feature points extracted from each layer of images reaches the set threshold, the corner points with the highest Harris corresponding values in each sub-node are retained. Finally, the FAST corner points extracted from the four sub-images are merged and the corresponding descriptions are calculated. , the adaptive threshold homogenization can be completed to extract ORB feature points.

Step 3: Line feature extraction based on EDLines algorithm;

Further, line feature extraction based on the EDLines algorithm in step 3 includes the following steps:

Step Ⅰ: In order to improve the recognition effect, first add distortion correction to the input image. The distortion setting parameter value is the one that comes with the camera at the factory;

Step Ⅱ: Image smoothing processing, using Gaussian filter to remove noise, and filtering to suppress noise in the image. Gaussian kernel is 5×5, σ=1;

Step III: Use the Sobel gradient operator to calculate the gradient and direction in the image. The calculation formula is:

Where I(x,y) is the pixel value of the image at (x,y), g(x,y) is the size of the gradient, and angle(x,y) is the angle of the horizontal line;

Step IV: Traverse each pixel, select the pixel whose gradient value in the gradient direction is greater than or equal to the gradient value of the adjacent pixel, and define it as an anchor point;

Step V: Select an anchor point as the starting point, compare the amplitude values of adjacent pixels, select the pixel with the largest gradient value as the next anchor point, and finally connect adjacent anchor points to form an edge pixel chain;

Step VI: Divide one or more line segments from the edge pixel chain, traverse the pixels in order, and use the least squares method to fit the pixels. The expression is:

In the formula, x _{i and} y _i are pixel coordinates. When the result exceeds the threshold, for example, 1 pixel has no error, it is truncated until all pixels are processed. The maximum root mean square fitting error and the shortest line segment length are involved in the line segment fitting process. The fitting error is calculated using equation (5), and the calculation formula for the shortest line segment length is:

where N is the width of the input image.

Step 4: Double-thread extraction of features and fusion of point and line features.

Compared with the existing technology, the present invention adopts the above technical solution and has the following technical effects:

1) The present invention is based on the feature extraction method of point-line feature fusion and adaptive threshold. In low-texture scenes, there are abundant line features. This is because compared to point features, line features are basically not affected by light, occlusion, viewing angle changes, etc. Affected by factors, it can be applied to a variety of scenarios. Therefore, in this scenario, introducing rich line features and fusing point-line features can make up for the problem of insufficient feature matching, thereby improving the positioning accuracy and robustness of the system; 2) Targeting features The point method visual odometry is insensitive to moving objects due to redundant feature points, which leads to the problem of reduced accuracy of the visual odometry. In the feature extraction stage of the visual odometry, the image is divided into regions and adapted according to the coefficient of variation of the regional grayscale. Set the feature point extraction threshold and use the quad-tree structure to manage the feature points to achieve uniform extraction of ORB (Oriented FAST and Rotated BRIEF, ORB) features; 3) Feature extraction is performed through point and line feature fusion. First, the texture is normal Feature points can be uniformly extracted in the scene through point feature adaptation. Secondly, in low-texture scenes, since line features are more advantageous, not only adaptive thresholds are used to extract point features, but also the applicability of line features is used to supplement the scene information. , better provides parameters for robot pose estimation, and achieves more accuracy and robustness for subsequent feature matching, positioning and mapping. Finally, through dual-threading, point and line features are extracted separately with high concurrency. Feature fusion.

Description of the drawings

Figure 1 is a flow chart of the feature extraction method based on point-line feature fusion and adaptive threshold;

Figure 2 is a flow chart of the method for extracting feature points using adaptive threshold ORB;

Figure 3 is a schematic diagram of sub-image area division;

Figure 4 is a schematic diagram of constructing a sub-image pyramid;

Figure 5 is a schematic diagram of feature point extraction based on quadtree;

Figure 6 is the feature extraction experimental test chart;

Figure 7 is a flow chart of the line feature extraction method based on the EDLines algorithm;

Figure 8 is a schematic diagram of the distribution of point and line features in a low-texture scene.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention is described below through specific examples in the drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. The structures, proportions, sizes, etc. shown in the drawings of this specification are only used to coordinate with the content disclosed in the specification for the understanding and reading of professionals familiar with this technology. They are not used to limit the conditions under which the present invention can be implemented. Therefore, It has no technical substantive significance. Any structural modifications, changes in proportions or adjustments in size shall still fall within the scope of the technology disclosed in the present invention as long as it does not affect the effectiveness and purpose of the present invention. within the scope of the content. Furthermore, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessarily confusing the concepts of the present invention.

Here, it should also be noted that, in order to avoid obscuring the present invention with unnecessary details, only the structures and processing steps closely related to the solution of the present invention are shown in the drawings, and those that are not closely related to the present invention are omitted. other details.

As shown in Figure 1, the specific embodiment of the present invention includes the following steps:

Step 1: Image input, prepare for feature extraction: collect surrounding images through binocular cameras in real time, collect binocular images through a fixed number of frames, and set the image resolution to 752*480. The entire process can be seen in Figure 1 It is processed in parallel, and the point and line features are processed separately. This is to have better robustness to low-texture images and to better supplement the features, so that the pose estimation of the unmanned vehicle is more accurate.

Step 2: Use the feature point extraction algorithm in ORBSLAM3 to extract ORB feature points based on the adaptive threshold;

Further, as shown in Figure 2, in step 2, using the feature point extraction algorithm in ORBSLAM3 according to the adaptive threshold to extract ORB feature points also includes the following steps:

Step a: As shown in Figure 3, the input image is divided into four areas (area one, area two, area three, and area four). Different FAST angles are defined based on the different degrees of confusion of the grayscale values of the sub-images corresponding to each area. point detection threshold;

Step b: Define image I ₁ as the feature point image to be extracted. The height of I ₁ is h _i and the width is w _i . In order to improve the uniformity of ORB features, I ₁ is divided into a height of h _i /2 and a width of Four sub-images of w _i /2;

Step c: Use the coefficient of variation to describe the degree of dispersion of the gray value of all pixels in each sub-image. The coefficient of variation s is the ratio of the standard deviation of a set of data to the average value. The calculation formula is:

where g _i represents the gray value of a certain pixel in the sub-image, represents the average gray value of the sub-image, and t represents the number of pixels. The larger the variation coefficient, the higher the degree of gray confusion. Define g _s = s*30, and use the following formula to define the initial corner detection threshold g _th :

Step d: As shown in Figure 4, after setting the adaptive detection threshold according to the variation coefficient s, in order to ensure scale invariance, an eight-layer image pyramid is constructed for the sub-image, and then feature points are extracted for each layer of the pyramid. The definition needs to be from I ₁ The total number of ORB feature points extracted in is x _t , and the scaling factor γ _s of the image pyramid, then the number of feature points x _ti that needs to be extracted in the i-th layer of each sub-image can be expressed as:

where n is the number of pyramid layers, inv(γ _s ) represents the reciprocal of the scaling factor;

Step e: After calculating the number of feature points that need to be extracted for each layer of the pyramid, use a square grid with a side length of 30 pixels to divide the pyramid into regions, and perform FAST corner point extraction in each grid. If the number of corner points extracted in the grid is is 0, then reduce the detection threshold of the area to g _th /2, and extract again. If the corner points are still not extracted, abandon the grid to ensure the number of feature points, and repeat the above operation until all the corner points in the grid are Corner point extraction is completed;

Step f: As shown in Figure 5, after extracting all corner points, use the quadtree to manage the corner points, define the root node of the quadtree as the entire sub-image, and divide the sub-image into 4 areas as child nodes of the root. , if the number of corner points in the child node is 2 (or more), continue to divide the quadtree into the child node. If the number of corner points is 1, retain the child node and not continue to divide it. If the number of corner points is 0, delete the child node. sub-nodes until the number of feature points extracted from each layer of the pyramid reaches the set threshold, the corner points with the highest Harris corresponding values in each sub-node are retained, and finally the FAST corner points extracted from the four sub-images are merged , and calculate the corresponding descriptor, you can complete the adaptive threshold uniformization to extract ORB feature points. Use the standard ORB feature extraction algorithm in the computer vision library (OPENCV) and this improved algorithm to conduct comparative experiments, and set the ORB feature extraction The numbers of points are 500 and 1000 respectively. The extraction results are shown in Table 1:

Table 1 Feature extraction experimental results

It can be seen from the table that this improved feature point extraction algorithm achieves uniform feature extraction and enhances the ability of feature points to describe images. Since the final extraction result of each layer of the pyramid may be greater than the set extraction number, the number of final extracted feature points is Slightly more than the set extraction number.

Step 3: Line feature extraction based on EDLines algorithm;

Further, as shown in Figure 7, the line feature extraction based on the EDLines algorithm in step 3 includes the following steps:

Step Ⅰ: In order to improve the recognition effect, first add distortion correction to the input image. The distortion setting parameter value is the one that comes with the camera at the factory;

Step Ⅱ: Image smoothing processing, using Gaussian filter to remove noise, and filtering to suppress noise in the image. Gaussian kernel is 5×5, σ=1;

Step III: Use the Sobel gradient operator to calculate the gradient and direction in the image. The calculation formula is:

Where I(x,y) is the pixel value of the image at (x,y), g(x,y) is the size of the gradient, and angle(x,y) is the angle of the horizontal line;

Step IV: Traverse each pixel, select the pixel whose gradient value in the gradient direction is greater than or equal to the gradient value of the adjacent pixel, and define it as an anchor point;

Step V: Select an anchor point as the starting point, compare the amplitude values of adjacent pixels, select the pixel with the largest gradient value as the next anchor point, and finally connect adjacent anchor points to form an edge pixel chain;

Step VI: Divide one or more line segments from the edge pixel chain, traverse the pixels in order, and use the least squares method to fit the pixels. The expression is:

In the formula, x _{i and} y _i are pixel coordinates. When the result exceeds the threshold, for example, 1 pixel has no error, it is truncated until all pixels are processed. The maximum root mean square fitting error and the shortest line segment length are involved in the line segment fitting process. The fitting error is calculated using equation (5), and the calculation formula for the shortest line segment length is:

where N is the width of the input image.

Step 4: Double-thread extraction of features and fusion of point and line features.

As shown in Figure 8, in scenes with low texture or low lighting, sparse point features cannot meet actual feature requirements, making it difficult to guarantee system performance. Through the fusion of point and line features, the impact of only point features can be avoided. Moreover, running the two in parallel can not only improve the positioning accuracy and robustness of the algorithm, but also improve the mapping effect of the system and make the map more intuitive.

It is obvious to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be considered illustrative and not restrictive in any respect.

In addition, it should be understood that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the embodiments, those of ordinary skill in the art should understand that the technical solutions in each embodiment are also They can be appropriately combined to form other embodiments that can be understood by those skilled in the art. Without departing from the spirit and scope of the technical solution of the present invention, they should all be included in the scope of the claims of the present invention.

Claims (3)

1. A feature extraction method based on point-line feature fusion and adaptive threshold, which is characterized by: including the following steps: Step 1: Image input, prepare for feature extraction; Step 2: Use the feature point extraction algorithm in ORBSLAM3 to extract ORB feature points based on the adaptive threshold; Step 3: Line feature extraction based on EDLines algorithm; Step 4: Double-thread extraction of features and fusion of point and line features. 2. The feature extraction method based on point-line feature fusion and adaptive threshold according to claim 1, characterized in that: in step 2, the feature point extraction algorithm in ORBSLAM3 is used to extract ORB feature points according to the adaptive threshold. Includes the following steps: Step a: Divide the input image into four areas, and define different FAST corner detection thresholds based on the different degrees of confusion of the sub-image grayscale values corresponding to each area; Step b: Define image I ₁ as the feature point image to be extracted. The height of I ₁ is _hi and the width is _wi . In order to improve the uniformity of ORB features, I ₁ is divided into a height of _hi /2 and a width of wi. Four sub-images of w _i /2; Step c: Use the coefficient of variation to describe the degree of dispersion of the gray value of all pixels in each sub-image. The coefficient of variation s is the ratio of the standard deviation of a set of data to the average value. The calculation formula is: where g _i represents the gray value of a certain pixel in the sub-image, represents the average gray value of the sub-image, and t represents the number of pixels. The larger the variation coefficient, the higher the degree of gray confusion. Define g _s = s*30, and use the following formula to define the initial corner detection threshold g _th : Step d: After setting the adaptive detection threshold according to the variation coefficient s, in order to ensure scale invariance, construct an eight-layer image pyramid for the sub-image, then extract feature points from each layer of the pyramid, and define the ORB feature points that need to be extracted from I ₁ The total number is x _t , and the scaling factor γ _s of the image pyramid, then the number of feature points x _ti that need to be extracted in the i-th layer of each sub-image can be expressed as: where n is the number of pyramid layers, inv(γ _s ) represents the reciprocal of the scaling factor; Step e: After calculating the number of feature points that need to be extracted for each layer of the pyramid, use a square grid with a side length of 30 pixels to divide the pyramid into regions, and perform FAST corner point extraction in each grid. If the number of corner points extracted in the grid is is 0, then reduce the detection threshold of the area to g _th /2, and extract again. If the corner points are still not extracted, abandon the grid to ensure the number of feature points, and repeat the above operation until all the corner points in the grid are Corner point extraction is completed; Step f: After extracting all corner points, use the quadtree to manage the corner points. Define the root node of the quadtree as the entire sub-image. Divide the sub-image into 4 areas as child nodes of the root. If the corner point in the child node If the number is 2 (and above), continue to divide the quadtree into the sub-node. If the number of corner points is 1, keep the sub-node and not continue to divide it. If the number of corner points is 0, delete the sub-node until the pyramid After the number of feature points extracted from each layer of images reaches the set threshold, the corner points with the highest Harris corresponding values in each sub-node are retained. Finally, the FAST corner points extracted from the four sub-images are merged and the corresponding descriptions are calculated. , the adaptive threshold homogenization can be completed to extract ORB feature points. 3. The feature extraction method based on point-line feature fusion and adaptive threshold according to claim 1, characterized in that: the step 3 is based on the EDLines algorithm for line feature extraction and further includes the following steps: Step Ⅰ: In order to improve the recognition effect, first add distortion correction to the input image. The distortion setting parameter value is the one that comes with the camera at the factory; Step Ⅱ: Image smoothing processing, using Gaussian filter to remove noise, and filtering to suppress noise in the image. Gaussian kernel is 5×5, σ=1; Step III: Use the Sobel gradient operator to calculate the gradient and direction in the image. The calculation formula is: Where I(x,y) is the pixel value of the image at (x,y), g(x,y) is the size of the gradient, and angle(x,y) is the angle of the horizontal line; Step IV: Traverse each pixel, select the pixel whose gradient value in the gradient direction is greater than or equal to the gradient value of the adjacent pixel, and define it as an anchor point; Step V: Select an anchor point as the starting point, compare the amplitude values of adjacent pixels, select the pixel with the largest gradient value as the next anchor point, and finally connect adjacent anchor points to form an edge pixel chain; Step VI: Divide one or more line segments from the edge pixel chain, traverse the pixels in order, and use the least squares method to fit the pixels. The expression is: In the formula, x _{i and} y _i are pixel coordinates. When the result exceeds the threshold, for example, 1 pixel has no error, it is truncated until all pixels are processed. The maximum root mean square fitting error and the shortest line segment length are involved in the line segment fitting process. The fitting error is calculated using equation (5), and the calculation formula for the shortest line segment length is: where N is the width of the input image.