CN111444948A - Image feature extraction and matching method - Google Patents

Abstract

The invention discloses a method for extracting and matching image features, comprising step 1: preliminary screening of feature points; step 2: secondary screening of candidate corner points obtained by S1 by using gradients in the X and Y directions in the candidate corner points; Step 3: Pixel-level corner detection; Step 4: Sub-pixel-level corner detection, obtain the sub-pixel-level corner coordinates of the pixel-level corner obtained in S3 by iteratively optimizing the Harris position; Step 5: Rotation-invariant description of rapid changes Sub-calculation; Step 6: Feature extraction and feature matching are performed. The invention is based on Harris corner detection, through two candidate corner screenings to improve the detection speed of the corner, and through iterative optimization to improve the position accuracy of the corner detection, and finally, using the rotation-invariant fast transformation descriptor to represent features.

Description

An image feature extraction and matching method

technical field

The invention relates to an image feature extraction and matching method, in particular to a Harris-based feature extraction and matching method, which belongs to the field of image processing.

Background technique

Image matching is a method of finding similar image parts in different images. It is widely used in image fusion, object recognition, computer vision and other fields. Currently, image matching can be divided into gray-based methods and feature-based methods. As we all know, features are very important information in images. For images, features are abstract descriptions of local information in images. Features can greatly reduce the amount of data while preserving key information about the image. Furthermore, these features have good adaptability to image noise, grayscale variation, image deformation and occlusion, so image feature-based matching is more and more widely used in practice. The commonly used method is Harris corner detection, but its accuracy is at the pixel level, there is no suitable descriptor, and the computational complexity is large.

SUMMARY OF THE INVENTION

In view of the above-mentioned prior art, the technical problem to be solved by the present invention is to provide an image feature extraction and matching method that effectively improves the speed and accuracy of image feature extraction and matching.

In order to solve the above-mentioned technical problems, an image feature extraction and matching method of the present invention includes the following steps:

S1: Preliminary screening of feature points, specifically:

Convert the collected color image to grayscale image, the conversion formula is as follows:

Gray=(306*R+601*G+1147*B)>>10

Among them, Gray represents the gray value of the image, and R, G, and B represent the values of the red, green, and blue channels, respectively. The candidate is selected by the similarity between each pixel in the image and 8 other pixels in its neighborhood. Corner, the similarity between two pixels is determined according to the grayscale difference between them. For the pixel point P at point (i, j), if the grayscale between 8 pixels in its neighborhood and this point is If the absolute value P of the difference is less than the set gray threshold T ₁ , the point is considered to be similar to the point P, and the similarity between the point P and the 8 pixel points in the field is detected, and the number of points similar to the point P is recorded, denoted as N (i,j);

P belongs to a pixel point in a local area. According to the N(i,j) value of point P, it is judged whether point P is a possible corner point. If N(i,j) of point P is in the interval (3,6) In between, point P is regarded as a possible corner, traverses all pixels in the image, and selects all eligible pixels as candidate corners;

S2: Use the gradients in the X and Y directions in the candidate corners to perform secondary screening on the candidate corners obtained by S1;

S3: pixel-level corner detection, specifically: calculating the autocorrelation matrix of each candidate corner obtained in step 2: calculating the gradient product corresponding to each candidate corner to obtain the autocorrelation matrix M ₁ :

I _x and I _y respectively represent the gradient values of the candidate corners in the x and y directions, and then use the Gaussian kernel function G(x, y, σ) and M ₁ for convolution to obtain a new autocorrelation matrix M ₂ ;

The corner response function value of the candidate corner is calculated and used to determine whether it is the correct corner. The corner response function value R is calculated as follows:

Det(M ₂ )=λ ₁ λ ₂

Tr(M ₂ )=λ ₁ +λ ₂

R=Det(M ₂ )-k*Tr ² (M ₂ )

where λ ₁ and λ ₂ are the eigenvalues of the autocorrelation matrix M ₂ , and k is a constant. If the CRF value R of the point is greater than the set threshold T ₃ , the point is selected as a pixel-level corner point;

S4: Sub-pixel-level corner detection, obtaining the sub-pixel-level corner coordinates of the pixel-level corner points obtained in S3 by iteratively optimizing the Harris position;

S5: Rotation-invariant fast-changing descriptor calculation, specifically:

The local area selected by the descriptor is a circular area centered on the feature point and a radius of 12 pixels. The selected local area is divided into three layers by three circles, and the radius centered on the feature point is 4 pixels, 8 pixels and 12 pixels, the circle in the middle is a sub-region, the ring in the middle layer is evenly divided into 4 sub-regions, the outermost ring is evenly divided into 8 sub-regions, a total of 13 sub-regions, in each sub-region Extract 8-direction gradient vector from , and finally obtain 104-dimensional feature vector as descriptor;

First, with the feature point as the center, according to the main direction of the neighborhood gradient of the feature point, all the pixels in the selected area are rotated in the same direction, and the main direction θ(i, j) satisfies:

Second, calculate the gradient direction and size of each pixel in the local area: calculate the gradient direction according to θ(i,j), divide the range from 0° to 360° into eight directions, each direction contains 45°, and determine each Which gradient direction does each pixel belong to, and the gradient magnitude m(i,j) satisfies:

m(i,j)=sqrt[(I(i+1,j)-I(i-1,j)) ² +(I(i,j+1)-I(i,j-1)) ² ]

The gradient weight of each pixel is determined by Gaussian, and the Gaussian weight w(i,j) of this point satisfies:

Finally, according to the position and gradient direction of each pixel, determine the statistical block it contributes and get the contribution of each pixel to the statistical block by multiplying the gradient interpolation coefficient by the gradient magnitude, each sub-region has 8 gradient directions , so there are 104 statistical blocks in total. By calculating the cumulative value of the contribution of all pixels to a statistical block, the gradient distribution eigenvalues of the corresponding sub-regions in the corresponding gradient directions are obtained, and a total of 104-dimensional gradient distribution eigenvectors are obtained. The formula for calculating the value coefficient is as follows:

The pixel contribution k(n) to the nth statistical block is:

k(n)=c·I(i,j)·w(i,j)·m(i,j)

Accumulate the contribution values of all pixels contributing to the nth statistical block to get K(n):

K(n)=∑k(n)

S6: Perform feature extraction and feature matching.

The present invention also includes:

1. In S2, the secondary screening of feature points using the gradients in the X and Y directions in the candidate corners is as follows: Assuming that the number of candidate corners retained after the initial screening is N, set 70% of the average value of the gradients in the X and Y directions as Threshold, eliminate pixels with gradient values less than the threshold, and keep pixels with gradient values greater than the threshold as new candidate corners.

图像原点指向距离角点O给定像素范围内的第i个点的向量为

则第k次迭代的向量

满足：2. Sub-pixel-level corner detection in S4, the sub-pixel-level corner coordinates of the pixel-level corner points obtained in S3 are obtained by iteratively optimizing the Harris position, specifically: the distance from any pixel-level corner point obtained in S3 O given pixel The points in the range include two types of points: the gray gradient value of the A-type point is 0, the gradient direction of the B-type point is perpendicular to the vector O; the vector of the image origin pointing to the point O is

The vector of the image origin pointing to the i-th point within a given pixel range from the corner point O is

then the vector of the k-th iteration

Satisfy:

是灰度梯度向量，

是第k次迭代的灰度梯度向量，在

对应的坐标点附近选取向量

得到

迭代直到满足条件

ε为设定的误差，则

对应的坐标为该角点O的亚像素级角点坐标。in,

is the grayscale gradient vector,

is the gray gradient vector of the k-th iteration, in

Select a vector near the corresponding coordinate point

get

iterate until the condition is met

ε is the set error, then

The corresponding coordinates are the sub-pixel-level corner coordinates of the corner O.

Beneficial effects of the present invention: The present invention fully considers the accuracy and efficiency of feature extraction. In view of the accuracy and efficiency of Harris corner detection, the present invention is based on Harris corner detection, through two candidate corner screenings to improve The detection speed of the corner points is improved, and the position accuracy of the corner point detection is improved through iterative optimization. Finally, the features are represented by the rotation-invariant fast transformation descriptor. The present invention can be used in the field of image processing. The main advantages of the present invention are reflected in:

1. The present invention greatly improves the speed of corner detection after two selections of candidate corner points.

2. The present invention uses the iterative optimization method to effectively improve the position accuracy of corner detection.

Description of drawings

Figure 1(a) is the effect diagram of Harris method;

Fig. 1 (b) is the effect drawing of the present invention;

Fig. 2 is the algorithm flow chart of the present invention;

Detailed ways

The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

2, the specific embodiment of the present invention includes the following steps:

Step 1: Preliminary screening of feature points;

Convert the collected color image to grayscale image, the conversion formula is as follows:

Gray=(306*R+601*G+1147*B)>>10 (1)

In formula (1), Gray represents the grayscale value of the image, and R, G, and B represent the values of the red, green, and blue channels, respectively. The candidate corners are selected by the similarity of each pixel in the image to 8 other pixels in its neighborhood. The similarity between two pixels is determined according to their grayscale differences. For the pixel point P at point (i, j), if the absolute value P of the grayscale difference between the 8 pixels in its neighborhood and the point is less than the set gray threshold T ₁ , it is considered that the point is different from the point P resemblance. Detect the similarity between point P and 8 pixels in the field, and record the number of points similar to point P, denoted as N(i,j).

According to the value N(i,j), it can be judged whether the point P is a possible corner point. If the N(i,j) of point P is very large, the nearby pixels are similar to point P, and P belongs to the pixels in a certain local area. If the N(i,j) of point P is relatively small, there is no point near it similar to P, and P is an isolated pixel or noise. In this paper, point P can be regarded as a possible corner point if N(i,j) of point P is between the interval (3,6). Traverse all pixels in the image and select all eligible pixels as candidate corners.

Step 2: Secondary screening of feature points;

After the initial screening, the number of pixels calculated in subsequent steps will be greatly reduced. Usually, the gray value of a corner point varies greatly around it, and the gradient value is relatively large. The gradients in the X and Y directions in the candidate corners can be used for secondary screening to further reduce the computational complexity of the feature extraction algorithm.

Assume that the number of candidate corner points retained after the initial screening is N. In this paper, 70% of the average value of the gradients in the X and Y directions is set as the threshold, the pixels with gradient values less than the threshold are eliminated, and the pixels with larger gradients are reserved as new candidate corner points. The formula for the secondary screening threshold is as follows:

表示第i个候选角点在x方向的梯度值，N表示候选角点的数量。公式(3)中，

表示第i个候选角点在y方向的梯度值，N表示候选角点的数量。In formula (2),

Represents the gradient value of the ith candidate corner in the x direction, and N represents the number of candidate corners. In formula (3),

Represents the gradient value of the ith candidate corner in the y direction, and N represents the number of candidate corners.

Step 3: Pixel-level corner detection;

Compute the autocorrelation matrix for each candidate corner. Calculate the gradient product corresponding to each candidate corner to obtain the autocorrelation matrix M ₁ as shown below.

In formula (4), I _x and I _y represent the gradient values of the candidate corner points in the x and y directions, respectively.

Then use the Gaussian kernel function G(x, y, σ) to convolve with M ₁ to obtain a new autocorrelation matrix M ₂ .

Next, the corner response function (CRF) value of the candidate corner is calculated and used to determine whether it is the correct corner. The eigenvalues of the autocorrelation matrix M ₂ are λ ₁ and λ ₂ . When both eigenvalues are small, it means that the point lies in a flat area. When one eigenvalue is small and the other is large, it means that the point is on the edge. When both eigenvalues are large, it is a corner point. To avoid solving for eigenvalues, corner response functions are often used. The CRF value R is calculated as follows:

Det(M ₂ )=λ ₁ λ ₂ (5)

Tr(M ₂ )=λ ₁ +λ ₂ (6)

R=Det(M ₂ )-k*Tr ² (M ₂ ) (7)

In formula (5), λ ₁ and λ ₂ respectively represent the eigenvalues of the autocorrelation matrix M ₂ , and Det(M ₂ ) represents the determinant of the matrix M ₂ . In formula (6), Tr(M ₂ ) represents the trace of the matrix M ₂ . In Equation (7), k is a constant with a value typically in the range of 0.04 to 0.06. The CRF values of the corners are positive and usually not very small. If the CRF value _R of a point is greater than the set threshold T3, the point is selected as a corner point.

Step 4: Sub-pixel corner detection;

More accurate sub-pixel corner coordinates are obtained by iteratively optimizing the Harris position. For the corner point O, the points close to the point O can be divided into two types, one is on the edge, and the other is not on the edge. The gray gradient value of point A is 0, and the gradient direction of point B is perpendicular to the vector OB, so it can be considered that the gray gradient near the corner point O is perpendicular to the line connecting this point to the corner point O.

The mathematical expression is:

是灰度梯度向量，

是图像原点指向O点的向量，

是图像原点指向第i个点的向量。In formula (8),

is the grayscale gradient vector,

is the vector from the origin of the image to point O,

is the vector from the image origin to the ith point.

In practice, images are often affected by noise, so the left-hand side of equation (10) is not equal to 0. Assuming that the error is ε, we have:

The sum of the cumulative error for all points near the corner O is E:

In this way, the problem of solving the exact position of the corner points is transformed into the problem of minimizing the sum of errors E. This problem can be solved iteratively by multiplying both sides of equation (10) by

Substituting all points in the region around point O into equation (11), the sum is obtained:

的表达式为：According to formula (12), it can be obtained

The expression is:

附近选取向量

继续执行公式(13)得到

不断迭代，直到满足条件

即可，一般选取ε＝1.0e^-6，最终获得了准确的亚像素级角点坐标。exist

pick vector nearby

Continue to execute formula (13) to get

Iterate continuously until the condition is met

That is, ε=1.0e ⁻⁶ is generally selected, and accurate sub-pixel-level corner coordinates are finally obtained.

Step 5: Rotation-invariant fast-changing descriptor calculation;

The local area selected by the descriptor is a circular area centered on the feature point with a radius of 12 pixels. The selected local area is divided into three layers by three circles, and the radii centered on the feature point are 4 pixels, 8 pixels and 12 pixels. The circle in the middle is a subregion, the ring in the middle layer is evenly divided into 4 subregions, and the ring in the outermost layer is evenly divided into 8 subregions, for a total of 13 subregions. 8-direction gradient vectors are extracted in each sub-region, and finally 104-dimensional feature vectors are obtained as descriptors.

First, with the feature point as the center, all the pixels in the selected area are rotated in the same direction according to the main direction of the neighborhood gradient of the feature point. The purpose of the rotation is to align the principal directions so that consistent feature vectors can be extracted in similar local regions, thus ensuring that the rotation is invariant. The main direction θ(i,j) is calculated as follows:

In formula (14), I(i,j) represents the gray value at point (i,j).

Second, the gradient direction and magnitude of each pixel in the local region are calculated. The gradient direction is calculated according to equation (14), the range of 0° to 360° is divided into eight directions, each direction contains 45°, and which gradient direction each pixel belongs to is determined. The gradient magnitude m(i,j) is calculated as follows:

m(i,j)=sqrt[(I(i+1,j)-I(i-1,j)) ² +(I(i,j+1)-I(i,j-1)) ² ] (15)

In formula (15), I(i,j) represents the gray value at point (i,j).

The magnitude of the gradient weight for each pixel is determined by a Gaussian. The Gaussian weight w(i,j) of this point is calculated as follows:

表示方差。In formula (16), r represents the distance from point (i, j) to the corner point,

represents the variance.

Finally, according to the position and gradient direction of each pixel, the statistical block contributed by it is determined, and its contribution is determined by linear interpolation. Each subregion has 8 gradient directions, so there are 104 statistical blocks in total. The contribution is obtained by multiplying the gradient interpolation coefficient by the gradient magnitude. The cumulative value of the contribution of all pixels to a certain statistical block is the characteristic value of the gradient distribution of the corresponding sub-region in the corresponding gradient direction. After completing the computation of all statistical blocks, a 104-dimensional gradient distribution feature vector can be obtained. in. The formula for calculating the difference coefficient d(i,j) is as follows:

In formula (17), r represents the distance from the point (i, j) to the corner point.

Then the contribution k(n) of the pixel to the nth statistical block is:

k(n)=c·d(i,j)·w(i,j)·m(i,j) (18)

In formula (18), c represents the contribution coefficient, d(i, j) represents the difference coefficient, w(i, j) represents the Gaussian weight coefficient, and w(i, j) represents the gradient magnitude.

Accumulate the contribution values of all pixels contributing to the nth statistical block to get K(n):

K(n)=∑k(n) (19)

Step 6: Perform feature extraction and feature matching.

The experimental verification is carried out by taking two pictures. In terms of algorithm time, the Harris algorithm takes 5.04s, the improved Harris algorithm takes 1.276s, and its time consumption is only 25.3% of the Harris algorithm. This algorithm effectively improves the speed of feature extraction and matching. In terms of feature matching, when there are 400 feature points in the graph, the number of correct matches for the Harris algorithm is 87, and the number of correct matches for the improved Harris algorithm is 121. The algorithm effectively increases the number of correctly matched feature points.

Claims (3)

1. an image feature extraction and matching method, is characterized in that, comprises the following steps: S1: Preliminary screening of feature points, specifically: Convert the collected color image to grayscale image, the conversion formula is as follows: Gray=(306*R+601*G+1147*B)>>10 Among them, Gray represents the gray value of the image, and R, G, and B represent the values of the red, green, and blue channels, respectively. The candidate is selected by the similarity between each pixel in the image and 8 other pixels in its neighborhood. Corner, the similarity between two pixels is determined according to the grayscale difference between them. For the pixel point P at point (i, j), if the grayscale between 8 pixels in its neighborhood and this point is If the absolute value P of the difference is less than the set gray threshold T ₁ , the point is considered to be similar to the point P, and the similarity between the point P and the 8 pixel points in the field is detected, and the number of points similar to the point P is recorded, denoted as N (i,j); P belongs to a pixel point in a local area. According to the N(i,j) value of point P, it is judged whether point P is a possible corner point. If N(i,j) of point P is in the interval (3,6) In between, point P is regarded as a possible corner, traverses all pixels in the image, and selects all eligible pixels as candidate corners; S2: Use the gradients in the X and Y directions in the candidate corners to perform secondary screening on the candidate corners obtained by S1; S3: pixel-level corner detection, specifically: calculating the autocorrelation matrix of each candidate corner obtained in step 2: calculating the gradient product corresponding to each candidate corner to obtain the autocorrelation matrix M ₁ :

I _x and I _y respectively represent the gradient values of the candidate corners in the x and y directions, and then use the Gaussian kernel function G(x, y, σ) and M ₁ for convolution to obtain a new autocorrelation matrix M ₂ ; The corner response function value of the candidate corner is calculated and used to determine whether it is the correct corner. The corner response function value R is calculated as follows: Det(M ₂ )=λ ₁ λ ₂ Tr(M ₂ )=λ ₁ +λ ₂ R=Det(M ₂ )-k*Tr ² (M ₂ ) where λ ₁ and λ ₂ are the eigenvalues of the autocorrelation matrix M ₂ , and k is a constant. If the CRF value R of the point is greater than the set threshold T ₃ , the point is selected as a pixel-level corner point; S4: Sub-pixel-level corner detection, obtaining the sub-pixel-level corner coordinates of the pixel-level corner points obtained in S3 by iteratively optimizing the Harris position; S5: Rotation-invariant fast-changing descriptor calculation, specifically: The local area selected by the descriptor is a circular area centered on the feature point and a radius of 12 pixels. The selected local area is divided into three layers by three circles, and the radius centered on the feature point is 4 pixels, 8 pixels and 12 pixels, the circle in the middle is a sub-region, the ring in the middle layer is evenly divided into 4 sub-regions, the outermost ring is evenly divided into 8 sub-regions, a total of 13 sub-regions, in each sub-region Extract 8-direction gradient vector from , and finally obtain 104-dimensional feature vector as descriptor; First, with the feature point as the center, according to the main direction of the neighborhood gradient of the feature point, all the pixels in the selected area are rotated in the same direction, and the main direction θ(i, j) satisfies:

Second, calculate the gradient direction and size of each pixel in the local area: calculate the gradient direction according to θ(i,j), divide the range from 0° to 360° into eight directions, each direction contains 45°, and determine each Which gradient direction does each pixel belong to, and the gradient magnitude m(i,j) satisfies: m(i,j)=sqrt[(I(i+1,j)-I(i-1,j)) ² +(I(i,j+1)-I(i,j-1)) ² ] The gradient weight of each pixel is determined by Gaussian, and the Gaussian weight w(i,j) of this point satisfies:

Finally, according to the position and gradient direction of each pixel, determine the statistical block it contributes and obtain the contribution of each pixel to the statistical block by multiplying the gradient interpolation coefficient by the gradient magnitude, each sub-region has 8 gradient directions , so there are 104 statistical blocks in total. By calculating the cumulative value of the contribution of all pixels to a statistical block, the gradient distribution eigenvalues of the corresponding sub-regions in the corresponding gradient directions are obtained, and a total of 104-dimensional gradient distribution eigenvectors are obtained. The formula for calculating the value coefficient is as follows:

The pixel contribution k(n) to the nth statistical block is: k(n)=c·I(i,j)·w(i,j)·m(i,j) Accumulate the contribution values of all pixels contributing to the nth statistical block to get K(n): K(n)=∑k(n) S6: Perform feature extraction and feature matching. 2. a kind of image feature extraction and matching method according to claim 1, is characterized in that: S2 described utilizing the gradient of X and Y direction in candidate corner point to carry out secondary screening to feature point specifically: Suppose after initial screening The number of retained candidate corners is N, and 70% of the average value of the gradients in the X and Y directions is set as the threshold, the pixels with gradient values less than the threshold are eliminated, and the pixels with gradient values greater than the threshold are reserved as new candidate corners. 3. a kind of image feature extraction and matching method according to claim 1 and 2, is characterized in that: the described sub-pixel level corner point detection of S4, obtains the sub-pixel level corner point obtained in S3 by iterative optimization Harris position The pixel-level corner coordinates are specifically: the points within the given pixel range of any pixel-level corner O obtained from the distance S3 include two types of points: the gray gradient value of the A-type point is 0, and the gradient direction of the B-type point is vertical. in vector O; the vector from the origin of the image to point O is

The vector of the image origin pointing to the i-th point within a given pixel range from the corner point O is

then the vector of the k-th iteration

Satisfy:

in,

is the grayscale gradient vector,

is the gray gradient vector of the k-th iteration, in

Select a vector near the corresponding coordinate point

get

iterate until the condition is met

ε is the set error, then

The corresponding coordinates are the sub-pixel-level corner coordinates of the corner O.

Images