CN108550165A - A kind of image matching method based on local invariant feature - Google Patents

Abstract

The present invention is to provide a kind of image matching methods based on local invariant feature.Integral image and Hessian determinants of a matrix are sought to initial pictures；It establishes scale space pyramid and carries out the positioning of characteristic point；The principal direction of characteristic point is determined by Haar small echos, completes feature point extraction；Calculate the invariable rotary LBP features of image-region around each characteristic point, construction feature description；Using the nearest neighbor method of Euclidean distance, completes feature and slightly match；(6) by random sampling coherence method, remaining Mismatching point after thick matching process executes is rejected, completes the matching of feature essence.The method of the invention of the present invention ensureing match time and under conditions of accuracy, image occur scale, illumination and it is rotationally-varying when also there is certain robustness.

Description

An Image Matching Method Based on Local Invariant Features

technical field

The invention relates to an image feature extraction and image processing method, specifically an image matching method.

Background technique

Image matching refers to the image analysis and processing technology that aligns two images containing the same scene and determines the corresponding relationship between them. It is widely used in navigation, map and terrain matching, biometric recognition, text recognition, medical image analysis, computer fields of vision. In practical applications, images that need to be matched are often obtained by different sensors at different times and under different conditions, and there are differences in translation, scale, rotation, illumination, noise, viewing angle, etc. between them. posed enormous challenges. The disadvantage of correlation methods based on pixel gray values (such as SSAD, NNPROD, etc.) is that they are sensitive to transformations such as image scale, rotation, illumination, and viewing angle. In contrast, methods based on local features (such as corners, SURF (Speed Up Robust Features) feature points, etc.) method is better. The local invariant feature of the image has the characteristics of invariance under various image transformations (such as geometric transformation, illumination transformation, etc.), low redundancy, no need to segment the image in advance, and uniqueness. It has been widely used in image matching, Object recognition, image classification and image retrieval and other fields.

The use of local features in image matching technology can convert the complex image matching problem into the measurement problem of feature vectors, thereby improving the speed and robustness of the algorithm. The basic idea of the image matching method based on local invariant features is generally to detect the feature point set of the image first, then generate the feature vector based on the feature point and its local neighborhood information, and finally measure the similarity between the feature vectors to complete the image matching question. The SIFT (ScaleInvariant Feature Transform) published on IJCV in 2004 and the SURF based on Hessian matrix and Haar wavelet proposed to improve the running speed of SIFT are the two most representative local feature algorithms in the field of image matching. The problem of changing images is inefficient and the accuracy is not very high. The LBP (Local Binary Patterns) method, which is robust to illumination changes, is a binary description that expresses the size relationship between a certain pixel of a grayscale image and surrounding pixels, and was originally applied to image texture description. In recent years, many extended LBPs have been proposed, which are simple to calculate and have the advantages of partial scale, rotation and brightness invariance.

In "Image Matching Algorithm Combining SIFT and Rotation Invariant LBP", the rotation invariant LBP feature is used to describe the local image area around the SIFT key point. This method is very robust to image transformations such as scale, rotation, and illumination. However, in terms of computing speed, it cannot meet the high real-time requirements.

In the patent document titled "An Image Matching Method Combining LBP Feature Extraction and SURF Feature Extraction Method", when describing feature points, the Haar feature descriptor in the original SURF method is retained, and the Haar feature descriptor is combined with LBP The combination of rotation invariant descriptors makes this image matching method have a better matching effect than the original SURF method, but the two feature descriptions increase the complexity of the method and decrease the operation speed.

As mentioned in the article "Review of Local Invariant Features", the matching methods of local invariant features that are widely used in the fields of image, video retrieval and target recognition include threshold-based matching methods, nearest-neighbor-based matching methods, and nearest-neighbor-based matching methods. Matching method for distance ratios. The three methods have their own advantages and disadvantages. Among them, the threshold-based matching method is simple and has a small amount of calculation, and the matching method based on the nearest neighbor distance ratio has high accuracy.

Due to the influence of various geometric and photometric transformations, noise, quantization errors, and similar local structures in images, there may still be false matches in the feature matching results based on similarity metrics. According to a set of sample data sets containing abnormal data, the random sampling consistency method calculates the mathematical model parameters of the data and obtains effective sample data. It was first proposed by Fischler and Bolles in 1981. It is widely used in image registration and splicing, and can find the optimal parameter model in a set of data sets containing "outside points" (points that do not conform to the optimal model) by means of continuous iteration.

Contents of the invention

The purpose of the present invention is to provide an image matching method based on local invariant features that is robust to changes in image scale, illumination and rotation under the condition of ensuring matching time and accuracy.

The purpose of the present invention is achieved like this:

(1) obtain the determinant of integral image and Hessian matrix to initial image;

(2) Establish a scale-space pyramid and locate feature points;

(3) Determine the main direction of the feature point by Haar wavelet, and complete the feature point extraction;

(4) Calculate the rotation-invariant LBP feature of the image area around each feature point, and construct the feature descriptor;

(5) Use the nearest neighbor method of Euclidean distance to complete feature rough matching;

(6) Through the random sampling consistency method, the remaining mismatching points after the execution of the rough matching method are eliminated, and the feature fine matching is completed.

Compared with background technology, main features of the present invention are:

1. The present invention introduces the rotation-invariant LBP feature into the description of key points, constructs a description method with simple calculation and low dimensionality for key point description, and enhances the illumination robustness of the image matching method.

2. In the feature point extraction part of the present invention, the feature point extraction method provided by SURF is adopted, and the Hessian matrix and the integral graph method are used to perform fast calculations, while ensuring the matching performance, it effectively improves the slow calculation speed of the SIFT method based on the gradient graph. question.

3. In the feature description, the present invention uses the LBP rotation invariant descriptor to replace the Haar feature descriptor in the original SURF method, which has a good effect when dealing with rotation changes, illumination changes, and viewing angle changes, and does not affect image matching. speed.

4. In order to improve the accuracy of image matching, the present invention adopts the Euclidean distance nearest neighbor method with higher accuracy for rough matching.

5. In fine matching, the present invention adopts the random sampling consistency method to eliminate the wrong matching in the rough matching result, and further improves the accuracy of image matching.

The present invention conducts research on key issues such as extraction and description of local feature points, feature matching, and removal of mismatching, and proposes an image matching method based on local invariant features, which can not only ensure better image matching performance, but also have relatively Good scale, lighting and rotation robustness. Under the condition of ensuring the matching time and correct rate, it also has certain robustness when the scale, illumination and rotation of the image change.

The beneficial effects of the present invention are: in the feature point extraction part, the Hessian matrix and the integral graph method are used for fast calculation, and the scale image pyramid is constructed to ensure the rapidity and scale invariance of image matching; in the feature description part, the rotation is not The variable LBP feature is introduced into the description of key points, and a description method with simple calculation and low dimensionality is constructed for key points, which enhances the robustness of scale and illumination; in the rough matching part of feature corners, Euclidean The nearest neighbor method of distance makes the corner matching faster; in the feature corner fine matching part, the random sampling consistency method is used to eliminate the remaining mismatching points after the rough matching method is executed, and the matching result is optimized to make the corner matching accurate rate is higher.

Description of drawings

Fig. 1 is a flowchart of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described below with reference to the accompanying drawings.

In conjunction with Fig. 1, the image matching method based on local feature point extraction and description of the present invention comprises the following steps:

Step 1: Image feature point extraction, including:

(1) Find the integral image of the initial image and the determinant of the Hessian matrix

The initial image that needs to be matched is traversed to obtain the integral image of the initial image, and the determinant of the Hessian matrix of each point on the image is obtained.

For a given pixel point (x, x)=f(x, y) in image 1 to be matched, the Hessian matrix H(f(x, y)) of the pixel point is:

The H matrix (abbreviation of Hessian matrix, the same below) discriminant det(H) is:

in, and is the second-order partial derivative of the pixel point f(x,y).

The value of the discriminant is the eigenvalue of the H matrix, and all points can be classified according to the sign of the judgment result, and whether the point is an extreme point can be judged by the positive or negative value of the discriminant.

An H matrix can be obtained for each pixel. In order to make the feature points independent of scale, Gaussian filtering needs to be performed on them before constructing the H matrix. After filtering, calculate the H matrix, then the Hessian matrix H(x,σ) on the σ scale of the pixel x is defined as:

where L _xx (x,σ) is the second derivative of the kernel function of the Gaussian sum function Convolution with image 1 at point X, L _xy (x,σ) and L _yy (x,σ) are calculated in the same way, where

Through this method, the decision value of its H determinant can be calculated for each pixel in the image, and this value can be used to distinguish feature points. Replacing L _xx , L _xy , and _{L yy} with the result D _xx , D _xy , D _yy of the convolution of the frame filter and the image respectively, the matrix H _approx of the approximate Hessian is obtained, and its determinant det(H _approx ) is:

det(H _approx )＝D _xx D _yy -(0.9D _xy ) ² (4)

(2) Establish scale space pyramid and locate feature points

In order to have scale invariance, a scale space pyramid must be established. Keep the size of the image unchanged, and build a scale image pyramid by changing the template size of the box filter. The specific construction method is: the image scale space pyramid is divided into four layers, and each layer is filtered four times, and the first layer is filtered for the first time. The size of the template is given, which is 9×9, and the size of adjacent templates in the first layer differs by 6 pixels, that is, the sizes of the first to fourth filtering templates in the first layer are 9×9, 15×15, and 21 respectively. ×21 and 27×27; the size difference of adjacent filter templates in the second layer is 12 pixels, the size difference of adjacent filter templates in the third layer is 24 pixels, and the size difference of adjacent filter templates in the fourth layer is 48 pixels pixels, the size of the first template in each layer is equal to the size of the second template in the previous layer, so the sizes of the first to fourth filtering templates in the second layer are 15×15, 27×27, 39×39, 51×51, the sizes of the first to fourth filtering templates in the third layer are 27×27, 51×51, 75×75, and 99×99 respectively, and the sizes of the first to fourth filtering templates in the fourth layer are 51× 51, 99×99, 147×147, 195×195, the approximate scale corresponding to each filtering can be calculated by formula (5).

where σ _approx represents the scale.

Feature point positioning process: set a threshold for the response value of the Hessian matrix, all points smaller than this threshold are removed, and then through non-maximum value suppression, the response values of those points that are larger than the adjacent layer and the scale of this layer are larger Or all small points are selected as feature points, and finally the three-dimensional quadratic function is fitted to accurately determine the position and scale of the feature points.

(3) Determine the main direction of the feature point by Haar wavelet

In the circular neighborhood with the feature point as the center and 6σ (σ is the scale value of the feature point) as the radius, use the Haar wavelet template with a side length of 4σ to find the Haar wavelet response in the X and Y directions, and use the standard deviation The Gaussian function of 2σ weights the filtered area, and then takes the feature point as the center, uses a sector with a central angle of π/3 to scan a circle in the circular neighborhood, and calculates the angle within each π/3 angle scanned by the sector The sum of the Haar wavelet responses of the included image points, the direction of the maximum response is taken as the main direction of the feature point, where σ is the scale value of the feature point, and X and Y are the plane Cartesian coordinate system where the circular neighborhood is located of the two directions.

Step 2: Generate feature descriptors, including:

The basic LBP feature is a binary description of a 3×3 neighborhood, and the disadvantage is rotation-related. In order to obtain rotation invariance, the present invention uses the rotation-invariant LBP feature description of the area around the key point.

Let p _i (r, c, σ, θ) be a key point obtained in the key point extraction in step 1, where it is the position coordinate of (r, c) on the original image, σ and θ are respectively the values of p _i scale and direction. According to the size of σ, on the corresponding layer of the Gaussian pyramid where p _i is located, a 9×9 image area with p _i as the center is taken as the area to be described. To obtain rotation invariance, the image region is rotated to the reference orientation according to the size of θ. The description steps of the rotation-invariant LBP feature of the area to be described are as follows:

(1) In the 8×8 area to be described, take each pixel point p _j as the center, and obtain the rotation-invariant LBP feature centered on it, denoted as lbp _j (j=1,2, ..., 64).

(2) Intuitively speaking, the farther the pixel point p _j is from the center, the smaller the amount of information it contributes to describe p _i , so lbp _j is weighted, and the weighting coefficient w _j is:

w _j ＝exp{ ^- [(r _j -r _i ) ² +(c _j -c _i ) ² ]/(2σ ₀ ² )}/(2πσ ₀ ² ) (6)

Where (r _j , c _j ) and (r _j , c _i ) are the coordinates of the pixel point p _j and the center point p _i in the image area to be described, and σ ₀ is a selected constant.

(3) Combine all the calculated weighted LBP eigenvalues into a one-dimensional vector, denoted as T _i ,

T _i ＝[w ₁ ·lbp ₁ w ₂ ·lbp ₂ ... w ₆₄ ·lbp ₆₄ ] (7)

(4) In order to eliminate the influence of illumination changes, T _i is normalized, that is

In summary, the finally obtained 64-dimensional vector T _i is the description of the area around the key point T _i .

Step 3: rough image feature matching, including:

After the description vector of the key point is generated, the present invention adopts the distance formula as

Formula (9) distance is used as the similarity judgment measure between key points. Among them, T _A =[a ₁ a ₂ ... a _n ] and T _B =[b ₁ b ₂ ... b _n ] are description vectors of key points A and B respectively. Matching strategy: Take a key point A in image 1, and find the two key points B and C closest to the description vector in image 2, if the shortest distance ||T _A -T _B || ₁ The ratio of the next closest distance ||T _A -T _C || ₁ is smaller than the threshold t, that is

Then the key point A is considered to match the nearest key point B.

Step 4: Fine matching of image features, including:

The random sampling consistency method is based on a set of sample data sets containing abnormal data, calculating the mathematical model parameters of the data, and obtaining effective sample data. The basic assumption of this method is that the sample contains correct data (data that fits the model) and abnormal data (data that does not fit the model). And given a correct set of data, there are ways to calculate the parameters of a model that fits those data.

Assuming that the corresponding matching feature point sets in the two images 1 and 2 to be stitched are f, the steps of removing the mismatching points by the random sampling consistency method are as follows:

(1) Randomly select three pairs of non-collinear matching points from the matching feature point set f, and calculate their transformation matrix M.

(2) Calculate the corresponding point according to the transformation matrix M, for example, for the point (a, b), calculate b'=M(a).

(3) Cut off the distance between b' and b, if it is within the threshold, then (a, b) is an inner point, otherwise it is an outer point.

(4) If the number of interior points satisfies the condition, then exit; after loop k steps, if the number of interior points does not meet the requirement, take the largest number of interior points. If the exit condition is not met, go to step 4 (1).

Claims (1)

1. A method for image matching based on local invariant features, characterized in that: (1) obtain the determinant of integral image and Hessian matrix to initial image; (2) Establish a scale-space pyramid and locate feature points; (3) Determine the main direction of the feature point by Haar wavelet, and complete the feature point extraction; (4) Calculate the rotation-invariant LBP feature of the image area around each feature point, and construct the feature descriptor; (5) Use the nearest neighbor method of Euclidean distance to complete feature rough matching; (6) Through the random sampling consistency method, the remaining mismatching points after the execution of the rough matching method are eliminated, and the feature fine matching is completed.