CN101567051B - Image matching method based on characteristic points - Google Patents

Abstract

The invention belongs to the technical field of pattern recognition and image processing, and relates to an image registration method based on feature points. According to the K nearest neighbor structure of feature points, the present invention defines a circular corner code model and uses the model for point pattern matching. The similarity ring corner code not only describes the spatial structure similarity of two feature points, but also can be used for local map estimation. The similarity of two feature points is determined by the similarity length of the maximum similarity ring-type corner code associated with them. The invention performs structural matching according to the similarity of local spatial structures of feature points, optimizes matching by using local mapping clustering, and optimizes matching results for optimal mapping estimation. This method can be used for image registration based on feature points, image stitching and mosaic, motion tracking, information fusion of medical single-modal and multi-modal images, and content-based image retrieval, etc.

Description

A Method of Image Registration Based on Feature Points

technical field

The invention belongs to the technical field of pattern recognition and image processing, and in particular relates to an image registration method based on feature points. This method can be used for image registration based on feature points, image stitching and mosaic, motion tracking, information fusion of medical single-modal and multi-modal images, and content-based image retrieval, etc. the

Background technique

Point pattern matching can be understood as feature-based image matching, which searches for correspondences from two point sets to be matched and estimates the mapping between point sets. Point pattern matching is widely used in remote sensing (image mosaic, image mosaic), medical image (condition diagnosis and tracking) and computer vision (target or scene recognition, motion tracking). the

Although many excellent point pattern matching algorithms have been proposed in recent years, almost none of them can meet the application requirements in terms of computing time and accuracy. Currently published algorithms can be roughly divided into 5 categories: clustering algorithm (see reference [1][2]), shape context algorithm (see reference [3]), slack label algorithm (see reference [4] [5]), stepwise optimization algorithm (see reference [6]), graph matching algorithm (see reference [7]). the

in,

The clustering algorithm assumes that the transformation model is a similar transformation, because only two pairs of matching points are needed to estimate the similarity transformation, the algorithm first selects two pairs of points from the two sets of points to be matched to combine, then estimates the transformation coefficient, and uses the estimated transformation to verify other possible matches right. Because the estimated transformation of the correctly matched points is also correct, clustering with this transformation will gather all possible correct matching points into this class, and if there are many correctly matched points, the corresponding class will also be very large. On the contrary, if the transformation is estimated by the wrongly matched point pairs, the transformation must be wrong, and one of the characteristics of the wrong transformation is randomness. Most of the wrong matches are random, that is to say, using the wrong transformation as a classifier for clustering will only gather a few points. According to this property, the algorithm clusters two feature point sets, and among the clustering results, the largest cluster will be output as a corresponding relationship for subsequent processing. The disadvantage of this type of algorithm is that the calculation complexity is high, time-consuming, and if the size difference between the two feature point sets is very large, the performance of the algorithm will be very poor. The Hough transform was later used in preprocessing to speed up calculations. However, matching is still a very time-consuming task, and when there are many noise points or the size of the two point sets is very different, the algorithm does not necessarily guarantee that it can Find the best correspondence. the

The shape context algorithm is also a point pattern matching technology that has been studied in recent years. The shape context describes the spatial distribution information of other feature points in a point set relative to a certain feature point. The matching feature points in the same shape of the two images have similar shape contexts, and each feature point corresponds to an orientation histogram to represent the spatial constraints of other feature points relative to itself. This algorithm can solve the point pattern matching of non-rigid transformation. Most similar algorithms use spline curves to estimate the transformation. However, the problem with this type of algorithm is that it is dependent on the sampling of feature points. Algorithms are mostly tested based on synthetic data. For simple shape matching applications, the performance of the algorithm is relatively good, but the computational complexity of this type of algorithm is relatively high. Each feature point needs to use all other points to calculate the shape context. Therefore, in In complex scenes, when the recognized objects are difficult to segment, the performance of the algorithm is very poor. the

The loose label algorithm defines a probability distribution on the transformation relationship, and then optimizes it with a discrete algorithm. The algorithm is actually an iterative process, starting with a relatively rough match, and then gradually updating the matching probability matrix. This algorithm is easy converges to a local extremum. the

The step-by-step optimization algorithm also uses the global feature points to construct a compatible function, and solves the correspondence by updating the compatible function. This algorithm has the same defect as the relaxation algorithm. The performance of the two types of algorithms is relatively poor when the feature point set is relatively large and the noise is relatively large, and there is no guarantee that the algorithm can converge in various situations. the

Graph matching algorithm is a hot point pattern matching algorithm researched in recent years. By constructing the two point sets to be matched into a weighted graph, an attribute relationship graph, etc., and then searching for compatible parts in the two graphs, that is, the parts that are likely to match, or perform subgraph matching, or complete mapping matching , to ensure that the optimization function is minimized. This type of algorithm can be attributed to the graph search problem, which is an NP-hard problem, and it cannot be guaranteed to have solutions for various test cases. Many algorithms also use the optimization function to estimate and get a better solution instead of the optimal solution. Another defect of graph matching is that it cannot handle two point sets that contain a lot of noise points. In addition to the long calculation time, the algorithm It is also difficult to obtain a reasonable correspondence. the

References relevant to the present invention are:

[1] A. Goshtasby. Description and discrimination of planar shapes using shape matrices. IEEE Trans. Pattern Analysis and Machine Intelligence, 7: 738-743, 1985.

[2] S.H.Chang, F.H.Cheng, W.H.Hsu, and G.Z.Wu. Fast algorithm for point pattern matching: Invariant to translations, rotations and scale changes. Pattern Recognition, 30: 311-320, 1997.

[3] S.Belongie, J.Malik, and J.Puzicha, "Shape Matching and Object Recognition Using ShapeContexts," IEEE Trans.Pattern Analysis and Machine Intelligence, vol.24, no.4, pp.509-522, Apr. .2002.

[4] W.J.Christmas, J.Kittler, and M.Petrou, "Structural Matching in Computer Vision Using Probabilistic Relaxation," IEEE Trans.Pattern Analysis and Machine Intelligence, vol.17, no.8, pp.749-764, Aug. 1994.

[5] R.C.Wilson and E.R.Hancock, "Structural Matching by Discrete Relaxation," IEEE Trans.Pattern Analysis and Machine Intelligence, vol.19, no.6, pp.634-648, June 1997.

[6] S.Gold and A.Rangarajan, "A Graduated Assignment Algorithm for Graph Matching," IEEE Trans.Pattern Analysis and Machine Intelligence, vol.18, no.4, pp.377-388, Apr.1996.

[7]Tibe’rio S.Caetano, Terry Caelli, Fellow, IEEE, Dale Schuurmans, and Dante A.C.Barone.Graphical Models and Point Pattern Matching.IEEE Trans.Pattern Analysis and Machine Intelligence, 28:1646-1663, 2006.

Contents of the invention

The present invention aims to overcome the deficiencies in the prior art, and proposes an image registration method based on feature points. the

The present invention first proposes a ring-type corner code model based on the spatial structure of feature point K neighbors, explores the similarity between ring-type corner codes and uses it to measure the similarity of the local space of two feature points . The similarity ring type corner code not only describes the similarity of the local spatial structure between two feature points, but also can be used to estimate the local mapping. The ring-type corner code model is established on the basis of similarity transformation, and the invention relaxes the similarity condition of the ring-type corner code at the same time, so that the matching algorithm is robust to a certain degree of affine and perspective transformation. The similarity between feature points is determined by the similarity length of the maximum similarity ring-type corner codes associated with them. The present invention first performs structural matching according to the similarity of local spatial structures of feature points, and then uses local mapping clusters associated with matching points to perform optimal matching. The results of the optimal matching are used to estimate the optimal mapping. In terms of speeding up the matching process, the present invention avoids a large number of unnecessary calculations by establishing an index in the search space, thereby ensuring that the matching algorithm meets the requirements of near-real-time applications while ensuring accuracy. the

The invention discloses a fast and effective point pattern matching algorithm, the flow chart of which is shown in accompanying drawing 1. Include the following steps:

Step 1): Feature extraction. Feature extraction is performed on two images to be matched. The matching algorithm is based on image feature points. The feature point set with zoom, rotation and translation invariance extracted in the early stage of matching is used as input, and the point sets can also have A certain degree of affine and perspective transformation;

Step 2): Similarity calculation, extracting the K-nearest neighbor structure of the feature points and constructing the circular corner codes of the feature points, indexing the search space, and calculating the maximum similarity between the feature points using binary search and incremental matching algorithms Corner code, and use the similarity length of the maximum similar circular corner as the similarity between two feature points, and determine the possible matching feature points while calculating the similarity;

Step 3): structure matching, the similarity between the feature points describes the similarity of the local spatial structure of the feature points in a physical sense, the present invention divides the possible matching feature points into three according to the similarity between the feature points Class: the feature point class badClass that cannot be matched, the feature point class unknownClass that may match but not unique, the feature point class goodClass that may match, and local mapping estimation for the matching points in goodClass;

Step 4): Optimizing the matching, using the local mapping associated with the matching points in goodClass to perform clustering. In the clustering results, the class with the most defined elements is the largest class, and the class with the second most elements is the second largest class. If the number of elements of the largest class is greater than a certain threshold and greater than the number of elements of the next largest class, or the ratio of the number of elements of the largest class to the second largest class is greater than a certain threshold, the largest class will be used as the matching result to estimate the optimal transformation;

Step 5): Optimal transformation estimation. The present invention assumes that the transformation between images is not only the similarity transformation, but also a certain degree of affine and perspective transformation, and the transformation estimation model is affine transformation. the

in particular,

The two point sets in step 1 of the present invention are defined as S and T, the sizes are n ₁ and n ₂ respectively, and points P _i ∈ S, Q _j ∈ T are defined at the same time.

The circular corner code in the described step 2 is defined as:

As shown in Figure 2, the K-nearest neighbor spatial structure of feature points can be represented by a circular corner code. For better definition ring type corner code, the present invention is defined as a corner code (Edge-Angle Code) EAC=(E, θ). Figure 2(a) has K corner codes (E _i , θ _i ), i=0, 1, . . . , K-1. These K corner codes are coded in counterclockwise order and they are connected into a ring structure successively to obtain the ring formula corner code of feature point P _c , as shown in accompanying drawing 2 (b), the present invention defines K as ring The length of the formula corner code.

The ring type corner code defined by the present invention has the following characteristics:

1) Connectivity: starting from any corner code and traversing forward, all corner codes can be traversed in order. For the convenience of expression, the present invention defines the traversal subscript as m _t , wherein t represents a non-negative integer, and m _t represents the result of taking the remainder of t to K, which can ensure that the subscript does not would cross the line.

在逆时针方向上邻接，而与邻居点 

在逆时针方向上间隔一个邻居点。因此对边角码顺序进行分析等价于对邻居点在空间结构上的约束关系进行分析。  2) Spatial constraints: the order of the corner codes describes the constraint relationship of the neighbor points of the feature point. Corner code order $({\mathrm{E.}}_{m_i},{\mathrm{\θ}}_{m_i}),({\mathrm{E.}}_{m_{(i+1)}},{\mathrm{\θ}}_{m_{(i+1)}}),({\mathrm{E.}}_{m_{(i+2)}},{\mathrm{\θ}}_{m_{(i+2)}})$ Describes that in the spatial structure, neighbor points point with neighbors

adjacency in the counterclockwise direction, and with neighbor points

Space one neighbor point in the counterclockwise direction. Therefore, analyzing the order of corner codes is equivalent to analyzing the constraint relationship of neighbor points on the spatial structure.

3) Equivalence: The same feature point can have K different ring-type corner codes, but they all describe the same local spatial structure of the feature point equivalently. As can be seen from accompanying drawing 2 (a), when using different neighbor points as the first corner code, the present invention will obtain different circular corner codes, but they are equivalent in a physical sense, that is, they all describe the same spatial structure. The present invention defines this characteristic of the ring type corner code as equivalence. the

The definition of the similar ring type corner code in the described step 2 is specifically:

As shown in accompanying drawing 3, if two ring-type corner codes satisfy the proportion of corresponding side from a certain opposite side, the corresponding angle is proportional and equal to 1, the present invention just says that these two ring-type corner codes are similar, wherein The length of a circular corner code is defined as the similar length of similar circular corner codes, and the corresponding spatial structure of similar circular corner codes is defined as similar spatial structure. The similar ring type corner code shown in accompanying drawing 3 satisfies the following formula from (E ₀ , E′ ₀ ):

$\frac{{{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; E.</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; |</span>{{\mathrm{<span\; class="notranslate">\; E.</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; |</span>{{\mathrm{<span\; class="notranslate">\; E.</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}_{\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span>}$

$\frac{{{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}}{{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}}<span\; class="notranslate">\; =</span>\frac{{{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; =</span>\frac{{{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}_{\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}$

For calculating the similar circular corner codes described in step two, the present invention also defines the addition operation of two adjacent corner codes, specifically:

$<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}_{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}_{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}_{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}}<span\; class="notranslate">\; )</span>$

The result of adding two adjacent corner codes is still one corner code, the side of the new corner code is the side of the first corner code, and the corner of the new corner code is the two added corner codes The sum of the angles of . The physical meaning of the addition operation actually means merging two counterclockwise adjacent neighbor points, that is, deleting the second neighbor point, and then updating the counterclockwise adjacent corner of the first side. the

For calculating the similar ring type corner code described in step 2, the present invention also defines the comparison operation of two corner codes, if two corner codes (E ₁ , θ ₁ ), (E′ ₁ , θ′ ₁ )satisfy

$\mathrm{<span\; class="notranslate">\; abs</span>}<span\; class="notranslate">\; (</span>\frac{<span\; class="notranslate">\; |</span>{{\mathrm{<span\; class="notranslate">\; E.</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; /</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span>}{<span\; class="notranslate">\; |</span>{{\mathrm{<span\; class="notranslate">\; E.</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; /</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1.0</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; \&epsiv;</span>}$ $\mathrm{<span\; class="notranslate">\; abs</span>}<span\; class="notranslate">\; (</span>\frac{{{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}^{<span\; class="notranslate">\; \&prime;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1.0</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; \&epsiv;</span>}$

Where ε is the error control factor, a positive number close to 0, (E ₂ , θ ₂ ), (E′ ₂ , θ′ ₂ ) are (E ₁ , θ ₁ ), (E′ ₁ , θ′ ₁ ) The next adjacent corner code, the present invention just stipulates (E ₁ , θ ₁ ), the comparison operation of (E' ₁ , θ' ₁ ) does not need to add operation, and the physical meaning that does not need to add operation is to represent by E ₁ , The triangle formed by θ ₁ , E ₂ is similar to the triangle formed by E′ ₁ , θ′ ₁ , E′ ₂ .

The similarity calculation of the second step is specifically:

(1) Establish a search space, extract the K nearest neighbors of feature points and construct the ring-type corner codes of feature points, the search space is n ₂ K corner codes on the ring-type corner codes of all feature points in the target point set, for These corner codes are indexed by angle to speed up searches. The schematic diagram is shown in the middle and right two-column tables of attached drawing 4.

(2) Determine the search object, and the purpose of matching is to find the matching point. Therefore, each corner code associated with the ring-type corner code with a single search object of P _i , as far as P _i is concerned, there are K search objects (E _k , θ _k ), as shown in the left table of Figure 4 .

(3) To determine the search strategy, the present invention first uses binary search for (E _k , θ _k ) to search for corner codes (E _n , θ _n ) with the same angle within the allowable error range in the search space, and the corresponding feature points are _Qj . If found, the present invention will also linearly search for other (E _n , θ _n ) with the same angle within the error tolerance range before and after the found corner code. If the comparison result of (E _k , θ _{k ) and} ( _{E n} , θ _{n )} does not need an addition operation, two equivalent Ring type corner code L _P = {A ₀ , A ₁ , ..., A _(K-1) } and L _Q = {B ₀ , B ₁ , ..., B _(K-1) }, where A _k , B _n represent the corner codes, write down E _k ^A , θ _k ^A , E _n ^B , θ _n ^B represent the sides and angles of A _k , B _n respectively. Then calculate their maximum similarity ring type corner code according to incremental matching algorithm. Incremental matching algorithm is as follows: first, compare A ₀ and B ₀ , if θ ₀ ^A , θ ₀ ^B are not equal, add the corner code with the smaller angle to the next adjacent corner code to generate a new Side angle code; if θ ₀ ^A , θ ₀ ^B are equal, verify whether the corresponding sides ( _{E 0 A} ^, _{E 0 B} ^{) and} (E ₁ ^A , E ₁ B ) adjacent to θ 0 ^A , θ 0 ^B are proportional , if proportional, keep A ₀ and B ₀ respectively in the obtained similar ring-type corner codes (empty at the beginning _{) ;} The next corner code is added to generate a new corner code, and then the next comparison operation is performed in a loop until one of the circular corner codes has been traversed. The pseudo code is expressed as follows:

Suppose t ₁ and t ₂ are two loop variables, t ₁ =0, t ₂ =0. Let temp _A ＝A _k , temp _B ＝B _n , rate＝E _n ^B /E _k ^A , then the incremental matching algorithm is as follows:

$\mathrm{<span\; class="notranslate">\; while</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="t"\; filenames="H2009100524538E00012.png,H2009100524538E00013.png"\; state="\{\{state\}\}">t</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="Kand\; t"\; filenames="H2009100524538E00012.png,H2009100524538E00013.png"\; state="\{\{state\}\}">Kand</figure-callout></span><figure-callout\; id="2"\; label="Kand\; t"\; filenames="H2009100524538E00012.png,H2009100524538E00013.png"\; state="\{\{state\}\}">\; </figure-callout>}{\mathrm{<span\; class="notranslate">\; </span>}}_{}{\mathrm{<span\; class="notranslate">t</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \{</span>$

$\mathrm{<span\; class="notranslate">\; if</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; A</span>}}}<span\; class="notranslate">\; <</span>{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="t"\; filenames="H2009100524538E00012.png,H2009100524538E00013.png"\; state="\{\{state\}\}">t</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="t"\; filenames="H2009100524538E00012.png,H2009100524538E00013.png"\; state="\{\{state\}\}">t</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; A</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; A</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; .</span><span\; class="notranslate">\; \}</span>$

$\mathrm{<span\; class="notranslate">\; else\; if</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; A</span>}}}<span\; class="notranslate">\; ></span>{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; tem</span>}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="t"\; filenames="H2009100524538E00012.png,H2009100524538E00013.png"\; state="\{\{state\}\}">t</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="t"\; filenames="H2009100524538E00012.png,H2009100524538E00013.png"\; state="\{\{state\}\}">t</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; .</span><span\; class="notranslate">\; \}</span>$

$\mathrm{<span\; class="notranslate">\; else</span>}<span\; class="notranslate">\; \{</span>$

${\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="t"\; filenames="H2009100524538E00012.png,H2009100524538E00013.png"\; state="\{\{state\}\}">t</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="t"\; filenames="H2009100524538E00012.png,H2009100524538E00013.png"\; state="\{\{state\}\}">t</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>{{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="t"\; filenames="H2009100524538E00012.png,H2009100524538E00013.png"\; state="\{\{state\}\}">t</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="t"\; filenames="H2009100524538E00012.png,H2009100524538E00013.png"\; state="\{\{state\}\}">t</figure-callout></span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; .</span>$

$\mathrm{<span\; class="notranslate">\; if</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; rate</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; =</span>{{\mathrm{<span\; class="notranslate">\; E.</span>}}^{\mathrm{<span\; class="notranslate">\; B</span>}}}_{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; /</span>{{\mathrm{<span\; class="notranslate">\; E.</span>}}^{\mathrm{<span\; class="notranslate">\; A</span>}}}_{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>$

$<span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; add</span>}{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; A</span>}}\mathrm{<span\; class="notranslate">\; into</span>}{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; add</span>}{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}\mathrm{<span\; class="notranslate">\; into</span>}{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; tem</span>}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; A</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; .</span><span\; class="notranslate">\; \}</span>$

$\mathrm{<span\; class="notranslate">\; else</span>}<span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; A</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; A</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; temp</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; t</span>}\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; .</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; \}</span>$

In the search process, only the similar ring-type corner codes with the largest similar length are kept, and when there are multiple ring-type corner codes with the largest similar length, all of them are kept. Take the similarity length of the maximum similarity ring-type corner code as the similarity between two feature points, and note that the similarity of all two feature points that have not been calculated in the search process is 0. the

(4) Determining the feature point of possible matching, the present invention defines S(A _k , B _n )={L _k , L _n } to represent the similar ring-type corner code with E _k ^A , E _n ^B as the starting corresponding side , L _k , L _n represent similar circular corner codes. Let the similarity length of S(A _k , B _n ) be len ^ij (k, n), then the similarity of feature points P _i , Q _j is:

$\mathrm{<span\; class="notranslate">\; similarity</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ]</span>}{\mathrm{<span\; class="notranslate">\; max</span>}}{\mathrm{<span\; class="notranslate">\; len</span>}}^{\mathrm{<span\; class="notranslate">\; ij</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>$

If _Qj ^* satisfies

${{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arg</span>}{\mathrm{<span\; class="notranslate">\; max</span>}}_{{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; T</span>}}\mathrm{<span\; class="notranslate">\; similarity</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span>$

The present invention considers that Q _j ^* is the possible matching point of P _i , and during the calculation process, P _i may have multiple possible matching Q _j ^* , all of which are reserved.

Step 3 structure matching of the present invention is specifically:

Assuming that there are NO Q _j ^* that P _i may match, the possible matching points (P _i , Q ^* _j ) are classified as follows:

$<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{{\mathrm{<span\; class="notranslate">\; Q</span>}}^{<span\; class="notranslate">\; *</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>\left\{\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; badClass</span>}& \mathrm{<span\; class="notranslate">\; if</span>}& \mathrm{<span\; class="notranslate">\; similarity</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{{\mathrm{<span\; class="notranslate">\; Q</span>}}^{<span\; class="notranslate">\; *</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; 3</span>}\\ \mathrm{<span\; class="notranslate">\; unknownClass</span>}& \mathrm{<span\; class="notranslate">\; if</span>}& \mathrm{<span\; class="notranslate">\; similarity</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{{\mathrm{<span\; class="notranslate">\; Q</span>}}^{<span\; class="notranslate">\; *</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Greater\; Equal;</span>\mathrm{<span\; class="notranslate">\; 3</span>}\mathrm{<span\; class="notranslate">\; andNO</span>}<span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; 1</span>}\\ \mathrm{<span\; class="notranslate">\; goodClass</span>}& \mathrm{<span\; class="notranslate">\; if</span>}& \mathrm{<span\; class="notranslate">\; similarity</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{{\mathrm{<span\; class="notranslate">\; Q</span>}}^{<span\; class="notranslate">\; *</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Greater\; Equal;</span>\mathrm{<span\; class="notranslate">\; 3</span>}\mathrm{<span\; class="notranslate">\; andNO</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right.$

The classification conditions are explained as follows: the similarity of (P _i , Q ^* _j ) in badClass is less than 3. When K is relatively large, such as K=15, it shows that the local spatial structure of (P _i , Q ^* _j ) is very different. Large, P _i or Q ^* _j may be a noise point, and the possibility of matching is very small; the similarity of (P _i , Q ^* _j ) in unknownClass is greater than or equal to 3, when K is relatively large, such as K=15 , indicating that the local spatial structure of (P _i , Q ^* _j ) is relatively similar, but there are more than one Q ^* _j that P _i may match, which shows that it is impossible to determine which Q ^* _j matches P _i only through the similarity of local spatial structure , need follow-up verification to confirm; the similarity of (P _i , Q ^* _j ) in goodClass is greater than or equal to 3, when K is relatively large, such as K=15, it shows that the local space of (P _i , Q ^* _j ) The structure is relatively similar, and there is only one Q ^* _j that P _i may match, the possibility of (P _i , Q ^* _j ) matching is very high, and the greater the similarity, the more similar the local space structure is, the more likely it is to match big.

According to the above analysis, the present invention regards (P _i , Q ^* _j ) in goodClass as the result of structural matching, and at the same time, the present invention also performs local similarity transformation estimation on (P _i , Q ^* _j ) in goodClass, and pays attention to local In the process of similarity transformation estimation, (P _i , Q ^* _j ) are combined with a set of corresponding neighbor points associated with them to estimate a set of local similarity transformations. If the similarity of (P _i , Q ^* _j ) is ls, then Estimate the local similarity transformation of the ls group, and the matching algorithm takes the mean value of the local similarity transformation of the ls group as the final estimated local similarity transformation.

In step 4, the optimization matching is specifically as follows:

(1) Sort (P _i , Q ^* _j ) in goodClass according to the similarity from large to small;

(2) Scan the sorted (P _i , Q ^* _j ), if (P _i , Q ^* _j ) has not been clustered, then take (P _i , Q ^* _j ) as the cluster center C _i , The local map T _i associated with (P _i , Q ^* _j ) is used as a classifier to verify the unclustered possible matching points and associated corresponding neighbor points in goodClass and unknownClass. The clustering process is as follows: First, the corresponding neighbor points associated with (P _i , Q ^* _j ) are classified into this class, because the classifier T _i is estimated from these corresponding neighbor points, and then the classifier is used to verify other goodClass and unknownClass For unclustered possible matching points and associated corresponding neighbor points, if the verified point pair satisfies the classifier, it will be marked, no further processing will be performed and it will be classified into this class, otherwise it will be ignored. Let the pair of points that need to be verified be (P, Q), let the point after P is mapped by T _i be T _i (P), if the Euclidean distance between T _i (P) and Q is acceptable within the error range, The present invention considers that (P, Q) conforms to T _i , and classifies (P, Q) as C _i . Expressed as:

|T _i (P), Q|<ε _T

ε _T is the error control factor. Note that each pair (P _i , Q ^* _j ) is only clustered once during the clustering process, that is, (P _i , Q ^* _j ) belongs to and can only belong to one class. A clustering process must verify all other possible matching points and associated corresponding neighbor points that have not been clustered. The starting condition for the next clustering is that there are (P _i , Q ^* _j ) that have not been clustered in goodClass.

(3) Analyze the clustering results, because there are usually wrong matches in the structure matching results, which must be deleted. In the clustering results, the class with the most elements is defined as the largest class, the class with the second most elements is the second largest class, and each element in a class represents a pair of matching points. Usually, the local mapping associated with the correctly matched feature points is also correct, and all the correct matched points will be grouped into one class within the allowable range of error, and there will be a lot of point pairs in the corresponding class; The associated local maps are very random, and the corresponding classes have very few elements. Therefore, the present invention optimizes matching as follows: if the ratio of the number of elements of the largest category to the second largest category is greater than a certain threshold, or the number of elements of the largest category is greater than the second largest category, and at the same time the number of elements of the largest category is greater than a certain threshold, the present invention The largest class will be output as the corresponding relationship for optimal transformation estimation, otherwise the present invention considers that the correlation between the two matched point sets is too poor to be matched. the

Description of drawings

Figure 1: Flowchart of the present invention. the

Figure 2: The KNN space structure diagram of the feature point P _c and the corresponding ring corner code.

Figure 3: Similar ring-type corner codes and corresponding similar spatial structures. the

Figure 4: Schematic diagram of similarity calculation. the

Figure 5: Two images to be matched. the

Figure 6: Matching results for the two images shown in Figure 5. the

Detailed ways

Specific examples are provided below to further illustrate the application of the present invention. the

Example 1

The matching objects are shown in the two images shown in Figure 5. The two images were taken at different times and from different angles, only part of the content is the same, and the image size is 440×330. the

The equipment running the present invention is Gateway T6307c notebook, Intel 2core 1.6G, 1G memory. The present invention has platform portability, and concrete realization platform is the Visual C++2005 environment under the GCC environment under Centos and Windows XP/Server 2003. The operating efficiency of GCC is better than that of Visual C++2005. the

The present invention stipulates that the error ε=0.1 when two scalar quantities are equal, and the Euclidean distance error ε _T =10 between two pixel points, and the definition of the two errors aims at calculating similar circular corner codes and clustering. At the same time, the present invention sets the number of neighbors of feature points as K=15.

The specific implementation is as follows:

Step 1: Obtain two feature point sets from two images to be matched as input, and the feature points only need to include two-dimensional coordinate information. The present invention adopts the single-scale Harris corner detection algorithm, Harris response coefficient α=0.06, difference scale σ _D =3, smooth scale σ _I =7, local non-maximum suppression window is 3×3, and the suppression value is the global maximum 0.01 times the response value of the corner point. The two images extract 189 and 208 feature points respectively. The corresponding point sets are denoted as S and T respectively, n ₁ =189 and n ₂ =208.

Step 2: Calculate the similarity, extract 15 neighbors of each feature point, and calculate the ring-type corner code of the feature point according to the definition of the ring-type corner code. During calculations, division by zero should be avoided. When doing scalar division, add or subtract a very small positive number as needed to avoid program crashes. the

According to the definition of similar circular corner codes, calculate the similarity of the two feature points to be matched, determine the elements in goodClass and unknownClass according to the classification rules, and calculate the local mapping associated with the elements in goodClass. Specific steps are as follows:

(1) Establish a search space. The 208×15 corner codes (E n , θ n ) of the circular corner codes (E _n , θ _n ) of 208 feature points in T are sorted according to the angle size to establish an index.

(2) Determine the search object. In order to obtain the point Q ^* _j that P _i may match, it is necessary to have 15 corner codes (E _k , θ _k ) (k=0, 1, ..., 15) on the circular corner code of P _i respectively to search.

(3) Determine the search strategy. Select each (E _k , θ _k ) in turn, search in the search space according to the angle, if the search is successful, then continue to search linearly before and after the searched corner code (E _n , θ _n ) and have the same angle within the error range For other corner codes (E _n , θ _n ), if the comparison result of (E _k , θ _k ) and (E _n , θ _n ) does not need to be added, then use (E _k , θ _k ) and (E _n , θ _n ) as the starting point to generate two equivalent circular corner codes and use the incremental matching algorithm to calculate their maximum similarity circular corner codes.

(4) Determine the possible matching feature points. After all the 15 (E _k , θ _k ) of P _i have been searched, the similar ring-type corner code with the largest similarity length in the calculation process is retained, and the retained maximum similar ring-type edge Another feature point Q _j associated with the corner code is used as the point Q ^* _j that P _i may match, and if there are more than one, all are saved.

Step 3: Structure matching, classify the 189 feature points in S according to the similarity and the number of possible matching feature points. Keep goodClass and unknownClass. And calculate the local mapping of element association in goodClass according to the similarity transformation model. The similar transformation model is as follows:

X＝sxcosθ-sysinθ+t _x

Y=sxsinθ+sycosθ+t _y

Where s, θ are scaling factor and rotation angle respectively, t _x , t _y are translation amounts in x and y directions respectively. (x, y) and (X, Y) are the coordinates of feature points before and after transformation respectively. Note that in the calculation process of local similarity transformation, a group of transformation coefficients can be estimated by matching points and their corresponding neighbor points, so if the similarity of P _i and Q ^* _j is ls, then the ls group of local similarity transformation can be estimated, The present invention takes the mean value of the local similarity transformation of the ls group as the final estimated local similarity transformation.

Step 4: Optimize the matching, and sort the elements in goodClass in step 3 according to the similar length from large to small. Because the larger the similarity length, the more similar the local spatial structure of the two feature points is, and the greater the possibility of matching is, starting clustering from P _i and Q ^* _j with the largest similarity length can ensure better clustering effect , the better the matching effect is. Therefore, clustering first selects a pair of feature points with the largest similarity that has not been clustered. Then cluster according to the clustering rules. For the splicing and recognition applications of large-scale scene images, let s ₁ and s ₂ be the number of elements of the largest category and the second largest category, if s ₁ ≥ 2s ₂ or s ₁ >s ₂ , s ₁ >23, the algorithm The element in the largest class will be output as the optimal matching result for optimal transformation estimation. Otherwise, the algorithm considers the match unsuccessful. For the example in Fig. 5, there are 88 clustering results in total, of which the largest number of elements is 33, and the second largest number of elements is 6. By observation, the elements of the largest category are all correctly matched within the error range, and the elements of the second largest category are all incorrectly matched within the error range. In the example shown in Figure 5, the ratio of the number of elements of the largest category to the next largest category is 5.5, which satisfies the condition and the matching is successful. The elements of the largest class will be used as correspondences for optimal transformation estimation.

Step 5: Transformation estimation, because usually, the transformation between scene images often has a certain degree of affine or perspective transformation, so the present invention uses an affine transformation model to estimate the transformation between scene images. The affine transformation model is as follows:

X＝ax+by+t _x

Y＝cx+dy+t _y

Where a, b, c, d are the rotation, scaling and stretching factors respectively, t _x , t _y are the translation amounts in the x and y directions respectively. (x, y) and (X, Y) are the coordinates of feature points before and after transformation respectively. For all matching points, a system of linear equations is formed according to the affine transformation model, and the least square fitting estimation is performed to obtain the optimal transformation. For the example of accompanying drawing 5, optimal transformation is $\left[\begin{array}{ccc}1.04237& 0.00366& -333.03\\ 0.01088& 1.02362& -10.1731\end{array}\right],$ Apply the optimal transformation to the matching image to obtain the matching result, see Figure 6. The results show that the present invention stitches together two images with only part of the same content within the error range.

Claims (2)

1. An image registration method based on feature points is characterized by comprising the following steps:

step 1): the characteristic extraction is carried out on the basis of the characteristic,

extracting feature points of two images to be registered by using a feature point extraction operator, and defining two obtained feature point sets as input data of the method;

step 2): the calculation of the degree of similarity is carried out,

establishing a search space, defining the two feature point sets as S and T, and defining the number of the included points as n₁And n₂Extracting K neighbor structures of the characteristic points and constructing ring-type corner codes of the characteristic points; the search space is defined as n corresponding to the point set T₂N on individual ring type corner code₂K corner pieces, to n₂K corner codes are sorted according to the size of the angle to establish an index; the K neighbor structure of the feature points is represented by a ring-type corner code, the maximum similar ring-type corner code between the feature points is calculated by utilizing a binary search and incremental matching algorithm, the similarity length of the maximum similar ring-type corner code is defined as the similarity between the two feature points, and the feature point pairs which are possibly matched are determined while the similarity is calculated;

the ring type corner connector is characterized in that: respectively connecting the characteristic points with K neighbor points in a K neighbor structure of the characteristic points through one edge; defining the combination of each edge and its counter-clockwise adjoining corner as an edge code EAC ═ E, θ, where there are K edge codes (Ei, θ i), i ═ 0, 1, …, K-1; coding the K corner codes according to a counterclockwise sequence, sequentially connecting the K corner codes into an annular structure to obtain the annular corner codes of the characteristic points, and defining K as the length of the annular corner codes; the two ring-type edge corner codes start from a certain pair of edges and meet the condition that the corresponding edges are in proportion, the corresponding angles are in proportion and equal to 1, the two ring-type edge corner codes are defined to be similar, and the length of one ring-type edge corner code is defined as the similar length of the similar ring-type edge corner code;

step 3): the structure of the utility model is matched with that of the utility model,

classifying the feature point pairs according to the similarity and the possible matching point number, and performing local mapping estimation to order (P)_i，Q^* _j) Representing a characteristic point pair, wherein the classification condition of the characteristic point pair is defined as:

<math> <mrow> <mrow> <mo>(</mo> <msub> <mi>P</mi> <mi>i</mi> </msub> <mo>,</mo> <msub> <msup> <mi>Q</mi> <mo>*</mo> </msup> <mi>j</mi> </msub> <mo>)</mo> </mrow> <mo>&Element;</mo> <mfenced open='{' close=''> <mtable> <mtr> <mtd> <mi>badClass</mi> </mtd> <mtd> <mi>if</mi> </mtd> <mtd> <mi>similarity</mi> <mrow> <mo>(</mo> <msub> <mi>P</mi> <mi>i</mi> </msub> <mo>,</mo> <msub> <msup> <mi>Q</mi> <mo>*</mo> </msup> <mi>j</mi> </msub> <mo>)</mo> </mrow> <mo>&lt;</mo> <mn>2</mn> </mtd> </mtr> <mtr> <mtd> <mi>unknownClass</mi> </mtd> <mtd> <mi>if</mi> </mtd> <mtd> <mi>similarity</mi> <mrow> <mo>(</mo> <msub> <mi>P</mi> <mi>i</mi> </msub> <mo>,</mo> <msub> <msup> <mi>Q</mi> <mo>*</mo> </msup> <mi>j</mi> </msub> <mo>)</mo> </mrow> <mo>&GreaterEqual;</mo> <mn>3</mn> <mi>and NO</mi> <mo>></mo> <mn>1</mn> </mtd> </mtr> <mtr> <mtd> <mi>goodClass</mi> </mtd> <mtd> <mi>if</mi> </mtd> <mtd> <mi>similarity</mi> <mrow> <mo>(</mo> <msub> <mi>P</mi> <mi>i</mi> </msub> <mo>,</mo> <msub> <msup> <mi>Q</mi> <mo>*</mo> </msup> <mi>j</mi> </msub> <mo>)</mo> </mrow> <mo>&GreaterEqual;</mo> <mn>3</mn> <mi>and NO</mi> <mo>=</mo> <mn>1</mn> </mtd> </mtr> </mtable> </mfenced> </mrow> </math>

in goodClass (P)_i，Q^* _j) Defined as the result of structure matching, for (P) in goodClass_i，Q^* _j) Local mapping estimation using a similarity transformation model, definition (P)_i，Q^* _j) Has a similarity of ls, (P)_i，Q^* _j) Combining the local similarity transformation of the ls group with each pair of neighbor points to estimate the local similarity transformation of the ls group, and defining the mean value of the local similarity transformation of the ls group as the final estimated local mapping; wherein Q^* _jIs P_iPossible matching points of (a); said NO is Q_j ^*The number of (2); similarity (P)_i，Q^* _j) Means that Q^* _jAnd P_iThe similarity of (2);

step 4): the optimization and matching are carried out, and the optimization and matching are carried out,

clustering is carried out by utilizing the structure matching result, and the corresponding relation of the characteristic points is determined by the following steps:

(a) for (P) in goodClass_i，Q^* _j) Sorting according to similarity from big to small;

(b) for ordered (P)_i，Q^* _j) Scanning in descending order according to the size of the similarity (P)_i，Q^* _j) Not clustered, then (P)_i，Q^* _j) Is a class center C_i，(P_i，Q^* _j) Associated partial map T_iAs a classifier, verifying the unclustered feature point pairs and the associated corresponding neighbor points in the goodClass and the unknown class;

firstly, the first step is to mix with (P)_i，Q^* _j) Directly classifying associated ls pairs into the class, verifying non-clustered possible matched pairs and associated matched neighbor pairs in other goodclasss and unknown classes by using a classifier, and satisfying the classifier T when the spatial coordinate relationship of the verified pairs meets the requirement of the classifier T in an error range_iIf so, marking the data, classifying the data into the category without subsequent processing, otherwise, ignoring the data;

(c) the result of the clustering is analyzed and,

in the clustering result, the class with the most elements is defined as the maximum class, and the class with the second most elements is the second largest class; the maximum number of class elements is defined as S₁The number of the next largest class element is defined as S₂When one of the following conditions is satisfied, the maximum class is used as the feature point correspondence, the optimal transformation is estimated,

①S₁＞t₁S₂，②S₁＞S₂at the same time S₁＞t₂，

Wherein t is₁、t₂Is a specific threshold value larger than 1, which is determined by the kind of the image to be matched;

step 5): the optimal transformation estimate is estimated by taking into account the transformation,

and performing least square estimation on the mapping between the two images by using an affine transformation model.

2. The feature point-based image registration method according to claim 1, wherein the similarity calculation is performed by:

(1) establishing a search space, defining the two feature point sets as S and T, and defining the number of the included points as n₁And n₂While defining P_i∈S、Q_jE, T, extracting K neighbor of the characteristic point and constructing a ring type edge code of the characteristic point according to definition; the search space is defined as n corresponding to the point set T₂N on individual ring type corner code₂K corner pieces, to n₂K corner codes are sorted according to the size of the angle to establish an index;

(2) determining a search object, a single search object being defined as P_iK search objects are searched for each edge code on the ring-type edge codes;

(3) determining a search strategy, defining (E)_k，θ_k) Is P_iUsing binary search and linear search to search all corner codes (E) with the same angle in the error tolerance range in the search space_n，θ_n) Definition of (E)_n，θ_n) The characteristic point of membership is Q_j；

(4) Computing using an incremental matching algorithmSimilar ring type corner connector, will (E)_k，θ_k) And each (E)_n，θ_n) Making a comparison when (E)_k，θ_k) And (E)_n，θ_n) When the comparison operation of (A) is true, respectively (E)_k，θ_k) And (E)_n，θ_n) Generating P as a starting point_iAnd Q_jTwo ring type corner connectors L_P＝{A₀，A₁，...，A_(K-1)And L_Q＝{B₀，B₁，...，B_(K-1)In which A is_k∈L_pAnd B_n∈L_QDenotes corner code, note E_k ^A，θ_k ^A，E_n ^B，θ_n ^BRespectively represent A_k，B_nThe edges and corners of (a);

first compare A₀，B₀If theta is greater than theta₀ ^A，θ₀ ^BIf the angle codes are not equal, the edge code with the small angle and the next adjacent edge code are subjected to addition operation to generate a new edge code; if theta is greater than theta₀ ^A，θ₀ ^BIf equal, verify θ₀ ^A，θ₀ ^BAdjacent corresponding edge (E)₀ ^A，E₀ ^B) And (E)₁ ^A，E₁ ^B) Whether or not to scale, if so, then A₀，B₀Keeping the same in the obtained similar ring-type corner codes, if not in proportion, adding A to the obtained similar ring-type corner codes₀，B₀Adding operation is carried out on the new corner codes and the adjacent next corner codes to generate new corner codes, then next comparison operation is carried out in a circulating mode, and the circulating process is carried out until one of the ring-type corner codes is traversed;

(5) determining possible matching points, defining S (A)_k，B_n)＝{L_k，L_nDenotes by E_k ^A，E_n ^BSimilar ring-type corner connectors, L, for the starting corresponding edge_k，L_nRepresenting similar ring type corner codes, let S (A)_k，B_n) Of similar length len^ij(k, n), characteristic point P_i，Q_jPhase ofThe similarity is defined as:

<math> <mrow> <mi>similarity</mi> <mrow> <mo>(</mo> <msub> <mi>P</mi> <mi>i</mi> </msub> <mo>,</mo> <msub> <mi>Q</mi> <mi>j</mi> </msub> <mo>)</mo> </mrow> <mo>=</mo> <munder> <mi>max</mi> <mrow> <mi>k</mi> <mo>,</mo> <mi>n</mi> <mo>&Element;</mo> <mo>[</mo> <mn>0</mn> <mo>,</mo> <mi>K</mi> <mo>-</mo> <mn>1</mn> <mo>]</mo> </mrow> </munder> <msup> <mi>len</mi> <mi>ij</mi> </msup> <mrow> <mo>(</mo> <mi>k</mi> <mo>,</mo> <mi>n</mi> <mo>)</mo> </mrow> </mrow> </math>

when Q is_j ^*Satisfy the requirement of <math> <mrow> <msup> <msub> <mi>Q</mi> <mi>j</mi> </msub> <mo>*</mo> </msup> <mo>=</mo> <msub> <mrow> <mi>arg</mi> <mi>max</mi> </mrow> <mrow> <msub> <mi>Q</mi> <mi>j</mi> </msub> <mo>&Element;</mo> <mi>T</mi> </mrow> </msub> <mi>similarity</mi> <mrow> <mo>(</mo> <msub> <mi>P</mi> <mi>i</mi> </msub> <mo>,</mo> <msub> <mi>Q</mi> <mi>j</mi> </msub> <mo>)</mo> </mrow> </mrow> </math> When, define Q_j ^*Is P_iPossible matching points of, Q_j ^*The number of (2) is defined as NO.

Images