KR20170066014A

KR20170066014A - A feature matching method which is robust to the viewpoint change

Info

Publication number: KR20170066014A
Application number: KR1020150172504A
Authority: KR
Inventors: 정현조; 유지상
Original assignee: 광운대학교 산학협력단
Priority date: 2015-12-04
Filing date: 2015-12-04
Publication date: 2017-06-14
Also published as: KR101753360B1

Abstract

A method for matching a feature point that is robust against a viewpoint change that classifies feature points required for building recognition using one or more multi-frames, the method comprising: (a) dividing the multi-frame image into blocks; (b) acquiring a plurality of feature point pairs by matching feature points extracted using a plurality of frames; (c) obtaining a homography matrix from a plurality of pairs of feature points and classifying the feature points necessary for the building; And (d) repeating classification using homography using the remaining pair of feature points.
By using the feature point classification method as described above, it is possible to significantly improve the accuracy of the feature point matching by using the point where the feature points of the background appear only in the limited image by searching for the association of the feature points using the RANSAC and finding and classifying the feature points repeatedly.

Description

[0001] The present invention relates to a feature point matching method robust to a viewpoint change,

The present invention relates to a feature point matching method that is robust against a viewpoint change in which feature points are matched by using feature point Accelerated Segment Test (FAST) feature point detector and SIFT (Scale Invariant Feature Transform) feature point descriptor.

In particular, the present invention extracts feature points using the FAST method, improves by applying principal curvatures, describes the extracted feature points through a SIFT descriptor, and assigns RANSAC The homography is calculated through the SAmple Consensus method and the matching points are classified through the Euclidean distance between the coordinate of the minutiae of the reference image and the coordinate of the minutiae of the image of different viewpoint by homography transformation , And a feature point matching method robust to a viewpoint change.

Generally, in the field of computer vision, the feature point matching method of images is widely used in various fields such as motion detection, face recognition, 3D image restoration, panoramic stitching, object recognition, and stereo similarity measurement. The feature point matching method includes a process of extracting feature points using a feature point extraction algorithm in the image, a process of describing the extracted feature points using feature descriptors, and finally, comparing feature points described in different images, And connecting and disconnecting incorrectly connected minutiae to separate correctly connected minutiae. The performance of the feature point matching method depends on the detection of unique feature points in the image and on the way in which the descriptors are generated for each feature point.

The Harris corner detector, which is an initial feature point detection method, has a drawback that it is vulnerable to image scale conversion by finding feature points of an image using eigenvalues of a Hessian matrix [Non-Patent Document 4]. The Laplacian of Gaussian proposed by Lindeberg is robust to the size change by obtaining the maximum point of Laplacian in the scale space [Non-Patent Document 1]. Lowe detects the feature points of the image by approximating the Laplacian of Gaussian with the Gaussian Difference of Gaussian and expresses the feature points by the histogram of the gradient direction around the feature points. (Schematic Invariant Feature Transform) method for performing a SIFT (Non-Patent Document 1). Mikolajczyk proposed a method of modifying the Harris corner detector to detect feature points that take Laplacian-based scale changes into account and generate a descriptor with a GLOH (Gradient Location and Orientation Histogram) modifying the SIFT feature point descriptor [ Patent Documents 2 and 3].

Bay proposed a SURF (Speeded Up Robust Feature) method which shows excellent matching performance while reducing the amount of computation [Non-Patent Document 6]. SURF detects the feature points using the maximum points of the Hashian matrix and generates the feature point descriptor in the Haar response, thereby performing the image matching. The integration method is used to greatly reduce the calculation amount [Non-Patent Document 7].

However, the SIFT method is robust to the illumination change of the image, the image size, and the rotation change, but has a problem that it can not be used in real time due to a large amount of calculation. The SURF method has a disadvantage in that the matching rate is lowered for an image whose viewpoint has changed with respect to the same object or place.

[Non-Patent Document 1] D. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints", International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, Nov. 2004. [Non-Patent Document 2] K. Mikolajczyk, " Scale & Affine Invariant Interest Point Detectors ", International Journal of Computer Vision, vol. 60, no. 1, pp. 63-86, Oct. 2004. [Non-Patent Document 3] K. Mikolajczyk, "A Performance Evaluation of Local Descriptors ", Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1615-1630, Oct. 2005. [Non-Patent Document 4] K. Midolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. Van Gool. "A Comparison of Affine Region Detectors ", International Journal of Computer Vision, vol. 65, no. 1-2, pp. 43-72, Nov. 2005. [Non-Patent Document 5] E. Rosten and T. Drummond, "Machine Learning for High-speed Corner Detection ", 9th European Conference on Computer Vision, Graz, Austria, pp. 430-443, May 2006. [Non-Patent Document 6] E. Rosten, "Faster and Better: A Machine Learning Approach to Corner Detection ", Pattern Analysis and Machine Intelligence, vol. 32, no. 1, pp. 105-119 Jan. 2010. [Non-Patent Document 7] H. Bay, A. Ess, T. Tuytelaars and L. V. Gool, "Speeded-up Robust Feature", Computer Vision and Image Understanding, vol. 10, no. 3, pp. 346-359, Jun. 2008. [Non-Patent Document 8] M. A. Fischler and R. C. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography", Communications of the ACM, vol. 24, no. 6, pp. 381-395, Jun. 1981. [Non-Patent Document 9] D. Comaniciu, V. Ramesh and P. Meer, "Real-time Tracking of Non-rigid Objects using Mean Shift", Proc. 2000 IEEE Conference on Computer Vision and Patter Recognition, vol. 2, pp. 142-149, Jun. 2000. [Non-Patent Document 10] http://www.robots.ox.ac.uk/~vgg/research/affine/ [Non-Patent Document 11] http://www.vision.caltech.edu/pmoreels/Datasets/ Home_Objects_06 / Test / [Non-Patent Document 12] M. M. Hossain, H. J. Lee and J. S. Lee, "Fast image stitching for video stabilization using sift feature points", The Korean Institute of Communications and Information Sciences, vol. 39, no. 10, pp. 957-966, Oct. 2014. [Non-Patent Document 13] B. W. Chung, K. Y. Park and S. Y. Hwang, "A Fast and Efficient Haar-like Feature Selection Algorithm for Object Detection", The Korean Institute of Communications and Information Sciences, vol. 38, no. 6, pp. 486-497, Jun. 2013. [Non-Patent Document 14] H. K. Jang, "The more environmentally robust edge detection of moving objects using improved Canny edge detector and Freeman chain code ", The Korean Institute of Communications and Information Sciences, vol. 37, no. 2, pp. 37-42, Apr. 2012.

SUMMARY OF THE INVENTION The object of the present invention is to solve the above problems, and it is an object of the present invention to extract feature points by the FAST method, to improve principal curvatures and to extract extracted feature points through a SIFT descriptor, Homography is calculated by RANSAC (Random Absolute Consensus) method on the matching pairs obtained from the two images, and the feature points of the reference image are transformed by homography transformation to the Euclidean The present invention provides a feature point matching method that is robust against a viewpoint change in which matching pairs are classified through an Euclidean distance.

In particular, it is an object of the present invention to provide a method of extracting feature points using the FAST method, and applying principal curvatures. That is, the present invention applies a FAST (Feature from Accelerated Segment Test) method (Non-Patent Document 5) that extracts feature points by minimizing a calculation amount by simply comparing pixel values to extract feature points quickly, In order to overcome the disadvantage of unnecessarily extracting a large number of feature points along the edge portion of the edge, a method of improving principal curvatures is provided. The FAST method is much better in terms of the speed of extracting feature points than the DOG method of SIFT or the Haar wavelet method of SURF.

In addition, the present invention uses a SIFT descriptor that can describe more information (128 dimensions) for extracted minutiae. Matching pairs obtained from two images with different viewpoints may cause not only a correct matching pair but also an error matching pair. If the distribution of the incorrect matching pair occupies a large part in the entire data, the accuracy of the matching of the characteristic point is degraded. Therefore, homography calculation is performed with four correctly matched pairs of feature points using RANSAC (RANOM SAmple Consensus) proposed by Fischler and Bolles in order to remove error matching pairs in the obtained matching pairs [Non-Patent Documents 8 and 9 ]. For the feature point matching robust to the viewpoint change, the Euclidean distance between the coordinates of the modified point and the minutiae points of the different viewpoints is calculated by using the homography matrix and the reference points of the reference image obtained through RANSAC, The distances are used to classify matching pairs by comparing them with a threshold.

According to an aspect of the present invention, there is provided a feature point matching method robust to a viewpoint change, the feature point matching method comprising: (a) extracting feature points from the image using a feature from Accelerated Segment Test (FAST); (b) obtaining a Hessian matrix for the extracted feature points, and improving the feature point extraction using the sum of the elements on the main diagonal of the obtained Hessian matrix and the determinant; (c) acquiring a plurality of feature point pairs by matching feature points extracted using a plurality of frames; (d) obtaining a homography matrix from a plurality of pairs of feature points and classifying the feature points necessary for the building; And (e) repeating classification using homography using the remaining pair of feature points.

According to another aspect of the present invention, there is provided a feature point matching method robust to a viewpoint change, wherein in the step (a), when a pixel p is referred to as a center pixel in the image, a circle having a predetermined distance therefrom is formed, When the brightness of the peripheral pixel is compared with the brightness of the central pixel and the number of neighboring pixels having a difference equal to or greater than a preset threshold value is more than N (N is a natural number) It is determined as a minutiae point.

According to another aspect of the present invention, there is provided a feature point matching method robust against a viewpoint change, wherein in the step (a), only a part of peripheral pixels that extend over the formed circle is compared with the brightness of the center pixel, When the number of neighboring pixels is equal to or greater than a predetermined ratio, all of the neighboring pixels other than the predetermined neighboring pixels are also compared to extract feature points. If the number of neighboring pixels is less than a predetermined ratio, .

According to the present invention, there is provided a feature point matching method robust to a viewpoint change, wherein in the step (a), a state of a contrast result between a center pixel and a surrounding pixel is configured in a decision tree structure, The state of all the pixels is initialized to u (unknown), and the brightness of the center pixel is compared with that of the surrounding pixels. The state of the surrounding pixels is determined according to the result, The state of the neighboring pixels is determined once again from the first pixel in the tree and the neighboring pixels are determined in the previous step and the state determination through the brightness contrast between the pixels is not performed .

Further, the present invention is characterized in that in the feature point matching method robust to a viewpoint change, in the step (a), the state of the pixel is determined in four states as shown in the following equation (1).

[Equation 1]

However, S _p _-> _xk are the peripheral pixels constituting the circle indicates the state of the k-th pixel, I _p is the luminance value of the central pixel p, I _p _-> _xk is the brightness value of the surrounding pixels of the center pixel p , t is a threshold value, d is darker than the center pixel, b is brighter than the center pixel, s is similar to the central pixel brightness, u is Indicates an undetermined state (unknown).

In addition, the present invention provides a robust feature point matching method at the time of change, in the step (a), according to the formula 1 and to determine the status of the peripheral pixels, which corresponds to the status b (x∈S _bright) in a decision tree structure A larger value is assigned to the corresponding feature point value by applying the following Equation 2 to the pixel corresponding to the d state (x∈S _dark ), and the feature point value is applied to all the adjacent feature points using Equation 2 And extracting feature points having the largest value as feature points of the cluster region.

[Equation 2]

Where V is the value of the feature point of the center pixel p, I _p _{-> x} and I _p are the brightness values of the surrounding pixels and the center pixel, respectively.

According to another aspect of the present invention, there is provided a feature point matching method robust to a viewpoint change, wherein in the step (b), a 2x2 Hessian matrix is obtained for the feature points extracted in the step (a) ] Is not satisfied, the feature point is excluded.

[Equation 3]

here,

.

,

I _xx is the second-order differential value in the x direction of the feature point pixel, I _yy is the second-order differential value in the y direction, I _xy is the differential value in the x and y directions, and r is a predetermined threshold value.

According to the present invention, there is provided a feature point matching method robust to a viewpoint change, wherein in the step (c), 16 blocks each having a size of 4x4 are displayed in eight directions, Value, and a descriptor is formed in the form of a vector composed of the values of the direction histogram.

In the feature point matching method robust to a viewpoint change, in the step (c), a descriptor is expressed in eight directions for each of 16 blocks for each feature point, and 128 (4x4x8) dimensions The Euclidean distance is calculated as a 128-dimensional vector between the minutiae extracted from the two images to be matched. When the distance value is smaller than the threshold value, the matched minutiae pairs are selected.

In addition, the present invention is characterized in that in the feature point matching method robust to a viewpoint change, in the step (d), a homography matrix is obtained by using four pairs of matched feature points.

According to another aspect of the present invention, there is provided a feature point matching method robust to a viewpoint change, wherein in the step (e), the feature points of the reference image are transformed through a homography matrix, and the transformed feature points and feature points matching the feature points The Euclidean distance is compared with the threshold value, and only the feature point is classified and classified if the Euclidean distance is within the threshold value.

The present invention also relates to a computer-readable recording medium on which a program for performing a feature point matching method robust to a change in viewpoint is recorded.

As described above, according to the feature point matching method robust to the viewpoint change according to the present invention, the main curvature is applied to the feature points extracted by the FAST method, and the matching pairs of the two images are classified using homography, It is possible to shorten the execution time by a calculation amount and to obtain a strong effect on the change of the viewpoint. Particularly, the present invention shows superior performance to the conventional feature point matching method.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram showing a configuration of an overall system for carrying out the present invention; Fig.
2 is a flowchart illustrating a feature point matching method robust to a viewpoint change according to an embodiment of the present invention.
3 is a diagram illustrating 16 pixels existing on a circle based on a center pixel p according to an exemplary embodiment of the present invention.
4 is an exemplary diagram of a decision tree structure according to an embodiment of the present invention;
FIG. 5 is an image obtained by extracting feature points using FAST according to an embodiment of the present invention. FIG. 5 illustrates a case where (a) the NMS method is not applied and (b) the NMS method is applied.
FIG. 6 is an image obtained by extracting feature points by applying the FAST method and the NMS method according to an embodiment of the present invention. FIG. 6 illustrates a case where only (a) NMS is applied and (b) a case where both NMS method and principal curvature are applied.
FIG. 7 illustrates a descriptor block form and image slope of a SIFT according to an embodiment of the present invention. FIG.
8 is a diagram illustrating a minutiae descriptor of a SIFT according to an embodiment of the present invention.
FIG. 9 is a view showing two images and corrected images at different viewpoints according to an embodiment of the present invention. FIG. 9 (a) is a reference image, (b) is a viewpoint different from viewpoint, and (c)
FIG. 10 is a view of the image of the 'Graffiti A' image according to the experiment of the present invention. FIG. 10 shows images of the method (upper), SURF (middle), and SIFT (lower) according to the present invention.
FIG. 11 is a view of the image of the 'Graffiti B' image according to the experiment of the present invention. FIG. 11 is a view of the method (image), SURF (middle), and SIFT (bottom) according to the present invention.
FIG. 12 is a view of a method (image), SURF (middle), and SIFT (bottom) according to the present invention as a matching result image of a 'Home Object A' image according to the experiment of the present invention.
FIG. 13 is a view showing a result image of a 'Home Object B' image according to an experiment of the present invention, and showing images (method), SURF (middle), and SIFT (bottom) according to the present invention.
FIG. 14 is a view showing the image of the method (phase), SURF (middle), and SIFT (bottom) according to the present invention as a matching result image of the 'Home Object C' image according to the experiment of the present invention.
15 is a table showing a comparison of execution time of 'Graffiti A' according to the experiment of the present invention.
16 is a table showing a comparison of execution time of 'Graffiti B' according to the experiment of the present invention.
17 is a table showing a comparison of total execution time for each image according to the experiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the drawings.

In the description of the present invention, the same parts are denoted by the same reference numerals, and repetitive description thereof will be omitted.

First, examples of the configuration of the entire system for carrying out the present invention will be described with reference to Fig.

As shown in FIG. 1, the feature point matching method robust to the viewpoint change according to the present invention includes a computer terminal 20 (FIG. 1) for receiving a multi-view image (or image) 10 and extracting and matching feature points for the image ). &Lt; / RTI > That is, the minutia matching method may be implemented by a program and installed in the computer terminal 20 and executed. A program installed in the computer terminal 20 can operate as a single program system 30. [

Meanwhile, as another embodiment, the feature point matching method robust to the viewpoint change may be implemented by a single electronic circuit such as an ASIC (on-demand semiconductor) in addition to being operated by a general-purpose computer. Or a dedicated computer terminal 20 dedicated to only matching feature points in an image. This will be referred to as a feature point registration device 30. Other possible forms may also be practiced.

On the other hand, the multi-view image 10 is composed of images having at least two view points. In particular, each image 10 is composed of consecutive frames in time. One frame has one image. Also, the image 10 may have one frame (or image). That is, the image 10 corresponds to one image.

Next, a feature point matching method robust to a viewpoint change according to an embodiment of the present invention will be described with reference to FIG. FIG. 2 shows a general flowchart of a feature point matching method robust to a viewpoint change according to the present invention.

As shown in FIG. 2, the feature point matching method robust to the viewpoint change according to the present invention includes a step of extracting feature points (S10), a step of applying a main curvature (S20), a step of generating feature point descriptors, Step S30, obtaining homography using the matched pair of feature points S40, and classifying the matched pair of feature points using homography S50.

First, feature points are extracted from the multi-view image using the FAST method (S10).

The FAST (Features from Accelerated Segment Test) method is a suitable method for real-time feature point extraction. It uses Harris corner detector, DOG (Difference Of Gaussian) of SIFT (Scale Invariant Feature Transform) Feature points can be extracted faster than Haar wavelet and Smallest Uni-Value Segment Assimilating Nucleus Test (SUSAN).

3 shows the relationship between the center pixel and the surrounding pixels for determining the feature point. The FAST method forms a circle with a distance of 3 pixels from the center of a pixel p in the image. Letting p-> x _k be the number of surrounding pixels in the formed circle, the threshold value is added or subtracted from the brightness value (I _{p - > xk} ) and the brightness value (I _p ) And determines whether or not the characteristic point If there are N or more pixels that are larger than the brightness value obtained by adding the threshold value to the center pixel p among the brightness values of the 16 pixels or smaller than the brightness value obtained by subtracting the threshold value from the center pixel p, . According to Non-Patent Document 5, if the value of N is 9, the repeatability in the next frame is the highest and the threshold value can be set by the user to detect the feature point.

In order to extract the feature point candidate more quickly, only the brightness values of the pixels located at 1, 5, 9, and 13 in FIG. 3 are compared instead of the brightness values of the 16 pixels constituting the circle. It is judged that the pixel may be a minutiae when three or more of the four pixels are brighter than Ip + t obtained by adding the threshold value t to the brightness value Ip of the central pixel or Ip - t obtained by subtracting the threshold value t from Ip do. Once it is determined that there is a possibility of a feature point, the brightness values of the sixteen pixels are compared to finally determine whether the center pixel p is a minutiae point. If the possibility of the minutiae point is not satisfied, it is not necessary to compare the brightness values of the 16 pixels, so that the speed of minutiae point extraction can be greatly increased.

The above procedure is performed for all the pixels of the image. Therefore, as the size of the inputted image increases, the calculation amount increases and the execution time increases. To solve this problem, the FAST method uses a decision tree structure. In order to construct the tree structure, the relation of each pixel is divided into four states as shown in Equation (1).

[Equation 1]

Where S _p _-> _xk represent the state of the k-th pixel of the 16 pixels making up the circle. If the brightness value (I _{p -> x k} ) of surrounding pixels over the circle is smaller than the value obtained by subtracting the threshold value from the brightness value (I _p ) of the center pixel p, the pixel is set to d and the center pixel brightness value and threshold value If the value is larger than the sum, the pixel is defined as b. If the value is between the value obtained by subtracting the threshold value and the sum, the value is defined as u.

FIG. 4 shows a process of extracting feature points from all the pixels by a depth first search method by constructing a tree using Equation (1). The alphabet within the block represents the state of the 16 pixels constituting the circle.

First, the state of all pixels is initialized to u. Then, the state of the sixteen pixels is determined using Equation (1) for the central pixel p. The depth search method repeats the process of comparing the pixel located at 1 in FIG. 3 again to the center pixel p and then comparing with the surrounding pixels. If the state comparison between the pixels is determined in the above process, the process of Equation (1) is not performed. Since the above situation is applied to all the pixels, the process of Equation (1) is omitted, and the execution time can be shortened.

Since the extracted feature points are extracted by using the difference between the brightness values of neighboring pixels, the performance is similar to that of the DOG feature point detection method of SIFT, and the processing speed is very fast.

However, unlike DOG of SIFT or Haar wavelet of SURF which is a problem of FAST, there is no process for robustness against the change of scale, and thus many extracted feature points are crowded. Since the feature points of the clustered regions have similar descriptors, the probability of error occurrence increases when feature point matching is performed. To solve this problem, one feature point having the largest value defined next is selected from the clustered feature points using the NMS (Non Maximum Maximum) method. The process of assigning a value to the minutiae is shown in Equation (2).

&Quot; (2) "

Where I _p _{-> x} and I _p are the brightness values of the surrounding pixels and the brightness values of the center pixel, respectively. First, the states of the surrounding 16 pixels are determined according to Equation (1). Then, Equation 2 is applied to a pixel corresponding to the state b (x? S _bright ) and a pixel corresponding to the state d (x? S _dark ) in the decision tree structure to give a larger value as the value of the corresponding feature point . The above process is applied to all the adjacent feature points, and the feature points having the largest value are extracted as the feature points of the cluster region.

FIG. 5 shows an image obtained by extracting feature points by applying the FAST method when N is 9 and the threshold value t is 20. FIG. 5A shows a case where the NMS method is not applied and FIG. 5B shows a case where the NMS method is applied. When the NMS method is applied, it can be seen that the number of minutiae that are unnecessarily extracted from the image is significantly reduced.

Next, feature point extraction is improved using principal curvatures (S20).

Despite applying the NMS method, the FAST method is disadvantageous in that many feature points are extracted along the edge of the image. In order to improve this, the main curvature, which is a value indicating the degree of curving of the curved surface along each direction, is used.

For each of the minutiae extracted above, a 2x2 Hessian matrix consisting of the second derivative values in each direction is obtained as shown in Equation (3).

&Quot; (3) "

Where I _xx is the second-order differential value in the x direction of the pixel, I _yy is the second-order differential value in the y direction, and I _xy is the differential value in the x and y directions, respectively. The principal curvature is calculated from the matrix H of equation (3). Since the eigenvalue of the matrix H is proportional to the principal curvature of the corresponding pixel, avoid calculating the eigenvalue directly and obtain the principal curvature only considering the ratio of eigenvalues. In this case, Equation 4 is calculated by setting a large value as? And a small value as? Among the eigenvalues.

&Quot; (4) "

Here, Tr (H) denotes the sum of elements on the main diagonal of the matrix H, and Det (H) denotes the determinant of the matrix H. In this case, when the value of Det (H) is negative and the expression (5) is not satisfied, it is determined that the pixel is a flat region or an edge [Non-Patent Document 2] In order to improve the disadvantage of this, it is excluded from the feature point. At this time r becomes a threshold value.

&Quot; (5) "

FIG. 6 shows an image obtained by extracting feature points by applying the FAST method and the NMS method. FIG. 6A shows a case where only the NMS method is applied, and FIG. 6B shows a case where both the NMS method and the principal curvature are applied. In (b) image, unlike image (a), it can be seen that the number of feature points existing along the edge part is reduced.

Next, SIFT feature point descriptor generation is generated and feature points are matched (S30).

After the feature points are extracted from the image by the method according to the present invention, a descriptor including information for each feature point is generated. In the present invention, each feature point is described using a descriptor of the SIFT that can describe more information about the feature point than the descriptor of the SURF for feature point matching that is robust to viewpoint change. The minutiae description is described locally with each minutiae as a center as shown in Fig.

16 blocks of 4 × 4 size are represented by 8 directions respectively, and the length of each arrow represents the direction histogram value. All the feature points are thus formed in the form of a vector of values of the direction histogram. FIG. 8 is an example of an 8-directional component of a directional histogram expressed by a SIFT descriptor vector in one of 16 blocks of 4 × 4 size in FIG. As described above, each feature point is represented by eight directions for each of 16 blocks, and they are all described as 128 (4 x 4 x 8) dimensions. The Euclidian distance is calculated as a 128-dimensional vector between feature points extracted from the two images to be matched. If the distance is smaller than the threshold value, the matching pair is selected.

Next, homography is obtained (S40), and the matching feature point pairs are classified using the homography (S50).

Homography is one of the methods of correcting distorted images and is a 3 × 3 matrix that defines a two-dimensional projective mapping between corresponding images. In order to match two images with different viewpoints, homography transformation should be applied centering on one camera coordinate system.

Four matched feature points are required to obtain the homography matrix. If an outlier is used in selecting four pairs of feature points, a false homography matrix can be obtained. In order to remove the anomalies in the process of finding the homography matrix, we use RANSAC (Random Absolute Consensus) method which minimizes the error by predicting the appropriate model from the mixed data of error and noise. The homography matrix is obtained by using the RANSAC method with the matched pair extracted through the 128 dimensional vector computation and the threshold comparison between the feature points extracted from the two different viewpoints.

Equation (6) is a homography coordinate conversion formula.

&Quot; (6) "

(X ₁ , y ₁ ) is the coordinate of the feature point in the reference image,

Is a coordinate transformed by homography transformation, and w ₁ is a scale constant.

In order to match feature points robust to the viewpoint change, the feature points of the reference image are transformed through homography transformation as in Equation (6) and the Euclidean distances between the two coordinates and the threshold value T) to determine the matching pair.

&Quot; (7) "

FIG. 8 shows two images at different viewpoints and an image corrected through homography transformation. It can be seen that the corrected image of FIG. 8 (c) and the image of FIG. 8 (b) are similar to each other.

Next, the effects of the present invention will be explained through experiments.

In the present invention, four graffiti images of 800x640 resolution (two of which are selected for matching and are referred to as A and B, respectively) in the database image provided by Mikolajczyk (Non-Patent Document 10) Three types of Home Object Images (A, B, and C) with a viewpoint change of 800x600 resolution among images [Non-Patent Document 11] were used as experimental images. Experimental environment was Intel i5 quad core CPU, 8GB RAM, Visual Studio 2013 by Microsoft, and OpenCV version 2.4.9. In order to evaluate the performance of the method according to the present invention, SIFT, SURF, etc., which have superior performance among the existing feature point matching methods, were measured and compared. FIGS. 10 to 14 show the results of matching of the method according to the present invention and the SIFT and SURF methods. It can be seen that the matching method is more accurate than the conventional matching methods when the method of the present invention extracts the same number of matching pairs in each image.

The table of FIG. 15 and the table of FIG. 16 are obtained by applying the SIFT and SURF method and the method according to the present invention to images of 800x640 size, and measuring the execution time of each process. Since the method according to the present invention uses the FAST method, it can be seen that the execution time is much shorter than the conventional extraction methods in extracting feature points. In the process of matching feature points, unlike the conventional matching method, the method according to the present invention further includes processes such as homography calculation using RANSAC or matching pair classification using homography transformation, but since unnecessary feature points are removed, Can be seen. Comparing the total execution time, it can be seen that the method according to the present invention is twice as fast as the SURF method and 3.5 times as fast as the SIFT method.

The table of FIG. 17 shows the result of measuring the SIFT, SURF method and the execution time of the method according to the present invention for various images whose viewpoints change. When extracting the same number of matching pairs for each matching method in each image, it can be seen that the method according to the present invention takes less time to perform in all images than in the conventional matching method.

In the present invention, a characteristic point matching method robust to a viewpoint change has been described using FAST (Features from Accelerated Segment Test) feature point detector and SIFT (Scale Invariant Feature Transform) feature point descriptor. In the conventional FAST method, many feature points are unnecessarily detected at the edge portion of the image. This disadvantage is improved by using the main curvature. In order to further describe the information for each extracted feature point, a matching pair was obtained using the SIFT feature point descriptor. In order to match the feature points robust to the viewpoint change, a homography matrix is obtained using RANSAC (Random Domain Consensus), and the feature points of the reference image are transformed by homography transformation and the coordinates of the minutiae points Matching pairs are classified through the Euclidean distance. The image matching method according to the present invention is characterized in that when the total execution time is compared in several images whose viewpoints are changed with respect to the same object or place than SIFT or SURF, the method according to the present invention is faster than the SURF method, Which is 3.5 times faster than that of the conventional matching method, and it can be seen that matching is more accurate than the conventional matching methods when extracting the same number of matching pairs.

Although the present invention has been described in detail with reference to the above embodiments, it is needless to say that the present invention is not limited to the above-described embodiments, and various modifications may be made without departing from the spirit of the present invention.

10: video 20: computer terminal
30: Program system

Claims

A feature point matching method robust to a viewpoint change in extracting feature points from images at at least two different viewpoints,
(a) extracting feature points from the image using a feature from Accelerated Segment Test (FAST) method;
(b) obtaining a Hessian matrix for the extracted feature points, and improving the feature point extraction using the sum of the elements on the main diagonal of the obtained Hessian matrix and the determinant;
(c) acquiring a plurality of feature point pairs by matching feature points extracted using a plurality of frames;
(d) obtaining a homography matrix from a plurality of pairs of feature points and classifying the feature points necessary for the building; And
and (e) repeating the classification using homography using the pair of left and right feature points.

The method according to claim 1,
In step (a), when a pixel p is referred to as a center pixel in the image, a circle having a predetermined distance therefrom is formed, and the brightness of the surrounding pixel is set to be Wherein the center pixel p is determined as a minutiae when the number of neighboring pixels having a difference of more than a predetermined threshold value is N (N is a natural number) or more, as compared with a brightness value. .

3. The method of claim 2,
In the step (a), if only a part of the surrounding pixels over the formed circle is compared with the brightness of the center pixel, if the number of surrounding pixels that differ by more than a predetermined threshold value is equal to or greater than a predetermined ratio, Wherein the feature point extraction unit extracts feature points by comparing all of the neighboring pixels and discriminates that the center pixel is not a feature point if the number of neighboring pixels is less than a predetermined ratio.

3. The method of claim 2,
In step (a), a state of a contrast result between a center pixel and a peripheral pixel is configured in a decision tree structure, and a feature point is extracted from all the pixels by a depth search method. The state of all pixels is initialized to u (unknown) The state of the surrounding pixels is determined according to the result of the brightness comparison with the surrounding pixels with respect to one central pixel, the center pixel is sequentially positioned from the first pixel in the tree of the surrounding pixels according to the depth search method, Wherein the state determination is not performed through the brightness contrast between the pixels when the state between the pixels is determined in the preceding step.

5. The method of claim 4,
In the step (a), the state of the pixel is determined as four states as shown in the following equation (1).
[Equation 1]

6. The method of claim 5,
In the step (a), the state of the peripheral pixels is determined according to Equation (1), and a pixel corresponding to the state b (x∈S _bright ) and the pixel corresponding to the state d (xεS _dark ) A larger value is assigned to the value of the corresponding feature point by applying the following Equation 2 and a feature point having the largest value is given as a feature point of the cluster region by applying the value of the feature point to the adjacent feature points using [Equation 2] The feature point matching method being robust to a viewpoint change.
[Equation 2]

The method according to claim 1,
In the step (b), a 2 × 2 Hessian matrix is obtained for the feature points extracted in the step (a), and if the following equation (3) is not satisfied, the feature points are excluded A feature point matching method that is robust to the change of the point of view.
[Equation 3]

here,

.

,

The method according to claim 1,
In the step (c), 16 blocks each having a size of 4 × 4 are displayed in eight directions with respect to the feature points, and the length of each arrow indicates a direction histogram value, and in the form of a vector composed of the values of the direction histogram Wherein the feature point matching step forms a descriptor.

The method according to claim 1,
In step (c), the descriptors are expressed in eight directions for each of the 16 blocks for each feature point, and all 128 (4x4x8) dimensions are described, and between the feature points extracted from the two images to be matched, 128 Wherein the Euclidean distance is calculated by the dimension vector and the matching point pair is selected when the distance value is smaller than the threshold value.

The method according to claim 1,
Wherein the homography matrix is obtained by using four pairs of matched pairs of feature points in step (d).

11. The method of claim 10,
In the step (e), the feature points of the reference image are transformed through the homography matrix, and the Euclidian distances between the transformed feature points and the feature points matching the feature points of the reference image are obtained and compared with the threshold, The feature points are selected and classified only when they are within a predetermined range.

12. A computer-readable recording medium having recorded thereon a program for performing a feature point matching method robust to the viewpoint change of any one of claims 1 to 11.