KR20200078723A - Method for Feature Point Matching using Variable Circular Template for Multi-resolution Image Registration - Google Patents

Abstract

In order to perform image fusion, change detection, and time series analysis for a multi-sensor image, an image registration process between two images is essential. According to the present invention, provided is a matching technique using a variable circular template, which can simultaneously detect a relative scale, rotation angle difference, and a feature point position between multi-resolution images by re-interpreting an image registration problem between two images having resolution difference in an aspect of affine transformation between two images having different scales. According to the present invention, a scale, a rotation angle and a central position of the variable circular template can be separately detected at a time when the amount of interactive information with a circular template of an image with a small scale is maximized while setting a circular template with respect to a feature point of the image with a small scale and changing a scale of the variable circular template of an image with a large scale to a constant scale unit.

Description

{Method for Feature Point Matching using Variable Circular Template for Multi-resolution Image Registration}

In order to utilize each image captured by various sensors with different resolutions for image fusion, change detection and time series analysis, the image registration process between each image is essential. For such image registration, it is necessary to accurately detect the difference in scale and rotation angle between multiple sensor images having different spatial resolutions. The present invention relates to a method of detecting a difference between a scale and a rotation angle and performing feature point matching for image registration for such multi-resolution images.

An image registration process that matches image coordinates between two different images for comparison of change detection, time series analysis, etc. using remote sensing images with different shooting conditions, such as spatial resolution and angle, although they are remote sensing images taken for the same subject. This is essential. In the image registration process, the feature point detection (feature matching) that detects the feature point that is the conversion reference point of the two image coordinates and the feature point pairs that are common to the two images is the key process. The feature point detection measures the amount of change in the brightness value based on the central pixel of the template, and detects a pixel having a larger amount of change than the neighboring pixels as a feature point (see Moravec, 1977, non-patent document list among the prior art documents, as follows). The geometric shape of these feature points is generally a corner shape, and there are various conventional methods of detecting a corner using eigenvalues of a gradient matrix (Harris and Stephens, 1988; Shi and Tomasi, 1994; Carneiro and Jepson, 2002; Ye, 2014a ). On the other hand, an image descriptor (descriptor) containing image information around the feature point is used for matching (feature matching) between the feature points detected in the two images. The image engineer is a vector that uses the image information around the feature point as its component, and representatively, the Scale Invariant Feature Transform (SIFT) (Lowe, 2004) is a well-known image engineer. Since SIFT-based methods basically use histograms that reflect image characteristics around feature points, they have strong characteristics against geometrical deformations such as scale and rotational changes (Song et. al., 2014; Gong et al. , 2013; Sun et al., 2014). However, the SIFT method is advantageous for detecting a blob-like feature point, but has a disadvantage that is not suitable for detecting a corner point such as a crossing point of a building or a road (Wu et al., 2012). The SIFT method is not suitable as a corner detector for image registration because the location of the detected feature point is detected around the corner, not the actual corner location.

Shechtman and Iran (2007) proposed a Local Self-Similarity (LSS) image descriptor that computes the similarity between the image template centered on the feature point and the image around the feature point and creates a correlation surface and expresses it in polar coordinates. A method of partially improving the performance of an LSS image technician having robust characteristics against changes in the brightness of an image has also been proposed (Liu et al., 2012; Liu and Zeng, 2012). The disadvantage of the LSS imaging technician is that the imaging technician's discriminability for various features is relatively low (Sedaghat and Ebadi, 2015). In this method, after expressing the correlation surface in the form of polar coordinates, after dividing a circular area that belongs to a radius of a certain length from the center of the polar coordinates into a small number of small areas, depending on the distance and direction from the center, it is correlated with the representative value of each area. Create the LSS descriptor vector by assigning the maximum value of the surface. Such a descriptor vector generation method is a method useful for expressing information around a feature point in a vector form, such as a SIFT-based descriptor, but it divides a region around the feature point into a small region of a certain size and assigns a representative value to the brightness or edge. (Edge) It acts as a factor that decreases the sensitivity of the image engineer to small changes in information. That is, since the LSS vector generated around the feature point and the LSS vector of the pixel close to the feature point have almost similar vector components, the ability of the image descriptor to identify the feature point and the pixels near the feature point is reduced. In the matching technique using an existing image descriptor, image information in a narrow area around a feature point is described as a feature vector, and a matching point having a feature vector having a similarity to the feature vector is found in the matching process. In this process, if the region where the feature vector is extracted is small, a problem that image information around the feature point may not be sufficiently reflected may occur. In addition, the process of extracting the feature vector itself may replace the vector with full brightness value information around the feature point. It has the fundamental limitation of using only some video information extracted with. On the other hand, if the surrounding area used in the feature point technique is sufficiently wide, even if some factors that reduce the similarity due to noise or other factors in the process of calculating the similarity for matching the feature points may be less affected. According to the previous study of the inventor (Ye, 2014b), if the size of the circular template used for registration gradually increases, the matching error decreases, and when the radius of the circular template increases to a relatively wide range of 100 to 500 pixels, the matching error It converges to a low level within 2 pixels. These results show the advantages of comparing the similarity of a large area compared to the similarity of a small area centered on a feature point.

The above-described conventional image descriptor-centered feature point matching techniques use a feature point extraction method and feature point similarity calculation method that are less affected by scale change and rotation change to compare feature point similarity between images including scale and rotation change. However, in view of the image transformation view between a large area including a feature point instead of the similarity-based similarity calculation approach for image registration, image registration can be understood as a process of determining affine parameters between two comparison images. In order to determine a general affine parameter between two images, at least three or more corresponding points are required, but from the viewpoint of conversion between two images centered on the matching feature points in the two images, affine transformation between the two images transforms the matching feature points between the two images It has the same meaning as the process of determining each parameter of scale transformation, rotation transformation, and translation transformation between centered images. When each parameter of scale transformation, rotation transformation, and movement transformation is obtained, each image transformation can be performed around the feature points of one image to match the image coordinates of the other. This process itself can be viewed as a kind of global image registration centering on feature points, and more precise image registration, such as a triangulation-based local transformation technique (Ye, 2014c). It can be used as an initial image registration method. When the difference in scale and rotation between the two images is corrected, the geometric difference between the two images is corrected, so that the accuracy of the similarity calculation can be increased in an additional precise image registration process. The problem to be solved in this approach is that the matching feature point between the two images is generally not known in advance, so the process of obtaining each affine parameter and the process of finding the matching feature point must be performed simultaneously.

In the present invention, based on the variable circular template matching technique (Ye, 2018) created by the present inventor, a feature matching technique of a multi-resolution image using affine parameters that can replace the existing feature point detection and matching method is provided. . To this end, we first analyze the effect of geometric correction between images on the improvement of similarity of feature point registration, and present a method of image conversion using a circular template between images with different scales. Next, a method of designing a circular template for calculating the similarity of feature points and a method of calculating the similarity based on the amount of mutual information between the circular templates are presented. In order to verify the performance of the created matching method, performance evaluation is performed using multipurpose practical satellite images #2, 3, and 3A that have different scales from the scale images generated under various conditions using multipurpose utility satellite #3 images. Each was performed.

Carneiro, G. and A. D. Jepson, 2002.Phase-based local features, Proceedings of the 7th European Conference on Computer Vision-Part I, pp.282-296. Gong, M., Zhao, S., Jiao, L., Tian, D., Wang, S., 2014.A novel coarse-to-fine scheme for automatic image registration based on SIFT and mutual information. IEEE Transactions on Geoscience and Remote Sensing, 52(7):4328-4338. Harris, C., and M. Stephens, 1988.A combined corner and edge detector, Proceedings of the 4th Alvey Vision Conference, pp.147-151. Liu, J. and G. Zeng, 2012. Description of interest regions with oriented local self-similarity, Optics Communications, 285:2549-2557. Liu, J., G. Zeng, and J. Fan, 2012. Fast Local Self-Similarity for describing interest regions, Pattern Recognition Letters. 33:1224-1235. Lowe, D. G., 2004. Distinctive image features from scale-Invariant keypoints, International Journal of Computer Vision, 60(2):91-110. Moravec, H.P., 1977.Towards automatic visual obstacle avoidance, Fifth International Joint Conference on Artificial Intelligent, pp.584. Sedaghat, A. and H. Ebadi, 2015. Distinctive order based self-similarity descriptor for multi-sensor remote sensing image matching, ISPRS Journal of Photogrammetry and Remote Sensing, vol. 108, pp. 62-71. Shechtman, E., Irani, M., 2007.Matching local self-similarities across images and videos, IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, USA, 17-22 June, pp. 1-8. Shi J. and C. Tomasi, 1994.Good Features to Track, IEEE conference on Computer Vision and Pattern Recognition, Seattle, USA, 21-23 June, pp.593-600. Song, Z., Zhou, S., Guan, J., 2014.A novel image registration algorithm for remote sensing under affine transformation. IEEE Transactions on Geoscience and Remote Sensing, 52(8):4895-4912. Sun, Y., Zhao, L., Huang, S., Yan, L., Dissanayake, G., 2014.L2-SIFT: SIFT feature extraction and matching for large images in large-scale aerial photogrammetry, ISPRS Journal of Photogrammetry and Remote Sensing, 91:1-16. Wu, B., Zhang, Y. & Zhu, Q. 2012. Integrated point and edge matching on poor textural images constrained by self-adaptive triangulations, ISPRS Journal of Photogrammetry and Remote Sensing, 68:40-55. Ye, C.H., 2014a. Feature detection using geometric mean of eigenvalues of gradient matrix, Korean Journal of Remote Sensing, 30(6):769-776. Ye, C.H., 2014b. Mutual Information-based circular template matching for image registration, Korean Journal of Remote Sensing, 30(5):547-557. Ye, C.H., 2014c. Image registration using outlier removal and triangulation-based local transformation, Korean Journal of Remote Sensing, 30(6):787-795. Ye, C.H., 2018.Multiresolution image matching using variable circular template, Proceedings of the KSRS Fall 2018, Muju, Korea, 24-26 October, pp.160.

An object of the present invention is to provide an accurate matching method between feature points detected in each image, such as accurately detecting a difference between a scale and a rotation angle between multiple sensor images having different spatial resolutions.

)를 검출하는 단계; (b) 상기 특징좌표(

)를 중심으로 반지름 R인 원형템플릿을 정하는 단계; (c) 상기 원형템플릿 안에서 상기 특징좌표(

)를 중심으로 거리가

, 각도가

인 제1화소들의 위치(

)를 각각 정하는 단계; (d) 상기 제1화소들 각각에 대한 밝기값을 포함하는 제1화소들의 집합(

)을 구하는 단계;를 포함하도록 하고, 상기 제1영상에 비하여 스케일이 큰 제2영상에 대하여는, (a) 후보특징점(

)을 정하는 단계; (b) 상기 후보특징점(

)을 중심으로 거리가

, 각도가

인 제2화소들의 위치(

)를 각각 정하는 단계; (c) 상기 제2화소들 각각에 대한 밝기값을 포함하는 제2화소들의 집합(

)을 구하는 단계; 를 포함하도록 한 후, 상기 제2화소들의 위치(

)를 갱신해 가면서 상기 제2화소들의 집합(

)을 재구성하는 과정을 각각 반복한 후, 상기 제1화소들의 집합(

)과 상기 제2화소들의 집합(

) 사이의 상호정보량(

)를 구하는 단계;를 수행하되 상기 제2화소들의 위치(

) 갱신은 (a) 상기 제2영상의 스케일범위 내에서 각각의 스케일(

)을 순차적으로 적용해가면서 상기 제2화소들의 위치(

)를 갱신하고, (b) 상기 제2화소들의 각도

를 일정각도(

)만큼 순차적으로 증가시키면서 상기 제2화소들의 위치(

)를 갱신하고, (c) 상기 후보특징점(

)에 대하여 일정한 탐색범위 내에서 이동변위(

)를 적용해가면서 상기 제2화소들의 위치(

)를 갱신하도록 하는 것이 바람직하다. 그리고 그 후에는 상기 상호정보량(

)가 최대값이 될 때의 파라미터(

)를 구하는 단계; 및 상기 파라미터(

)를 적용한 어파인 변환계수를 이용하여 상기 특징좌표(

)에 대응되는 상기 제2영상에서의 특징점(

)을 구하는 단계; 를 포함하도록 하는 것이 바람직하다.In order to achieve the above object, the feature matching method using a variable circular template when registering a multi-resolution image according to the present invention is a method of performing feature matching for multi-resolution image registration by an information system, for the first image , (a) Feature coordinates (

Detecting); (b) The characteristic coordinates (

Determining a circular template having a radius R around ); (c) the feature coordinates (

Centered on)

, The angle

Location of the first pixels (

Respectively); (d) a set of first pixels including brightness values for each of the first pixels (

), and for a second image having a larger scale than the first image, (a) a candidate feature point (

Determining); (b) The candidate feature points (

)

, The angle

Location of the second pixels (

Respectively); (c) a set of second pixels including brightness values for each of the second pixels (

); After including the, the position of the second pixels (

) While updating the second set of pixels (

) After repeating the process of reconstructing each of the first set of pixels (

) And the set of second pixels (

) Amount of mutual information

); but the location of the second pixels (

) Update (a) each scale within the scale range of the second image (

) While sequentially applying the positions of the second pixels (

), and (b) the angles of the second pixels.

The constant angle (

) While sequentially increasing the positions of the second pixels (

), and (c) the candidate feature points (

), the displacement within a certain search range (

) While applying the position of the second pixels (

It is desirable to update ). And after that, the amount of mutual information (

) Is the maximum value parameter (

); And the parameter (

) Using the affine transformation coefficient to which the above is applied.

) Corresponding to the feature point in the second image (

); It is preferable to include.

)는, 일정 범위 내에서 코너 응답값이 최대가 되는 코너점의 위치를 기하평균 기반의 특징점 검출기를 이용하여 검출하는 것을 특징으로 하는, 다중해상도 영상등록 시 가변 원형템플릿을 이용한 특징정합 방법으로 하는 것도 바람직하다.In addition to the above-described features, the feature coordinates (

) Is a feature matching method using a variable circular template when registering a multi-resolution image, characterized in that the position of the corner point at which the corner response value is the maximum within a certain range is detected using a feature point detector based on a geometric mean. It is also preferred.

)는 아래 식에 의하여 구하며,And the position of the first pixels (

) Is obtained by the following equation,

은 원형템플릿의 반지름 위에 위치하는 전체 화소수,

는 원형템플릿의 원주 위에 위치하는 전체 화소수이다.)(

Is the total number of pixels located above the radius of the circular template,

Is the total number of pixels located on the circumference of the circular template.)

)는 다음의 식에 의하여 구하며,The position of the second pixels (

) Is obtained by the following equation,

는 제1영상과 제2영상의 스케일 크기를 반영한 스케일 파라미터다.)(

Is a scale parameter reflecting the scale size of the first image and the second image.)

) 및 상기 제2화소들의 집합(

)은 다음 각각의 식에 의하여 구하는 것을 특징으로 하는, 다중해상도 영상등록 시 가변 원형템플릿을 이용한 특징정합 방법The first set of pixels (

) And the set of second pixels (

) Is a feature matching method using variable circular templates when registering multi-resolution images,

는 상기 제1영상의 (

) 위치에 존재하는 화소의 밝기값이다.)(

Is the first image (

) It is the brightness value of the pixel existing at the position.)

는 상기 제2영상의 (

) 위치에 존재하는 화소의 밝기값이다.)(

Is the second image (

) It is the brightness value of the pixel existing at the position.)

)은 아래 식에 의하여 구해지는 것을 특징으로 하는, 다중해상도 영상등록 시 가변 원형템플릿을 이용한 특징정합 방법으로 하는 것도 가능하다.The amount of mutual information (

) Is obtained by the following equation, and it is also possible to use a feature matching method using a variable circular template when registering a multi-resolution image.

(Where H is entropy)

)과 회전각도(

)의 차이를 정확하게 검출하고, 이들을 통하여 상기 제2영상에서의 대응 특징점(

)도 찾아서 제공하는 등 각각의 영상에서 검출된 특징점 간의 정확한 정합방법을 제공할 수 있는 효과가 있다.As described above, according to the present invention, in the feature matching method using a variable circular template when registering a multi-resolution image, the scale between the first image and the second image having different spatial resolutions (

) And rotation angle (

) To accurately detect the difference, and through these, corresponding feature points in the second image (

) Also has the effect of providing an accurate matching method between feature points detected in each image, such as by finding and providing the same.

Recently, as various kinds of high-resolution satellite images have appeared, efforts to utilize these multi-resolution sensor images together for various purposes have been active. In order to use multiple sensor images for image fusion, change detection, and time series analysis, the image registration process between two images is essential. For image registration between satellite images having different spatial resolutions, SIFT-based feature matching method can be applied, but the location of the detected feature points is detected around the corner, and the original feature is extracted through the process of extracting feature vectors. It has a fundamental limitation that uses only some feature information instead of image brightness value information. In the present invention, the image registration problem between two images having a resolution difference is reinterpreted from the viewpoint of affine transformation between two images having different scales, thereby simultaneously detecting relative scale, rotation angle difference, and feature point location between multiple resolution images. Invented a matching technique using a variable circular template that can be done. The matching method using the variable circular template according to the present invention sets a circular template around the feature points of the small scale image and changes the scale of the variable circular template in a certain scale unit in a large scale image while the circular template of the small scale image It is possible to detect the scale, rotation angle, and the central position of the variable circular template, respectively, when the mutual information amount with the maximum becomes maximum.

While the existing image registration method presents various image descriptors and matching methods in terms of detecting and matching feature points that are invariant to scale and rotation, the present invention solves the image registration problem from a new perspective of image conversion for a large area around a feature point. . Through the new approach from the image conversion perspective, it is possible to accurately obtain the relative scale, rotation, and feature point positions between images with different resolutions. In relation to the relative scale detection, in the prior art, a characteristic scale is detected based on the features present in the two images, and then the size is compared and the relative scale is estimated, but in this case, the feature points that must first find the corresponding two feature points There has been a problem that it is difficult to apply it to automatic image registration due to a matching problem, but the present invention primarily provides a method for automatically detecting a relative scale between multi-resolution images, and the relative rotation angle difference and feature point location between two images. It is possible to accurately detect and provide a feature matching method between multi-resolution images.

스케일 펙터

를 이용하여, 위치좌표 (

)에 대응되는 픽셀위치를 정의하는 도면이다.
도 6은 원형템플릿의 픽셀들을 2차원 배열 형태로 표시한 것이다.
도 7은 원형템플릿 내에서 각각의 픽셀 위치를 각도

만큼 변화시켜 가는 것을 도시한 것이다.
도 8은 다목적실용위성 Kompsat-2, Kompsat-3, Kompsat-3A호에 대한 각각의 영상이다.
도 9는 Kompsat-3 영상에 대하여 코너 검출 결과를 보여주는 사진이다.
도 10은 Kompsat-3 영상에 대한 테스트 셋트 영상에 대한 사진이다.
도 11은 회전각도와 스케일팩터의 변화에 따른 상호정보량의 변화를 도시한 것이다.
도 12는 다양한 스케일 팩터와 회전성분을 포함한 저해상도 영상의 예를 보여준다.
도 13은 상호간의 스케일 팩터 변화에 따른 상호정보량의 변화를 보여준다. 1 is a flowchart illustrating a method of matching a feature using a variable circular template when registering a multi-resolution image according to the present invention.
2 is a photograph showing a comparison of the Ikonos satellite image and the Quickbird satellite image taken in the same region and affine conversion.
FIG. 3 is a diagram showing similarities before and after affine transformation for a Quickbird satellite image, and is a similarity calculated using mutual information (MI).
4 is a diagram comparing similarities between a small-scale image and an affine-transformed large-scale image.
5 is a rotation angle

Scale factor

Using, position coordinate (

).
6 shows pixels of a circular template in a two-dimensional array.
7 shows the angle of each pixel position within the circular template.

It shows what changes as much as possible.
8 is a video for each of the multi-purpose satellites Kompsat-2, Kompsat-3, Kompsat-3A.
9 is a photograph showing a corner detection result for the Kompsat-3 image.
10 is a photograph of a test set image for the Kompsat-3 image.
11 shows a change in the amount of mutual information according to a change in the rotation angle and scale factor.
12 shows examples of low-resolution images including various scale factors and rotational components.
13 shows a change in the amount of mutual information according to a change in scale factor between each other.

Hereinafter, the present invention will be described in detail so that the above-mentioned objects and features become clear, and accordingly, a person skilled in the art to which the present invention pertains can easily implement the technical spirit of the present invention. In addition, in describing the present invention, among the known technologies related to the present invention, it is already well known in the technical field, and a detailed description thereof is determined when it is determined that the detailed description of the known technology may unnecessarily obscure the subject matter of the present invention. It will be omitted.

In addition, the terminology used in the present invention has been selected as a general terminology that is currently widely used as much as possible, but in certain cases, the term is arbitrarily selected by the applicant, and in this case, its meaning is described in detail in the description of the applicable invention. It is intended to clarify that the present invention should be grasped as a meaning of a term rather than a name of. Terms used in the description of the embodiments are only used to describe specific embodiments, and are not intended to limit the embodiments. Singular expressions include plural expressions unless the context clearly indicates otherwise.

The embodiments may be modified in various forms and may have various additional embodiments, in which specific embodiments are shown in the drawings and detailed descriptions thereof are described. However, this is not intended to limit the embodiments to a specific form, it should be understood that it includes all changes or equivalents or substitutes included in the spirit and scope of the embodiments.

In the description of various embodiments, expressions such as “first”, “second”, “first”, or “second” may modify various components of the embodiments, but do not limit the components. For example, the above expressions do not limit the order and/or importance of the components. The above expressions can be used to distinguish one component from another component.

Hereinafter, the present invention will be described with reference to the accompanying drawings. First, FIG. 2 is a photograph showing an example of comparison and affine conversion of the Ikonos satellite image and the Quickbird satellite image taken in the same region. Heterogeneous sensor (multi-sensor) images with different resolutions reduce the similarity between feature points in the image registration process due to geometric differences caused by differences in image acquisition conditions. FIG. 2 shows a comparison thereof, and FIGS. 2(a) and 2(b) show examples of Ikonos satellite images and Quickbird satellite images taken in the same region. And Figure 2 (c) is an image affine (affine) a Quickbird satellite image based on the Ikonos satellite image, as shown in each enlarged image at the bottom of Figure 2 Quickbird satellite image and Ikonos after affine conversion It can be seen that the geometrical difference from the satellite image is reduced.

In addition, through geometric correction through affine transformation, it is possible to expect an improvement in consistency when matching feature points for image registration. FIG. 3 is a diagram showing the result of similarity calculation, and is located at the center of the white circle in FIG. 2(a). It shows the similarity calculated by using the mutual information to find the corresponding point for the corner pixel of the Ikonos satellite image in the Quickbird satellite image before and after the affine transformation. 2(a) is a result before affine conversion, and FIG. 3(b) is a result after affine conversion, and the maximum value of the inter-information amount (MI) of a matched pixel is 0.514 and 0.613, respectively, before and after affine conversion. The result shows a higher degree of similarity after conversion. In particular, it can be seen that the difference in the similarity between the matching point and the surrounding pixels is greatly increased after the affine transformation, as shown in FIG. Therefore, it can be seen that by correcting the geometrical difference between the two images, the matching performance can be improved by increasing the similarity between the matching points and increasing the difference in the similarity between the matching point and the surrounding pixels. For geometric correction between two images, in general, after matching the feature points between the two images, an image transformation is performed using the matched feature points. In the present invention, a method of simultaneously performing image transformation and feature point matching using a circular template is performed. to provide.

), 스케일이 큰 영상의 대응점을 (

), 스케일이 큰 영상의 대응점 좌표(

)에 대한 어파인 변환 후의 좌표를 (

)라 할 때, 대응점 (

)과 변환 후 좌표 (

) 사이의 관계는 일반적으로 아래의 수학식 1과 같은 어파인 변환 관계로 주어진다.Hereinafter, image conversion using a circular template will be described. The matching of the feature points between two images having different scales due to the difference in resolution can be interpreted as a process of finding an affine transform component connecting corresponding points between the two images. Correspondence point of relatively small image (

), the corresponding point of the large-scale image (

), coordinates of the corresponding point of the large-scale image (

Coordinates after affine transformation for ()

), corresponding point (

) And coordinates after conversion (

) Is generally given as an affine transformation relationship as in Equation 1 below.

)로 나타내면 아래 수학식 2와 같이 표현된다. 따라서 스케일이 작은 영상의 대응점(

)과의 유사도가 가장 높은 어파인 변환 후의 좌표(

)를 구하면 어파인 변환 계수를 이용하여 스케일이 큰 영상에서의 대응점(

)을 구할 수 있다.And this is scale component s, rotation angle θ, displacement

), it is expressed as Equation 2 below. Therefore, the corresponding point of the small-scale image (

), the coordinate after the transformation, which has the highest similarity to (

) To obtain the corresponding point (

).

)과 어파인 변환 후 좌표(

)와의 유사도를 측정하기 위하여 스케일이 작은 영상의 대응점(

)과 스케일이 큰 영상의 후보대응점(

)을 중심으로 원형템플릿을 각각 생성한다. 원형템플릿 내에 위치하는 화소들은 두 대응점 간의 유사도 계산에 사용되며 대응점을 중심으로 영상이 회전 하여도 원형템플릿 내부에 포함되는 화소들은 변하지 않는 회전 불변(rotation invariance) 특성을 가진다. 따라서 대응점 (

) 중심의 원형템플릿은 고정하고 스케일이 큰 영상의 후보대응점(

) 중심의 원형템플릿은 일정각도 단위로 회전시키면서 두 템플릿의 유사도가 최고가 되는 회전각도를 측정하여 두 템플릿의 상대적인 회전각도 차이, 즉 두 영상의 상대적인 회전각도 차이를 구한다. Correspondence point (

) And affine coordinates after transformation (

) To measure the similarity with)

) And candidate correspondence points for large-scale images (

Each circle template is created around ). The pixels located in the circular template are used to calculate the similarity between the two corresponding points, and even if the image is rotated around the corresponding point, the pixels included in the circular template have a rotation invariance characteristic that does not change. So the corresponding point (

) The centered circular template is a candidate for a fixed and large-scale image (

) The circular template in the center measures the rotation angle at which the similarity between the two templates becomes the highest while rotating in a certain angle unit to obtain the relative rotation angle difference between the two templates, that is, the relative rotation angle difference between the two images.

에 대응되는 스케일이 큰 영상템플릿 내의 위치

를 스케일 파라미터

를 이용하여

로 계산한 후에 두 원형템플릿 반지름 상에 동일한 개수의 화소를 배치한다. 원형템플릿의 반지름 상에 동일 개수의 화소를 배치하면 스케일 파라미터

의 변화에도 불구하고 두 원형템플릿의 유사도 계산에 사용되는 화소들의 개수를 일치시킬 수 있다. On the other hand, since the relative scale difference exists between a small-scale image and a large-scale image, if two circular templates are set to the same size, the circular template of the large-scale image is actually smaller than the circular template of the small-scale image. It will include. Therefore, before comparing the similarity between two circular templates, it is necessary to first match the scales of the two circular templates. To this end, as shown in Figure 4, the position in the small scale image template

Position in a large-scale video template corresponding to

Scale parameter

Using

After calculating as, the same number of pixels are placed on the radius of the two template templates. If the same number of pixels are placed on the radius of the circular template, the scale parameter

Despite the change of, the number of pixels used for calculating the similarity between the two circular templates can be matched.

)를 결정하기 위하여 도 5에서 보는 바와 같이 극좌표(polar coordinates) 방식을 이용하는데, 스케일이 작은 영상에서 중심 좌표가 (

)이고 원점에서의 거리가

, 각도가

인 화소의 위치는 아래 수학식 3과 같이 구한다. 여기서 도 5(a)는 스케일이 작은 영상에서 픽셀위치를 정의하는 도면이며, 도 5(b)는 스케일이 큰 영상에서 픽셀위치를 정의하는 도면이다.Next, the design of the circular template and the similarity calculation method will be described. In the present invention, the position of the pixels belonging to the circular template (

In order to determine ), as shown in FIG. 5, the polar coordinates method is used. In a small scale image, the center coordinate is (

) And the distance from the origin

, The angle

The position of the phosphorus pixel is obtained as in Equation 3 below. Here, FIG. 5(a) is a diagram for defining a pixel position in a small scale image, and FIG. 5(b) is a diagram for defining a pixel position in a large scale image.

)는 아래 수학식 4와 같이 주어진다. And in this way, the position of the corresponding pixel in the large-scale image (

) Is given as in Equation 4 below.

은 원형템플릿의 반지름 위에 위치하는 전체 화소수이며,

는 원형템플릿의 원주 위에 위치하는 전체 화소수를 의미한다. 도 5에서는

=8,

=8인 원형템플릿의 생성 예를 보여준다. 그리고

사이의 모든 화소의 위치를 거리

과 각도

의 2차원 좌표평면에 도시하면 도 6과 같이 2차원 배열 형태로 표시된다. 수평방향의 축은 원형템플릿 중심에서 방사적으로 떨어진 위치

을 나타내고, 수직방향의 축은 반시계 방향의 회전각도

를 나타낸다. 여기서 도 6(a)는 스케일이 작은 영상의 픽셀들을 2차원 배열한 모습이며, 도 6(b)는 스케일이 큰 영상의 픽셀들을 2차원 배열한 모습이다.here

Is the total number of pixels located above the radius of the circular template,

Is the total number of pixels located on the circumference of the circular template. In Figure 5

=8,

Here is an example of creating a circular template with =8. And

Distance between all pixels

And angle

When shown in the two-dimensional coordinate plane of the is displayed in the form of a two-dimensional array as shown in FIG. The horizontal axis is located radially away from the center of the circular template.

And the vertical axis represents the counterclockwise rotation angle.

Indicates. Here, FIG. 6(a) shows a 2-dimensional array of pixels of a small-scale image, and FIG. 6(b) shows a 2-dimensional array of pixels of a large-scale image.

) 위치에 존재하는 화소의 밝기값을

라 하면, 스케일이 작은 원형템플릿에 포함되는 화소들의 집합

은 아래 수학식 5와 같이 주어진다.On the other hand, the

) The brightness value of the pixel at the position

Is a set of pixels included in a small-scale circular template.

Is given by Equation 5 below.

) 위치에 존재하는 화소의 밝기값을

라 하면 스케일이 큰 원형템플릿에 포함되는 화소들의 집합

는 아래 수학식 6과 같이 주어진다.And of the large-scale video (

) The brightness value of the pixel at the position

Is a set of pixels included in a large-scale circular template.

Is given by Equation 6 below.

이며

는 스케일이 큰 영상의 스케일 범위를 나타나는 인덱스로서

의 범위를 가진다. 그리고

는 스케일이 큰 영상에서 생성된 원형템플릿의 회전각도를 나타내며 아래 수학식 7과 같이 주어진다. 상기 수학식 7에서 기본값으로 이 T=180이 주어지며 회전각도를 보다 미세하게 조정하기 위해서는 T를 보다 큰 값으로 조정하는 것이 바람직하다. here

Is

Is an index indicating the scale range of a large-scale image.

Has a range of And

Denotes the rotation angle of a circular template generated from a large-scale image and is given as Equation 7 below. In Equation 7, this T=180 is given as a default value, and it is preferable to adjust T to a larger value in order to finely adjust the rotation angle.

과

사이의 유사도를 측정하기 위하여 아래 수학식 8과 같이 주어지는 상호정보량 MI를 계산한다. Meanwhile, a set of pixels included in a circular template of a small-scale image and a circular template of a large-scale image

and

In order to measure the similarity between them, the mutual information amount MI given as Equation 8 below is calculated.

Here, H represents entropy.

)를 검출해 내는데, 이를 위해서는 스케일이 작은 영상 중 일정 범위 내에서 코너 응답값이 최대가 되는 코너점의 위치를 종래기술(Ye, 2014a)에 의한 기하 평균 기반의 특징점 검출기를 이용하여 검출하도록 하는 것이 바람직하다. 검출된 특징좌표 (

)를 중심으로 반지름이 R인 원형템플릿에 속하는 화소집합

을 설정한 후에 스케일이 큰 영상의 스케일범위

에 속하는 각 스케일

에 대해 상기 수학식 6을 이용하여 원형템플릿 화소집합

를 설정한다. 이때 원형템플릿을 도 7과 같이

만큼 순차적으로 증가시키면서 상기 수학식 6을 이용하여 (

) 위치를 갱신한다. 여기서 도 7(a)는 초기 위치이며, 도 7(b)는 각도가

만큼 증가된 후의 위치이다. 이 과정을 통해 초기에 만들어진 원형템플릿 화소집합

는 갱신된 (

)에 위치하는 화소집합으로 재구성된다. 재구성된 원형템플릿의 중심 위치 (

)를 일정한 탐색 범위 내의

와

값에 따라 변경하면서 아래 수학식 9와 같이 상호정보량 MI가 최대값이 될 때의 각 파라미터(

) 의 값을 구한다. 여기서 스케일이 큰 영상의 중심위치 (

)는

가 되도록 하는 것이 바람직하다. 따라서 처음에는 스케일이 큰 영상의 중심위치 (

)를 스케일이 작은 영상의 특징좌표(

)와 동일하게 하는 것도 가능하다.In order to calculate the mutual information amount MI in this way, first, feature coordinates of a small-scale image (

), in order to detect the position of the corner point where the corner response value is the maximum within a certain range among small scale images, using a geometric average-based feature point detector according to the prior art (Ye, 2014a). It is preferred. Featured coordinates (

A set of pixels belonging to a circular template with a radius of R around

After setting the scale range of images with large scale

Each scale belonging to

For the circular template pixel set using Equation (6) above

To set. At this time, the circular template is shown in FIG.

While increasing sequentially by using the equation (6) (

) Update the location. Here, Figure 7 (a) is the initial position, Figure 7 (b) is an angle

This is the position after the increase. The circular template pixel set created through this process

Is updated (

). The center position of the reconstructed circular template (

) Within a certain search range

Wow

Each parameter (when the mutual information amount MI becomes the maximum value as shown in Equation 9 below while changing according to the value)

). Here, the center position of the large-scale image (

) Is

It is desirable to be. Therefore, initially, the center position of the large-scale image (

) Is the characteristic coordinate of the small-scale image (

).

사이의 간격과 회전각도

사이의 간격을 각각 일정한 간격을 두고 생성하도록 하는 것이 바람직하다. 원형템플릿의 반지름 R이 300 화소이면 총

=300개의

를 생성할 수 있으나, 3 화소 간격으로 화소값을 취하면

=100개의

를 생성하게 된다. 이 경우 상호정보량 계산에 이용되는 화소의 수가 1/3로 감소하여 연산시간도 함께 감소한다. 회전각도

의 간격도 2도 간격으로 생성하면 각

위치마다

=360개의 화소를 생성할 수 있으나 5도 간격으로 생성하면

=72개의 화소가 생성되어 상호정보량 계산에 이용되는 화소의 수가 1/5로 감소하게 된다. 또한 스케일이 큰 영상에서의 정합 위치 (

)를 탐색하는데 소요되는 시간을 줄이기 위하여 스케일이 큰 영상의 전체화소 그레디언트의 히스토그램에서 상위 임계비율

이상인 화소를 정합 후보점으로 한정하면 탐색 시간을 크게 단축할 수 있다. When performing these processes, in order to reduce the time required to calculate the amount of mutual information, the location located radially away from the center of the template when creating the circular template

Interval and rotation angle

It is desirable to create the intervals between each at regular intervals. If the radius R of a circular template is 300 pixels, the total

=300

Can be generated, but taking pixel values at 3 pixel intervals

=100

Will generate In this case, the number of pixels used for calculating the amount of mutual information is reduced to 1/3, and the calculation time is also reduced. Rotation angle

If you create an interval of 2 degrees,

Per location

=360 pixels can be generated, but if they are created at 5 degree intervals

=72 pixels are generated and the number of pixels used for calculating the mutual information amount is reduced to 1/5. Also, the registration position (

) In order to reduce the time required to search ), the upper threshold ratio in the histogram of the full pixel gradient of a large-scale image

If the above-described pixels are limited to matching candidate points, the search time can be significantly shortened.

Meanwhile, FIG. 1 is a flowchart illustrating a method of matching a feature using a variable circular template when registering a multi-resolution image according to the present invention. Hereinafter, a detailed implementation method of the present invention will be described step by step with reference to FIG. 1 based on the contents described above.

)를 검출하는 단계(s110)를 수행하도록 하는 것이 바람직하다. 여기서 상기 제1영상은 상술한 설명에서 ‘스케일이 작은 영상’을 말한다. 그리고 상기 제1영상에 대하여 상기 특징좌표(

)를 중심으로 반지름 R인 원형템플릿을 정하는 단계(s120)를 수행하도록 하는 것이 바람직한데, 상기 특징좌표(

)는 상기 제1영상 중 일정범위 내에서 코너 응답값이 최대가 되는 코너점의 위치로서, 종래기술에 의한 기하평균 기반의 특징점 검출기를 이용하여 검출되는 지점으로 하는 것이 바람직하다. 상기 원형템플릿이 정해진 후에는 상기 원형템플릿 안에서 상기 특징좌표(

)를 중심으로 거리가

, 각도가

인 제1화소들의 위치(

)를 각각 정하는 단계를 수행하도록 하는 것이 바람직한데(s130) 상기 제1화소들의 위치(

)는 상기 수학식 3과 같이 표현될 수 있다. 그 후에는 상기 제1화소들 각각에 대한 밝기값을 포함하는 제1화소들의 집합(

)을 구하는 단계(s140)를 수행하도록 하는 것이 바람직한데, 상기 제1화소들의 집합(

)은 상기 수학식 5와 같이 구하는 것이 더욱 바람직하다.The present invention is a method for performing feature matching for multi-resolution image registration by an information system. First, as shown in FIG. 1, feature coordinates for a first image (

It is preferable to perform the step of detecting (s110). Here, the first image refers to an'image having a small scale' in the above description. And the feature coordinates for the first image (

It is preferable to perform the step (s120) of determining a circular template having a radius R around the center.

) Is a position of a corner point at which a corner response value becomes the maximum within a predetermined range among the first images, and is preferably a point detected using a geometric average-based feature point detector according to the prior art. After the circular template is determined, the feature coordinates in the circular template (

Centered on)

, The angle

Location of the first pixels (

It is preferable to perform each step of determining (s130) the position of the first pixels (

) May be expressed as Equation 3 above. After that, a set of first pixels including brightness values for each of the first pixels (

It is preferable to perform the step of obtaining (s140), the first set of pixels (

) Is more preferably obtained as in Equation 5 above.

)을 정하는 단계(s210)를 수행하도록 하는데, 상기 제2영상은 상술한 설명 중 ‘스케일이 큰 영상’을 말한다. 그리고 후보특징점(

)은 상기 제2영상에 대한 특징점(

)을 찾기 위한 후보지점을 말하며 상술한 설명에서 후보대응점으로 언급한 바 있으며, 상기 제1영상의 특징점인 상기 특징좌표(

)와 동일하게 하는 것도 가능하다. 상기 후보특징점(

)을 정한 후에는 상기 후보특징점(

)을 중심으로 거리가

, 각도가

인 제2화소들의 위치(

)를 각각 정하는 단계(s220)를 수행하게 되는데, 상기 제2화소들의 위치(

)는 상기 수학식 4와 같이 표현될 수 있다. 그 후에는 상기 제2화소들 각각에 대한 밝기값을 포함하는 제2화소들의 집합(

)을 구하는 단계(s230)를 수행하도록 하는 것이 바람직한데, 상기 제2화소들의 집합(

)은 상기 수학식 6에 의하여 구하는 것이 더욱 바람직하다.Meanwhile, a candidate feature point for a second image having a larger scale than the first image (

) Is performed, the second image refers to an'large-scale image' in the above description. And candidate features (

) Is a feature point for the second image (

) Refers to a candidate point for finding and has been referred to as a candidate correspondence point in the above description, and the feature coordinate (which is a feature point of the first image)

). The candidate feature points (

After determining ), the candidate feature points (

)

, The angle

Location of the second pixels (

) To determine each step (s220), the position of the second pixels (

) May be expressed as Equation 4 above. After that, a set of second pixels including a brightness value for each of the second pixels (

It is preferable to perform the step (s230) of obtaining the set of the second pixels (

) Is more preferably obtained by the above equation (6).

)과 상기 제2화소들의 집합(

)이 구해진 후에는 상기 제1화소들의 집합(

)과 상기 제2화소들의 집합(

) 사이의 상호정보량(

)를 구하는 단계(s310)을 수행하도록 하는 것이 바람직한데, 상기 상호정보량(

)은 상기 수학식 8에 의하여 구하도록 하는 것이 바람직하다. 상기 상호정보량(

) 산출은 상기 제2화소들의 위치(

)를 계속하여 갱신해 가면서(s320) 상기 제2화소들의 집합(

)을 재구성한 후 상기 상호정보량(

) 산출을 반복하도록 하는 것이 바람직한데, 미리 정해진 조건에 따라 더 이상 갱신할 수 없을 때 까지(종료조건 충족 시까지) 반복하도록 하는 것이 더욱 바람직하다(s330). 여기서 상기 제2화소들의 위치(

)에 대한 계속적인 갱신(s320)은 (a) 상기 제2영상의 스케일범위 내에서 각각의 스케일(

)을 순차적으로 적용해가면서 상기 제2화소들의 위치(

)를 갱신하고, (b) 상기 제2화소들의 각도

를 일정각도(

)만큼 순차적으로 증가시키면서 상기 제2화소들의 위치(

)를 갱신하고, (c) 상기 후보특징점(

)에 대하여 일정한 탐색범위 내에서 이동변위(

)를 적용해가면서 상기 제2화소들의 위치(

)를 갱신하도록 하는 것이 바람직하다(s331). The first set of pixels (

) And the set of second pixels (

After the) is obtained, the first set of pixels (

) And the set of second pixels (

) Amount of mutual information

It is preferable to perform the step of obtaining (s310), the amount of mutual information (

) Is preferably determined by Equation (8). The amount of mutual information (

) Calculate the location of the second pixels (

) While continuously updating (s320) the set of the second pixels (

After reconstructing ), the amount of mutual information (

) It is preferable to repeat the calculation, and it is more preferable to repeat until it can no longer be updated according to a predetermined condition (until the end condition is satisfied) (s330). Here, the position of the second pixels (

) Is continuously updated (s320) (a) each scale within the scale range of the second image (

) While sequentially applying the positions of the second pixels (

), and (b) the angles of the second pixels.

The constant angle (

) While sequentially increasing the positions of the second pixels (

), and (c) the candidate feature points (

), the displacement within a certain search range (

) While applying the position of the second pixels (

) Is preferably updated (s331).

) 중에서 상기 상호정보량(

)이 최대일 때의 파라미터(

)를 구하도록 하는 것이 바람직한데(s340), 상기 상호정보량(

)이 최대일 때의 파라미터(

)는 상기 수학식 9에 의하여 산출하는 것이 바람직하다. 그리고 상기 파라미터(

)를 적용한 어파인 변환계수를 이용하여 상기 특징좌표(

)에 대응되는 상기 제2영상에서의 특징점(

)을 구하도록 하는 것이 바람직한데(s350), 상기 제2영상에서의 특징점(

)은 상기 수학식 2에 의하여 구할 수 있을 것이다. After that, the amount of mutual information calculated in step s310 (

), the amount of mutual information (

) Is the maximum parameter (

It is preferable to obtain (s340), the amount of mutual information (

) Is the maximum parameter (

) Is preferably calculated by Equation (9) above. And the parameter (

) Using the affine transformation coefficient to which the above is applied.

) Corresponding to the feature point in the second image (

It is desirable to obtain (s350), the feature point in the second image (

) May be obtained by Equation 2 above.

Hereinafter, the present invention will be further described through experimental examples conducted to verify the present invention.

Experimental Example

<Generate the subject image>

의 범위가 1.2~3.0이 되도록 Kompsat-3의 해상도를 변환하고 회전각도

={0°, 90°, 180°, 270°}인 4종류의 영상을 생성한 후에 10%의 균일잡음(uniform noise)를 추가하여 총 40장의 작은 스케일 영상을 사용하였다. 두 번째 B 유형은 넒은 범위의 스케일 팩터(2.0≤

≤10.0)와 회전각도를 적용한 총 36장의 작은 스케일 영상을 사용하였다. 세 번째 C 유형은 동일한 지역을 촬영한 Kompsat-2, Kompsat-3, Kompsat-3A 영상을 조합하여 사용하였다. In the present invention, the performance of the above-described algorithm was verified using three types of image data as shown in Table 1. First, the A type uses the Kompsat-3 video as a large-scale video, and the scale factor

Convert the resolution of Kompsat-3 so that the range of the range is from 1.2 to 3.0, and the rotation angle

After generating 4 types of images, ={0°, 90°, 180°, 270°}, a total of 40 small scale images were used by adding 10% of uniform noise. The second type B has a wide range of scale factors (2.0≤

≤10.0) and a total of 36 small scale images using rotation angles were used. The third type C was a combination of Kompsat-2, Kompsat-3, and Kompsat-3A images of the same region.

The experimental images used in the present invention are multi-purpose satellites No. 2, No. 3, and No. 3A of Niger's Niamey, which have spatial resolutions of 1.0 m, 0.7 m, and 0.55 m, respectively (see FIG. 8). Figure 8 (a) is a Kompsat-2 image taken on April 18, 2009, Figure 8 (b) is a Kompsat-3 image taken on January 21, 2013, Figure 8 (c) is 2015 This is a Kompsat-3A video shot on November 29th.

=2로 변환하고, 10%의 균일잡음을 추가한 것이며, 도 10(c)는 스케일 팩터

=3으로 변환하고, 10%의 균일잡음을 추가한 상태에서 90도를 회전한 것이며, 9(d)는 스케일 팩터

=3으로 변환하고, 10%의 균일잡음을 추가한 상태에서 180도 회전한 것이다.In order to evaluate the performance of the method according to the present invention, after detecting a control point (actual reference point) at which a corner response is maximized using a geometric average-based corner detector as shown in FIG. 9 for A type Kompsat-3 images, in FIG. The Kompsat-3 simulation image with noise components added while having various scale factors and rotational components was created. FIG. 9 shows the result of corner detection for the Kompsat-3 image. FIG. 9(a) is a Kompsat-3 panchromatic image, and FIG. 9(b) shows the corners detected by the geometric-based corner detector in red circles. . The center of the cross-shaped circle is the point where the corner response becomes the maximum, and is used as a control point (feature point) when estimating affine parameters. And Figure 10 is a test set of Kompsat-3 images, Figure 10 (a) is the original image, Figure 10 (b) is a scale factor

=2, 10% uniform noise is added, and Fig. 10(c) is a scale factor.

Converted to =3, rotated 90 degrees with 10% uniform noise added, and 9(d) is a scale factor

Converted to =3, and rotated 180 degrees with 10% uniform noise added.

<Experiment contents and results>

)를 구하기 위하여 반지름 R=300인 원형템플릿에 대해,

=100,

=72인 원형템플릿을 스케일이 작은 Kompsat-3 영상의 제어점(특징점) 위에 생성하였다. 스케일이 큰 영상인 Kompsat-3 영상의 제어점(특징점) 후보는 그레디언트 히스토그램의 상위 임계비율

=0.05 이상인 화소로 한정하였다. 스케일 팩터

는 0.1 간격으로, 템플릿의 회전각도

는 0.1도 간격으로 변화시키면서 다양한 조건에서 생성된 각 영상과의 상호정보량 유사도가 최대가 될 때의 스케일 팩터

, 회전량

, 정합위치(

)를 각각 구하였다. Table 2는 다양한 스케일 팩터 및 회전각 조건에서 검출한 스케일 팩터를 보여주며 총 40가지 실험 조건에서 스케일 팩터를 모두 정확하게 검출하였다. Table 3는 스케일이 작은 영상과 스케일이 큰 영상의 상대적인 회전각들을 모든 스케일에서 정확하게 검출한 결과를 보여준다. As shown in Table 2, the scale factor was generated at 0.2 intervals from 1.2 to 3.0, and rotational components were generated at 0 degrees, 90 degrees, 180 degrees, and 270 degrees. The noise component added uniform noise of 10% to all generated images. Affine parameters between Kompsat-3 images and various Kompsat-3 images with small scales (

For a circular template with radius R=300,

=100,

A circular template with =72 was created on the control point (feature point) of the small scale Kompsat-3 image. The control point (feature point) candidate for the Kompsat-3 image, which is a large-scale image, is the upper threshold ratio of the gradient histogram.

=0.05 or more. Scale factor

Is 0.1 interval, the rotation angle of the template

Is a scale factor when the degree of similarity of the amount of mutual information with each image generated under various conditions is maximized while changing at 0.1 degree intervals.

, Rotation amount

, Registration position (

) Respectively. Table 2 shows the scale factors detected under various scale factors and rotation angle conditions. All scale factors were accurately detected under a total of 40 experimental conditions. Table 3 shows the results of accurately detecting the relative rotation angles of the small-scale image and the large-scale image at all scales.

=3인 경우, 스케일이 큰 영상에서 3×3 크기의 화소 영역은 스케일이 작은 영상에서 1화소 크기의 영역에 해당이 된다. 따라서 스케일이 큰 영상에서의 위치 오차는 스케일 팩터 증가에 따라 비례적으로 증가하기 때문에 위치 오차 분석에서 스케일 팩터의 영향을 배제하기 위하여 평균 오차와 스케일 팩터의 비(ratio)를 구하였다. 이 값은 스케일이 작은 영상 관점에서의 위치 오차로 이해할 수 있으며 1.2~3.0 사이의 스케일 팩터에 대해 0.42~0.69 화소 범위의 우수한 위치 오차 성능을 보였다. Table 4 below shows the position error between the detected control point (feature point) and the actual reference point. The average values of the row and column errors of the detected control points (feature points) were respectively obtained, and the average error results in the range of 0.50 to 1.62 pixels were obtained for the four rotation angles of each scale. As the scale factor increases, the relative scale difference between the two images increases, so the scale factor

When =3, a 3×3 pixel area in a large scale image corresponds to a 1 pixel size area in a small scale image. Therefore, since the position error in an image with a large scale increases proportionally with an increase in the scale factor, the ratio of the average error and the scale factor is calculated to exclude the influence of the scale factor in the analysis of the position error. This value can be understood as a position error from the viewpoint of a small scale image, and showed excellent position error performance in a range of 0.42 to 0.96 pixels for a scale factor between 1.2 and 3.0.

가 0도 및 90도일 때, 스케일 팩터

가 0.05 간격으로 변할 때의 상호정보량의 변화를 보여준다. 상호정보량의 최고값은 인접한 스케일 팩터

를 포함하여 다른 스케일 팩터의 상호정보량 과 비교해서 회전량

가 0도 및 90도인 두 경우 모두 큰 차이가 남을 볼 수 있다. 도 11(b)는 스케일 팩터

=2.0일 때, 회전량

가 변할 때의 상호정보량의 변화를 보여준다. 회전량

가 90도일 때 상호정보량이 최대가 되고 그 외의 다른 각도에서의 상호정보량과 그 차이가 뚜렷함을 알 수 있다. Meanwhile, FIG. 11 illustrates a change in the amount of mutual information according to a change in the rotation angle and scale factor. In FIG. 11(a), the rotation amount of a large scale image for a small scale image

Scale factor when 0 and 90 degrees

Shows the change in the amount of mutual information when it changes at 0.05 intervals. The highest value of the mutual information amount is the adjacent scale factor

The amount of rotation compared to the mutual information amount of other scale factors including

In both cases where is 0 degrees and 90 degrees, a large difference can be seen. Figure 11 (b) is a scale factor

= 2.0, rotation amount

Shows the change in the amount of mutual information when is changed. Rotation

It can be seen that when is 90 degrees, the amount of mutual information becomes the maximum, and the difference from the amount of mutual information at other angles is distinct.

≤10.0 사이의 스케일 팩터와 4가지 회전각도

={0,90,180,270}를 적용한 유형 B의 저해상도 영상을 사용하였다. 도 12는 스케일 팩터가 2.0, 4.0, 6.0, 8.0, 10.0이고 회전 성분을 포함한 다양한 저해상도 영상의 예를 보여준다. 도 12(a)는 원본영상이며, 도 12(b)는 스케일 팩터를 2.0으로 하고, 회전각도를 90도로 한 것이며, 도 12(c)는 스케일 팩터를 4.0으로 하고, 회전각도를 180도로 한 것이며, 도 12(d)는 스케일 팩터를 6.0으로 하고, 회전각도를 270도로 한 것이며, 도 12(e)는 스케일 팩터를 8.0으로 하고, 회전각도를 90도로 한 것이며, 도 12(f)는 스케일 팩터를 10.0으로 하고, 회전각도를 180도로 한 것이다.2.0≤ for experiments with a wider range of scale factors

Scale factor between ≤10.0 and 4 rotation angles

A low resolution image of Type B to which ={0,90,180,270} was applied was used. 12 shows examples of various low-resolution images with scale factors of 2.0, 4.0, 6.0, 8.0, 10.0 and rotational components. 12(a) is an original image, FIG. 12(b) is a scale factor of 2.0, and the rotation angle is 90 degrees, and FIG. 12(c) is a scale factor of 4.0, and the rotation angle is 180 degrees. 12(d) is a scale factor of 6.0, the rotation angle is 270 degrees, FIG. 12(e) is a scale factor of 8.0, the rotation angle is 90 degrees, and FIG. 12(f) is The scale factor is 10.0 and the rotation angle is 180 degrees.

=10.0인 경우에도 평균 위치 오차와 스케일 팩터의 비가 0.24의 낮은 수준을 보였다. 이는 제안한 방법이 스케일이 작은 영상 관점에서 두 영상의 스케일이 2~10배 정도 차이가 나더라도 0.24~0.42 화소 수준의 높은 위치 정확도를 가짐을 보여준다. When applying the method according to the present invention, as shown in Table 5 below, both the scale factor and the rotation angle conditions were accurately detected. The rotation angle between the two comparison images was also accurately detected under all conditions as shown in Table 6. Table 7 shows the position error of the detected control point (feature point) with the actual reference point, and showed the position error of 0.5 to 2.38 pixels within the scale range. The ratio of the average position error considering the scale and the scale factor ranged from 0.24 to 0.42, and the scale factor

Even at =10.0, the ratio of the average position error and the scale factor showed a low level of 0.24. This shows that the proposed method has a high positioning accuracy of 0.24 to 0.42 pixels even if the scale of the two images differs by 2 to 10 times from the viewpoint of the image with a small scale.

On the other hand, Fig. 13 is a third C type image combination of the Kompsat-3 image and the Kompsat-2 image (Fig. 13(a)), the Kompsat-3A image and the Kompsat-2 image (Fig. 13(b)), and the Kompsat-3A image. And Kompsat-3 images (FIG. 13(c)) show the change in the amount of mutual information according to the change in the scale factor. When the unit of change of the scale factor is 0.01, the scale factors when the amount of mutual information becomes the maximum in each image combination are 1.43, 1.79, and 1.25, respectively. Table 8 to Table 10 show the results for each image combination when the scale factor change unit is more precise 0.001. When the scale factor changes to 0.001 units, the scale factor where the mutual information amount is the largest appears over several values, and the last detected scale factor takes the average value of the scale factors where the mutual information amount is the maximum and rounds the value of the fourth decimal place. Was estimated. In the experiment of the combination of the Kompsat-3 image and the Kompsat-2 image, the scale factor in which the amount of mutual information is the largest was found in the range of 1.427 to 1.429, and the estimated scale factor as shown in Table 8 was 1.428, which is the same as the reference scale factor. The relative rotation angle error of the two images was 0.2 degrees, and the position error of the control point (feature point) was 1 pixel error in the row and column directions. In the experiment of the combination of the Kompsat-3A image and the Kompsat-2 image, the scale factor that maximizes the amount of mutual information was found in the range of 1.790 to 1.793. The estimated scale factor of 1.792 showed an error of 0.004 from the reference scale factor. The rotation angle error was 0.3 degrees, and the position of the reference point corresponding to the control point (feature point) in the row and column direction was accurately detected (see Table 9). In the experiment of the combination of the Kompsat-3A image and the Kompsat-3 image, the scale factor where the maximum amount of mutual information was the maximum was in the range of 1.254 to 1.257, and the estimated scale factor of 1.256 showed an error of 0.004 with the reference scale factor. The rotation angle error was 0.1 degrees, and the position of the reference point corresponding to the control point (feature point) in the row and column direction was accurately detected (see Table 10).

를 3화소 단위로, 회전각도

를 5도 간격으로 샘플링하여

=100,

=72로 총 7,200개의 화소 밝기값이 유사도 계산에 사용되었다. 반지름 R=300인 원형템플릿이 실제로 포함하는 영역의 이론적인 면적은 대략 300 x 300 x 3.14 = 282,600화소로 일반적인 특징정합 방식에서 사용되는 특징의 수에 비해 매우 넓은 영역에 분포하는 화소들이 유사도 계산의 특징으로 사용이 된다. 따라서 유사도 비교시 특징점 주변의 좁은 영역이 아닌 넓은 영역에 걸쳐서 유사도 비교가 수행이 되기 때문에 일부 영역에서 정합 성능을 떨어뜨리는 요인이 작용하더라도 전체 유사도 계산 과정에서의 영향은 미미하게 작용하고 오정합 가능성을 크게 낮추는 효과를 얻을 수 있다. 넓은 영역을 포함하는 원형템플릿의 이러한 특징은 스케일 팩터와 회전각도를 매우 정밀하게 조절할 수 있도록 한다. 또한 상호정보량 기반의 유사도 측정 방식은 화소의 밝기값을 직접 비교하는 상관(correlation) 관계 기반의 유사도 측정 방식과 달리 두 원형템플릿에 포함되는 화소들을 통계적으로 분석한다. 따라서 제안하는 방법은 좁은 영역에 대한 상호정보량 기반의 유사도 측정 대신에 넓은 영역에 대한 상호정보량 기반의 유사도 측정 방식을 적용하기 때문에 비교 대상이 되는 두 원형템플릿 내부 영역에서 나타나는 미세한 차이를 넓은 영역에 걸쳐 통계적으로 측정할 수 있는 장점을 가진다. The high accuracy of the scale factor, rotation angle, and feature point location obtained from the experimental results is closely related to the size and similarity calculation method of the circular template. The radius R of the circular template used in the experiment is 300 pixels.

Is in 3 pixels, rotation angle

Sampling at 5 degree intervals

=100,

=72, a total of 7,200 pixel brightness values were used in the similarity calculation. The theoretical area of the area actually included in the circular template with a radius of R=300 is approximately 300 x 300 x 3.14 = 282,600 pixels, so that pixels distributed over a very large area are compared with the number of features used in the general feature matching method. It is used as a feature. Therefore, when comparing similarity, similarity comparison is performed over a wide area rather than a narrow area around the feature point. Even if a factor that decreases the matching performance is applied in some areas, the effect in the overall similarity calculation process is negligible and the possibility of mismatch The effect of significantly lowering can be obtained. This feature of a circular template with a large area allows for very precise adjustment of the scale factor and angle of rotation. In addition, the similarity measurement method based on the mutual information amount statistically analyzes pixels included in two circular templates, unlike the correlation-based similarity measurement method that directly compares the brightness values of pixels. Therefore, the proposed method applies the similarity measurement method based on the mutual information amount for a large area instead of the similarity measurement based on the mutual information amount for a narrow area. It has the advantage of being statistically measurable.

<Summary of Experiment Results>

After creating Kompsat-3 images with a large scale and creating a Kompsat-3 image with a small scale so that the scale factor ranges from 1.2 to 3.0, a total of 40 experimental conditions applying 4 types of rotation angle and 10% uniform noise In, both the scale factor and the relative rotation angle were accurately detected, and excellent position error results with a ratio of the average error of the position of the small scale image perspective and the scale factor of 0.42 to 0.96 were obtained. Scale factors and relative rotation angles were accurately detected even in a total of 36 experimental conditions that applied 4 kinds of rotation angle conditions between 2.0 and 10.0, which have a wider scale factor range, and the ratio of average position error and scale factor considering scale is 0.24 to The result was in the 0.42 pixel range. In experiments on multipurpose utility satellite images 2, 3, and 3A, which have different spatial resolutions, the scale factor error is 0.004 or less, the rotation angle error is 0.3 degrees or less, and the control point (feature point) position error is 1 pixel or less. Matching performance results were obtained.

Although the present invention has been described in various examples, the present invention is not necessarily limited to these examples, and may be variously modified without departing from the spirit of the present invention. Therefore, the examples disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these examples. The scope of protection of the present invention should be interpreted by the claims below, and all technical ideas within the equivalent range should be interpreted as being included in the scope of the present invention.

Claims (5)

As a method of performing feature matching for multi-resolution image registration by an information system,

About the first video,
(a) Characteristic coordinates (

Detecting);
(b) The characteristic coordinates (

Determining a circular template having a radius R around );
(c) the feature coordinates (

Centered on)

, The angle

Location of the first pixels (

Respectively);
(d) a set of first pixels including brightness values for each of the first pixels (

);

For the second image having a larger scale than the first image,
(a) Candidate features (

Determining);
(b) The candidate feature points (

)

, The angle

Location of the second pixels (

Respectively);
(c) a set of second pixels including brightness values for each of the second pixels (

);

The position of the second pixels (

) Is updated as shown below and the second set of pixels (

) After repeating the process of reconstructing each of the first set of pixels (

) And the set of second pixels (

) Amount of mutual information

);
(a) Each scale within the scale range of the second image (

) While sequentially applying the positions of the second pixels (

Update)
(b) the angles of the second pixels

The constant angle (

) While sequentially increasing the positions of the second pixels (

Update)
(c) The candidate feature points (

), the displacement within a certain search range (

) While applying the position of the second pixels (

Update)

The amount of mutual information (

) Is the maximum value parameter (

); And
The above parameter (

) Using the affine transformation coefficient to which the above is applied.

) Corresponding to the feature point in the second image (

); Characterized in that, characterized in that the registration method using a variable circular template when registering a multi-resolution image According to claim 1,
The characteristic coordinates (

) Is a feature matching method using a variable circular template when registering a multi-resolution image, characterized in that the position of the corner point at which the corner response value is the maximum within a certain range is detected using a feature point detector based on a geometric mean. According to claim 1,
The positions of the first pixels (

) Is obtained by the following equation,

(

Is the total number of pixels located above the radius of the circular template,

Is the total number of pixels located on the circumference of the circular template)
The position of the second pixels (

) Is a feature matching method using a variable circular template when registering a multi-resolution image, which is obtained by the following equation.

(

Is a scale parameter reflecting the scale size of the first image and the second image) According to claim 1,
The first set of pixels (

) And the set of second pixels (

) Is a feature matching method using variable circular templates when registering multi-resolution images,

(

Is the first image (

) The brightness value of the pixel at the location)

(

Is the second image (

) The brightness value of the pixel at the location) According to claim 1,
The amount of mutual information (

) Is a feature matching method using a variable circular template when registering a multi-resolution image, which is obtained by the following equation.

(H is entropy)

Images