KR101279484B1 - Apparatus and method for processing image - Google Patents

Abstract

PURPOSE: An image processing apparatus and a method thereof are provided to expand a Euclidean model using an interest point extracting technique based on a similar shape model without drastic correction. CONSTITUTION: An image processing unit(130) respectively generates an L-scale space expanding or minimizing a scale about two images and generates a normalized local patch by resampling a local patch of the L-scale space with a predetermined ratio. The image processing unit extracts an interest point by applying an interest point function to the normalized local patch and generating an M-scale space. [Reference numerals] (110) Input unit; (120) Database; (130) Image processing unit; (140) Output unit

Description

Image processing apparatus and method {APPARATUS AND METHOD FOR PROCESSING IMAGE}

The present invention relates to an image processing apparatus and method, and more particularly, to an image processing apparatus and method for extracting a point of interest of an image.

The image interest point technique is a technique for extracting corner points and symmetry points from an image. Image search, image registration, and image panorama are performed using this method of interest. Conventional techniques include a Euclidean model based point of interest extraction technique (harris corner detector, KLT corner detector, GST) that is robust against rotation, movement, and noise, and a similar model based point of interest extraction technique that is robust against rotation, movement, scale, and noise. (LoG, Harris-Laplacian, Hessian-Laplacian, SIFT, SURF). Another model is the point-of-interest extraction technique (Affine-point, ASIFT) that is robust to affine transformation.

The easiest to develop is based on the Euclidean model. Besides, due to the difficulty of development, many kinds of interest extraction techniques have not been developed. In particular, the similar model-based interest extraction technique is more useful and efficient than other models in the small-size interest scheme. The key to the point of interest technique is the design of the point of interest function, which calculates the strength of the point of interest. In the similar model, the conventional techniques have many forms of extending the existing Euclidean model-based point of interest extraction technique that adds scale normalization terms to the scale space method for simulating scale.

However, the biggest problem in extending the form transformation model that resembles the existing Euclidean model-based point of interest extraction is the normalization term processing. This is because the intensity value of the point of interest function is dependent on the size of the local area, which is an input value of the point of interest function, when the point of interest is extracted by applying the scale space technique.

Accordingly, an object of the present invention is to provide an image processing apparatus and method capable of extracting more types of points of interest by extending the Euclidean model based point of interest extraction method to a similar model based point of interest extraction method without significant modification. will be.

In order to solve this problem, an image processing apparatus for matching a point of interest from two images according to an embodiment of the present invention generates an L-scale space in which scales are enlarged or reduced for the two images, respectively. An image processor extracting a point of interest by generating a normalized local patch by resampling a local patch of the L-scale space using a predetermined magnification and generating an M-scale space by applying a point of interest function to the normalized local patch. It includes.

The image processing unit may extract a plurality of poles from the M-scale space, group the extracted plurality of poles, and extract points of interest of the two images in preparation for the grouped poles.

The image processing unit groups the extracted poles while traversing the extracted poles from an upper level to a lower level of the M-scale space, and obtains representative positions, representative intensities, and scale persistences of the grouped poles. It can be set as the representative value of the pole.

The image processor extracts a local descriptor that is a feature vector of a local patch around the pole, measures a distance between local descriptors of the two images, and generates a corresponding relationship between the grouped poles of the two images according to the measured distance. can do.

The image processor may match the points of interest of the two images according to the number of correspondence relations of the grouped poles.

The image processor may generate a similarity table according to a corresponding relationship between the grouped poles of the two images and match the points of interest of the two images based on the similarity table.

The image processor may compare the degree of correspondence between the grouped poles with a threshold and match the grouped poles having a degree of correspondence larger than the threshold with the points of interest of the two images.

The L-scale space may generate two times reduction and / or expansion between each octave each time a scale space is generated.

The function of interest calculates the intensity for the normalized local patch and may be any one of a Harris corner detector, a Kanade-Lucas-Tomasi (KLT) detector, and a generalized symmetric transform (GST).

By normalizing the local patch of the L-scale space in proportion to the scale, a normalized local patch normalized to P × P can be generated.

According to another aspect of the present invention, there is provided an image processing method for matching a point of interest from two images, the method comprising: generating an L-scale space in which the scale is enlarged or reduced for each of the two images; Resampling using a predetermined magnification for the local patch to generate a normalized local patch, and extracting the interest by generating an M-scale space by applying a point of interest function to the normalized local patch.

The extracting the points of interest may include extracting a plurality of poles from the M-scale space, grouping the extracted plurality of poles, and extracting the points of interest of the two images in preparation for the grouped poles. It may include.

The pole grouping step may be performed by grouping the extracted poles while traversing the extracted poles from a higher level to a lower level of the M-scale space, and obtaining representative positions, representative strengths, and scale persistence of the grouped poles. And setting the representative value of the pole.

Extracting a local descriptor that is a feature vector of a local patch around the pole, and measuring a distance between local descriptors of the two images to generate a corresponding relationship between the grouped poles of the two images according to the measured distance It may further comprise a step.

The method may further include matching the points of interest of the two images according to the number of correspondence relations of the grouped poles.

The method may further include generating a similarity table according to the correspondence relation of the grouped poles of the two images, and matching the points of interest of the two images based on the similarity table.

The method may further include matching the grouped poles having a degree of correspondence greater than the threshold with the points of interest of the two images by comparing the degree of correspondence of the grouped poles with a threshold.

A computer-readable medium according to another embodiment of the present invention records a program for executing any one of the above methods.

According to the present invention, more various types of points of interest can be easily extracted by extending the Euclidean model based point of interest extraction technique to a similar model based point of interest extraction technique without significant modification.

In addition, the various extracted points of interest may widen the range of application of the point of interest image matching, and may overcome the normalization of scale factors, which is a fundamental problem when the Euclidean transform model is extended.

1 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of the present invention.
2 is a schematic diagram illustrating an image processing process performed by an image processing apparatus according to an exemplary embodiment of the present invention.
3 is a schematic diagram of a pyramid L-scale space according to an embodiment of the invention.
4 is a schematic diagram illustrating a process of generating an M-scale space according to an embodiment of the present invention.
5 is a schematic diagram illustrating a pole grouping process according to an embodiment of the present invention.
6 is an EPR model diagram according to an embodiment of the present invention.
7 is a schematic diagram illustrating an EPR bowling process according to an embodiment of the present invention.

In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. Unless otherwise indicated in the context, like reference numerals in the drawings generally refer to like elements. The illustrative embodiments described in the detailed description, drawings, and claims are not meant to be limiting. Other embodiments may be utilized and other changes may be made without departing from the spirit or scope presented herein. The components herein can be arranged, substituted, combined, and designed in different configurations within a wide variety of different configurations, as generally described herein and shown in the figures, all of which are clearly It will be readily understood that it has been considered and forms part of this application.

Next, an image processing apparatus according to an exemplary embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of the present invention.

Referring to FIG. 1, an image processing apparatus 100 according to an exemplary embodiment of the present invention includes an input unit 110, a database 120, an image processor 130, and an output unit 140.

The input unit 110 receives image data from an external device (not shown), transmits the data into the image processing apparatus 100, and performs appropriate processing so that the image processing apparatus 100 may process the image data. In addition, the input unit 110 receives a command according to a user input so that the command can be transmitted to the image processing apparatus 100 and executed.

The database 120 stores image data related to image processing performed by the image processing apparatus 100. The image data is data transmitted from the input unit 100 or processed by the image processing apparatus 100.

The image processor 130 reads image data from the input unit 110 or the database 120 and extracts a point of interest (or a feature point) from the image. In addition, the image processor 130 may perform image search, image registration, and image panorama by using the extracted points of interest.

The output unit 140 exports the result processed by the image processing apparatus 100 to the outside.

Next, a method of extracting an image point of interest performed by the image processor 130 will be described in more detail.

2 is a schematic diagram illustrating an image processing process performed by an image processing apparatus according to an exemplary embodiment of the present invention.

As shown in FIG. 2, the image processor 130 performs image processing on two images (images A and B) through a feature extraction process and a feature matching process. The feature extraction process is a process of extracting local features from each image, and the feature matching process is a process of finding a corresponding relationship between two images based on local features. Local features include points of interest, local descriptors, and so forth. From a hierarchical processing structure point of view, based on the correspondence information (location of interest, local descriptor, etc.) that is the result of the interest matching process, subsequent processing, for example, model factor estimation, image retrieval, image registration, etc. , 3D reconstruction, and the like.

Points of interest generally refer to points of interest in the image, and points of interest can be specified, such as edges or corners of objects, and points of interest can also be outlines of closed regions. It can be a line or a point extracted from. For example, it may be a corner point, a symmetry point, an inflection point, or the like, and a line or a point may be used as a point of interest from a circular or rectangular outline.

The feature extraction process involves generating a pyramid linear scale space (L-scale space) that simulates scale changes from a two-dimensional image, and applies an interest function to the L-scale space to apply an M-scale. Process of creating space (measurement scale space, M-scale space), extracting poles (or points of interest) from the M-scale space, clustering poles and using Extreme Points and Route (EPR) And extracting the local descriptors of the poles in the EPR.

The feature matching process includes a local descriptor matching process for calculating the distance between all poles based on the local descriptor, and a voting table for obtaining a voting table that can numerically indicate the degree of correspondence between EPRs. And a process of generating a connection relationship between EPRs of two images by extracting EPRs having a strong correspondence by applying a threshold to the bowing table. Feature matching is a type of image matching that creates a linkage relationship between images (correspondence between points of interest, such as a transformation matrix between two images), and especially between images based on extracted local descriptors (feature vectors). It means creating a relationship.

Each process will be described in more detail with reference to the accompanying drawings.

3 is a schematic diagram of a pyramid L-scale space according to an embodiment of the invention.

The scale space method is a framework for expressing multi-scale signals developed in computer vision, image processing, signal processing, etc., so that images can be expressed at different scales (or resolutions) by blurring the image to a two-dimensional image. Pyramid scale space reduces the image by 2 times every 2 times to increase the performance when generating the 2D image in the scale space. That is, x1.0, x1.3, x1.5, x1.7, x1.9, x2 (downsampling the image at this time), and then again 1.0xx, x1.3, x1.5, x1 .7, x1.9 times Through this process, a scale space is created in a pyramid shape as shown in FIG. 3.

The pyramid L-scale space according to an embodiment of the present invention uses a scale space that is a three-dimensional image in which a scale axis is expanded from a two-dimensional image, which is spatial information, to extract a point of interest that is robust to a change in resolution. By using the kernel function of convolution, the lowpass filter is gradually stronger in the image, while generating a scale space of coarse scale level at the fine scale level. Handle the scale space. This lowpass filter simulates a scale change.

부터

배로 점차적으로 증가시키면서, 다음 [수학식 1]과 같은 컨볼루션 연산을 적용하여 영상을 점차적으로 흐리게 만든다.

는 보통 0.5~1.2 범위에서 선택한다. 또한 스케일 스페이스 방법의 계산 속도를 높이기 위하여, 2배가 되는 시점에 디지털적으로 확대/축소를 행한다. 이렇게 디지털적으로 처리를 하여 피라미드 L-스케일 스페이스를 생성한다.The specific method for generating the L-scale space is as follows. Early gaussian

from

The image is gradually blurred by applying a convolution operation as shown in [Equation 1] while gradually increasing the fold.

Is usually selected in the range 0.5 to 1.2. In addition, in order to increase the calculation speed of the scale space method, digital enlargement / reduction is performed at a time point that doubles. This digital processing produces the pyramid L-scale space.

[Equation 1]

Where (x, y) is the two-dimensional coordinates of the image, I is the image brightness at that coordinate, and G is a two-dimensional special Gaussian kernel.

As shown in FIG. 3, one octave is divided into two-fold units at the full scale for the simulation, and the pyramid structure reduces (or enlarges) the resolution by two times each time it becomes two-fold (or half-fold). Here, an octave refers to a set of images having the same level of image when constructing a pyramid scale space.

Next, a process of applying an interest point function to generate an M-scale space will be described with reference to FIG. 4. A point of interest function is a function that measures the intensity of a local patch. Usually, a very large value is more likely to be a point of interest.

4 is a schematic diagram illustrating a process of generating an M-scale space according to an embodiment of the present invention.

The M-scale space according to the embodiment of the present invention is an intensity scale space obtained by calculating a local patch of points in the L-scale space as a point of interest function. That is, the M-scale space is a space calculated by applying a point of interest function (μ: R → R → R) to each pixel having the possibility of corner, edge, blob in the L-scale space of the image. . To generate the M-scale space, the point of interest function [mu] satisfies the following axioms of similarity interest point function.

1) Transfer Constant:

2) Rotation Invariant:

3) size constant:

은 두 집합 A와 B는 육안으로 보았을 때 동일한 유사성을 가진다는 것을 의미하고, card()는 집합의 크기(size)이다.Here the sign

Means that both sets A and B have the same similarity with the naked eye, and card () is the size of the set.

또는

iff B가 A보다 관심점에 더 가깝다.4) Ordered group:

or

iff B is closer to the point of interest than A.

Many heuristic Euclidean points of interest functions in discrete space do not decimate if the scales are not integer multiples of different scales. You need to increase the size of a local patch, which means some small area (usually a square). For this reason, the size of the local patch needs a normalization factor according to the change. In general, points of interest functions that are mathematically well-designed by the Euclidean transform model can be easily developed by normalization. However, heuristic interest functions, such as generalized symmetric transform (GST) and fast radial symmetric transform (FRST), are difficult to find normalization elements. In other words, comparable comparisons are not possible because three times the similarity interest function axiom is violated (scale normalization problem). However, existing Euclidean point of interest functions are generally designed to allow comparisons between the same scale. Based on this design, the basic idea of how to solve the scale normalization problem is as follows. Neighboring points apply the point-of-interest function with the same number of pixels, so even if the resolutions are of different scales, re-sampling the number of different pixels to the same number of pixels before measuring the intensity, thus reducing the Euclidean point of interest function. If applied, scale normalized problems can be avoided. Therefore, the local patch is resampled in advance to obtain a normalized local patch whose resolution is normalized, and then the intensity interest function μ is applied.

Referring to FIG. 4, the process of applying a point of interest function calculates the strength in the L-scale space to generate an M-scale space having the point of interest intensity according to the scale. Here, the point of interest function is a process of calculating an input value of a local area and an output value of an intensity value, which may be the conventional Euclidean model based point of interest functions. In the present invention, the local region is normalized in proportion to the scale in advance in order to generate a point of interest function that is robust to the scale. The local patch normalized to P × P according to this scale proportionality is called a normalized local patch.

In FIG. 4, the two points on the L-scale space on the left are points derived from the same (x, y, t ₀ ) and the area shaded around the two points represents a local patch. The two normalized local patches in the middle are resampled to P × P sizes using 1 / s ₁ and 1 / s ₂ magnification, respectively. The two points on the M-scale space on the right side of FIG. 4 are intensity measured locations using the point of interest function μ for normalized local patches. Equation 2 applies M to the L-scale space by applying a normalization local patch function (ω: R × R × R → R × R) and a point of interest function (μ) to the P × P size. It is a modification of the scale space generation process.

&Quot; (2) "

Where P is a constant as the normalized local patch size, and t is an auxiliary variable for creating three-dimensional virtual coordinates as the scale axis. The point of interest function μ may be a Harris corner detector, a Kanade-Lucas-Tomasi (KLT) detector, a generalized symmetric transform (GST), or the like.

Next, a method of extracting the aforementioned pole set interest point (EPR) from the M-scale space will be described with reference to FIG. 5. 5 is a schematic diagram illustrating a pole grouping process according to an embodiment of the present invention. The pole is a point that is stronger than the surroundings in the image and is generally robust to distortion, and EPR is a collection of these poles. Generally, in order to extract poles, 32/8 neighbor poles are extracted in M-scale space and each pole is defined as one point of interest, but in the present invention, the poles simulated from one point are grouped into one point of interest. Extract the EPR.

Generally, there are 28-neighbor pole extraction methods and 8-neighbor pole extraction techniques. The present invention applies an 8-neighbor pole extraction technique. As shown in FIG. 5, the poles extracted at each scale level group the poles while traversing from the high level (coarest scale level) to the low level (finest scale level). Because of the pyramid structure, the coordinates are doubled between the different octaves in the grouping from the upper level to the lower level so that the grouping is not broken. The poles thus grouped are Extreme Points and Routs. The representative value of the EPR consists of a representative position, representative intensity (Rep) and scale persistence (nTrack). The representative position is called the representative position of the pole closest to the lowest level. Representative intensities can be the maximum, minimum, medium, or average among the grouped poles. Scale persistence can be defined as the number of poles, the maximum scale-minimum scale among the poles.

First, the intensity of the point of interest is calculated from the M-scale space as follows. As an example, in the present embodiment, a Harris corner detector is used as a point of interest function. An x-axis partial differential image (dx) and a y-axis partial differential image (dy) are generated for the normalized local patch (W), and the Harris corner intensities for the normalized local patch using the following [Equation 3] Calculate

&Quot; (3) "

이고, Det()는 행렬식(determinant), Tr()은 행렬의 대각 원소들의 합,

는 사용자 입력 변수로 일반적으로 0.04 ~ 0.15 사이 값으로 한다.Where matrix M _c is

Det () is a determinant, Tr () is the sum of the diagonal elements of the matrix,

Is a user input variable, and generally is a value between 0.04 and 0.15.

If a point of interest is continuously observed at each resolution in a multi-resolution spatial scale space, this point of interest will be robust even if the resolution changes. In other words, if the persistence of the point of interest increases in the scale space, the crossover is increased in the two images with different resolutions. If similar objects exist, the points of interest of the objects detected in one image are found again as points of interest in the other image. It is likely to be. As a result, the higher the persistence of the point of interest, the higher the repeatability of the point of interest on multiple resolutions.

Referring to FIG. 5, local pole detection, pole trajectory tracking between adjacent scales, and clustering of poles on the same track are performed for EPR extraction. The representative position, representative strength (Rep) and scale persistence (nTrack) of the EPR are then calculated. Since the representative position is usually shaken as the highest scale is processed, the coordinates of the pole close to the lowest scale are determined. The representative intensity is treated as the minimum, maximum, or average value among the M-scale space intensities of the tracked poles. Finally, scale persistence is defined as the minimum and maximum scale range of the poles.

은 동시의 관심점 위치

와 관심점 로컬 패치의 크기

를 결정시켜 준다. 극점들은 편의상

로 재표기한다. 여기서 아래 첨자 o는 스케일 스페이스 상의 옥타브 고유 번호, s는 한 옥타브 내의 스케일 고유 번호이고, 위 첨자 i는 (o, s) 내에 존재하는 극점들의 순서 번호이다.The pole detection method tests the pole with eight neighboring points at each t level of resolution. The detected pole

Points of interest at the same time

And points of interest local patch size

To determine. The poles are for convenience

Remark as Where the subscript o is the octave unique number on the scale space, s is the scale unique number in one octave, and the superscript i is the sequence number of the poles present in (o, s).

6 is an EPR model diagram according to an embodiment of the present invention. 5 and 6, clustering between adjacent poles in a top-down manner from a coarsest scale level to a bottom scale level on a scale space in order to group the poles extracted for each resolution. Proceed. M-scale space-based Extreme Points Clustering is performed in the following manner.

(1) pole clustering

for each oi = [C-1: 0], where oi is an octave number and C is the number of octaves

i) Clustering poles of different levels within an octave

for each si = [S-1: 1], where si is the scale number and S is the number of scales

① Assigns unique group number to unassigned poles at (oi, si) level

에 대하여 (oi, si-1) 레벨의 극점들

과

의 거리가 k이하인 점들을

의 그룹 번호로 할당.Connection of poles of (oi, si) and (oi, si-1) levels: of poles of all (oi, si) levels

Poles of (oi, si-1) level with respect to

and

Points at distances k or less

Assigned to the group number.

ii) clustering poles of different levels of octave

① Assign unique group number to unassigned poles at level (oi, 0)

에 대하여 (oi-1, S-1) 레벨의 극점들

과

의 거리가 2×k이하인 점들을

의 그룹 번호로 할당.② Pole connection of (oi, 0) and (oi-1, S-1) levels: of poles of all (oi, 0) levels

Poles at (oi-1, S-1) level

and

Points whose distance is less than 2 × k

Assigned to the group number.

(2) generation of pole orbit (EPR) information

집합으로 생성하고, 집합

내에서 대표 값

와 추적 회수

를 계산한다. 도면 상의 원 모양은 극점들의 윈도우 크기이다. 원 모양이 연속적으로 모여 있는 것이 EPR 모형도를 나타낸 것이다. 강인한 EPR을 추출하기 위하여 EPR의 대표 강도(Rep)와 스케일 지속성(nTrack)의 임계값을 설정할 수 있다. 대표 강도(Rep)의 임계값이 클수록 EPR(관심점)일 가능성이 클 것이고, 스케일 지속성(nTrack)의 임계값 클수록 스케일 지속성이 큰 EPR일 것이다.The poles with gid-th identical group number are one EPR.

Create as a set,

Representative value within

And tracking recall

. The circle on the figure is the window size of the poles. The continuous cluster of circles shows the EPR model. In order to extract robust EPR, threshold values of representative strength (Rep) and scale persistence (nTrack) of the EPR may be set. The larger the threshold of the representative intensity Rep, the greater the likelihood of an EPR (point of interest), and the larger the threshold of scale persistence (nTrack), the greater the EPR.

The following describes the local descriptor extraction process in detail.

의 M×M이다. L을 x축과 y축으로 편미분한다. 편미분 값을 이용하여 L의 지배적인 방향을 찾기 위하여, 36방향의 그래디언트 강도 히스토그램을 생성한다. 그리고 그래디언트 강도 히스토그램을 6번 반복하면서 부드럽게 만들어 주고, 로컬 극값들을 찾고, 최상위에 큰 극값에서 0.8배 작은 극값들은 삭제한다. 여기서 만약 E개의 살아남은 극값이 존재한다면, L의 지배적인 방향은

가 된다. L과 지배적인 방향 사이의 SIFT 로컬 디스크립터 추출은 다음 과정으로 시행된다.The local descriptor is a feature vector of the local patch, and may represent a feature of interest by using peripheral information of the image from the pole, and the size of the vector may be 64, 128, or 256 dimensions. Local descriptor extraction may use, for example, a scale invariant feature transform (SIFT) capable of extracting a point of interest from an image. First we obtain a normalized local patch for local descriptor extraction for SIFT local descriptor extraction. The size of the normalized local patch (hereafter referred to as L) for the local descriptor is

Is M × M. Differentiate L into the x and y axes. To find the dominant direction of L using partial derivatives, a gradient intensity histogram in 36 directions is created. It then smoothes the gradient intensity histogram six times, finds local extremes, and deletes the extremes 0.8 times smaller from the largest extreme at the top. Here, if there are E surviving extremes, the dominant direction of L

. Extraction of the SIFT local descriptor between L and the dominant direction is done by

function LocalDesciptors = ExtractDescriptor (L)

for i = 0: E

)R = imrotate (L,

)

R,

R, NumofSubBlock, NumofAngle)LocalDescriptors [i] = Extract_SIFT (

R,

R, NumofSubBlock, NumofAngle)

end

end

만큼 시계방향으로 회전하는 함수이고, Extract_SIFT는 매우 잘 알려진 SIFT의 로컬 디스크립터 추출 함수이며, NumOfSubBlock은 R을 x축과 y축으로 각각 나눌 서브 블록의 개수이고, NumOfAngle은 각각의 서브 블록 내에서 그래디언트 히스토그램을 생성할 각도의 개수이다.This process rotates L to call out the Extract_SIFT function to remove the rotation attribute to match the reference with other images. Where imrotate is centered on the input image L

Is a function that rotates clockwise, Extract_SIFT is a well-known local descriptor extraction function of SIFT, NumOfSubBlock is the number of subblocks to divide R into the x and y axes, and NumOfAngle is the gradient histogram within each subblock. The number of angles to generate.

Extract_SIFT function can be written as follows.

R,

R, NumofSubBlock, NumofAngle)function LocalDescriptor = Extract_SIFT (

R,

R, NumofSubBlock, NumofAngle)

LD = zeros (NumofSubBlock, NumofSubBlock, NumOfAngle)

R,2)width = size (

R, 2)

R,1)height = size (

R, 1)

for yi = 0: (height-1)

sbyi = NumofSubBlock * yi / height

for xi = 0: (width-1)

sbxi = NumofSubBlock * xi / width

R(yi,xi),

R(yi,xi))/360Theta = NumOfAngle * angle (

R (yi, xi),

R (yi, xi)) / 360

R(yi,xi),

R(yi,xi))Mag = magnitude (

R (yi, xi),

R (yi, xi))

for byi = 0: (NumofSubBlock-1)

for bxi = 0: (NumofSubBlock-1)

for ti = 0: (NumOfAngle-1)

weight = trilinear_weight (byi, bxi, ti, sbyi, sbxi, Theta)

LD (byi, bxi, ti) + = Mag * weight

end

end

end

end

end

LocalDescriptor <-enumerator_1D (LD)

end

Where LD is a three-dimensional matrix (indexes are zero-based), zeros is a three-dimensional matrix filled with zeros of all elements, and trilinear_weight is (sbyi, sbxi, Theta) in three-dimensional space (byi, bxi, ti). ) Is a linear weight calculation function that is close to 1 and close to 0 when angle is close, angle is the angle of the gradient, magnitude is the strength of the gradient, enumerator_1D is the column-first alignment of the input matrix, and LocalDescriptor is the local descriptor of L. .

The feature matching process consists of interest point matching, voting, and thresholding, and finally, corresponding points that are robust to the scale are extracted. The point-of-interest matching process performs a distance measurement process that fits the local descriptors of individual poles irrespective of the EPR to generate primary correspondence points in the one pole dimension closest to each other. 7 is a schematic diagram illustrating an EPR bowling process according to an embodiment of the present invention. The arrows between the poles of FIG. 7 are the correspondences of the poles generated based on the local descriptor. Points of interest and local descriptors are paired one by one, and the similarity between the two points of interest is calculated based on the local descriptors. If the similarity is greater than the threshold, a linkage relationship between the two points of interest is created; otherwise, no relationship is created.

The bowing process generates an EPR similarity table that indicates the poles correspondence between one EPR of one image and another EPR of another image. In the present invention, the values of the EPR similarity table may be the number of correspondences of the poles, the sum of the weights of the local descriptor distances of the poles, the average, the minimum, and the maximum.

In order to find the correspondence of two EPRs extracted from two images of different viewpoints, the correspondence of the EPRs is finally found by comparing the distances with local descriptors of element poles of the EPR (using a local descriptor of SIFT). The EPR correspondence process proceeds as follows.

집합,

집합)Function name: EPR_VOTING (

set,

set)

1) Extreme point matching

와

의 로컬 디스크립터 기반 거리 판단을 근거로, 두 쌍

을 찾는다. 여기서 로컬 디스크립터 기반 거리 측정 방법은 로컬 디스크립터를 SIFT로 사용하여 SIFT 매칭(matching)을 사용한다.All extreme

Wow

Two pairs, based on a local descriptor-based distance determination of

Find it. Here, the local descriptor based distance measurement method uses SIFT matching using a local descriptor as a SIFT.

2) Voting

들의 그룹 번호(윗첨자 gid, EPR의 고유 번호)에 가중시켜서 최종적으로 두 EPR의 대응 관계 정도를 수치적으로 찾는다. 일반적으로 EPR 보우팅 과정 중에서 해당 EPR에 가중치를 1로 적용하여, 유사도를 표현하는 EPR 유사도 테이블을 생성한다.Unique pair

Finally, the group number (superscript gid, unique number of the EPR) is weighted to find the degree of correspondence between the two EPRs. In general, the EPR similarity table representing the similarity is generated by applying a weight of 1 to the corresponding EPR in the EPR bowling process.

3) Thresholding

By applying the threshold (thVoting) the degree of correspondence between the two EPRs, only the EPRs with strong correspondences are extracted. Threshold processing is based on the threshold (thVoting) in the EPR similarity table. Thresholding process can finally generate a correspondence relationship between EPRs by applying any threshold value appropriate to the environment to the EPR similarity table. Here, the threshold application process may be said to be larger or smaller than the given threshold.

As such, the present invention uses a method of indirectly size normalizing the input local region rather than inserting the scale normalization term directly into the point of interest function. Thus, a model that is robust to the scale of the point of interest function of the existing Euclidean model technique without special modification is used. Easy to expand Therefore, more various types of points of interest extraction are possible, thereby extending the application area of the points of interest extraction technique.

Other embodiments of the invention include a computer readable medium having program instructions for performing various computer implemented operations. This medium records a program for executing the image processing method described so far. The medium may include program instructions, data files, data structures, etc., alone or in combination. Examples of such media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD and DVD, programmed instructions such as Floptical Disk and magneto-optical media, ROM, And a hardware device configured to store and execute the program. Or such medium may be a transmission medium, such as optical or metal lines, waveguides, etc., including a carrier wave that transmits a signal specifying a program command, data structure, or the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

Although the preferred embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concepts of the present invention defined in the following claims are also provided. It belongs to the scope of right.

100: image processing apparatus 110: input unit
120: database 130: image processing unit
140:

Claims (21)

An image processing apparatus for matching a point of interest from two images,
Generating an L-scale space each of which the scale is enlarged or reduced for the two images, resampling using a predetermined magnification with respect to a local patch of the L-scale space, to generate a normalized local patch, and the normalized local patch An image processor extracting a point of interest by generating an M-scale space by applying a point of interest function to
And the image processing apparatus. In claim 1,
The image processing unit extracts a plurality of poles from the M-scale space to group the extracted plurality of poles, and extracts the points of interest of the two images in preparation for the grouped poles. 3. The method of claim 2,
The image processing unit groups the extracted poles while traversing the extracted poles from an upper level to a lower level of the M-scale space, and obtains representative positions, representative intensities, and scale persistences of the grouped poles. An image processing apparatus that sets the representative value of the pole. 3. The method of claim 2,
The image processor extracts a local descriptor that is a feature vector of a local patch around the pole, measures a distance between local descriptors of the two images, and generates a corresponding relationship between the grouped poles of the two images according to the measured distance. Image processing device. 5. The method of claim 4,
And the image processor matches the points of interest of the two images based on the number of correspondences of the grouped poles. 5. The method of claim 4,
And the image processor generates a similarity table according to a corresponding relationship between the grouped poles of the two images and matches the points of interest of the two images based on the similarity table. 5. The method of claim 4,
And the image processing unit compares the degree of correspondence between the grouped poles with a threshold and matches the grouped poles having a degree of correspondence greater than the threshold with the points of interest of the two images. In claim 1,
And the L-scale space generates two times reduction and / or enlargement between each octave each time a scale space is generated. In claim 1,
The point of interest function calculates an intensity for the normalized local patch and is any one of a Harris corner detector, a Kanade-Lucas-Tomasi (KLT) detector, and a generalized symmetric transform (GST). In claim 1,
And a normalized local patch normalized to P × P by performing a normalization process on the local patch of the L-scale space in proportion to a scale. An image processing method of matching a point of interest from two images,
Generating L-scale spaces each of which the scale is enlarged or reduced for the two images;
Generating a normalized local patch by resampling the local patch of the L-scale space using a predetermined magnification; and
Extracting a point of interest by generating an M-scale space by applying a point of interest function to the normalized local patch
And an image processing method. 12. The method of claim 11,
The interest extraction step,
Extracting a plurality of poles from the M-scale space to group the extracted plurality of poles, and
Extracting the points of interest of the two images in preparation for the grouped poles
And an image processing method. The method of claim 12,
The pole grouping step may be performed by grouping the extracted poles while traversing the extracted poles from a higher level to a lower level of the M-scale space, and obtaining representative positions, representative strengths, and scale persistence of the grouped poles. An image processing method comprising the step of setting the representative value of the pole. The method of claim 12,
Extracting a local descriptor that is a feature vector of a local patch around the pole; and
Measuring a distance between local descriptors of the two images and generating a corresponding relationship between the grouped poles of the two images according to the measured distances
Further comprising the steps of: The method of claim 14,
And matching the points of interest of the two images according to the number of correspondences of the grouped poles. The method of claim 14,
Generating a similarity table according to a corresponding relationship between the grouped poles of the two images, and
Matching interests of the two images based on the similarity table
Further comprising the steps of: The method of claim 14,
And comparing the degree of correspondence of the grouped poles with a threshold and matching the grouped poles having a degree of correspondence larger than the threshold with the points of interest of the two images. 12. The method of claim 11,
And the L-scale space is reduced and / or enlarged twice in size between each octave each time a scale space is generated. 12. The method of claim 11,
The point of interest function calculates an intensity for the normalized local patch and is any one of a Harris corner detector, a Kanade-Lucas-Tomasi detector, and a generalized symmetric transform. 12. The method of claim 11,
And a normalized local patch normalized to P × P by performing a normalization process on the local patch of the L-scale space in proportion to a scale. A computer readable medium having recorded thereon a program for executing the method of claim 11.

Images