KR101511087B1 - Method for rotation invariant feature matching for the real-time processing from extracted features from images - Google Patents

Abstract

The present invention relates to a method for matching minutiae points in real-time whereby minutiae points in an image are not changed by rotation. More specifically, identical minutiae points are searched in different images based on minutiae points extracted from input images. The method includes a minutiae point extraction step, a descriptor generation step, and a descriptor comparison step.

Description

[0001] The present invention relates to a real-time feature point registration processing method and a real-time feature point registration processing method,

The present invention relates to a real-time feature point matching method having rotation invariance for feature points in an image, and more particularly to a real-time feature point matching method for finding identical feature points among different images based on feature points extracted from input images. .

A real-time feature point matching technique that finds identical feature points between different images based on feature points extracted from input images is generally performed by extracting feature points from an image to generate descriptors, extracting feature points from other images, and generating descriptors Later, a comparison between the technicians can identify the same object by looking for minutiae with the same descriptor.

Here, the minutiae are representative pixels capable of expressing the objects in the image, and the minutiae point extraction is to extract and extract pixels satisfying a specific condition in the image, the descriptor expresses various information about the minutiae, And comparing feature descriptors of feature points extracted from other images with each other to find feature points having the same descriptor (feature).

The feature points can be tracked if the movement amounts of the matching feature points are found. If the feature point extraction and matching are used in the camera image, the feature points can be sequentially found in the input image and the same feature points can be traced between the images.

In the feature point matching step, a descriptor is used to search for features having the same characteristics among features extracted from different images. In order to accurately locate feature points having the same characteristics, the descriptor includes various information. Since rotations occur frequently, especially for minutiae, the information about the rotation is described in the descriptor for comparison considering the rotating characteristics. However, engineers considering the rotation characteristics require a lot of effort to process well-known matching techniques in real time because complex operations are involved in the comparison process.

Korean Patent Laid-Open No. 10-2010-0125016 discloses a video matching apparatus and a video matching method thereof.

Korean Patent No. [10-2010-0125016]

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above-mentioned problems, and it is an object of the present invention to provide a method and apparatus for generating a simple descriptor having a rotation invariant characteristic, simplifying descriptors, And to provide a real-time feature point registration processing method having a rotation invariance with respect to feature points in an image.

A real-time feature point matching processing method having rotation invariance for feature points in an image according to the present invention includes a feature point extracting step (S10) for extracting a feature point (100) by receiving image information; (FCS-LBP: Fast Circular Shift-Local Binary Pattern) -based rotation invariant characteristic improved LBP (Local Binary Pattern) for the feature point 100 extracted in the feature point extracting step S10 A descriptor generating step (S20) of generating a descriptor (200) And a descriptor comparison step (S30) of receiving a different image and finding a minutiae point 100 having the same descriptor (200).

According to the real-time feature point matching method having the rotation invariance for the feature points in the image according to the embodiment of the present invention, by using a simple descriptor having the rotation invariant characteristic for the feature points, Can be performed more quickly.

Further, it is possible to generate descriptors more simply and quickly by addition, subtraction, and comparison operations of an integer type.

In addition, it is possible to generate descriptors including the features of the pixels around the feature points by designating the neighboring pixels around the feature points as different regions according to the distance.

In addition, when using a general color image as well as using a gray color, it is possible to compare feature points and surrounding pixels in real time even with a small implementation cost.

In addition, by obtaining the value to move the bits of the cyclic register using the corresponding table, a descriptor including the rotation invariant characteristic can be generated faster than a complicated calculation such as a trigonometric function.

In addition, feature points can be matched only by adding, subtracting, comparing, and exclusive OR (XOR) operations of an integer type, complementing the disadvantage that resolution is reduced by a degree of rotation by the first threshold, There is an effect that correction of another error occurring in the resolution supplemental to the degree of rotation by the rotation can be effected.

As a result, the calculation structure is simple, it is easy to design with a digital circuit, and it is easy to detect when the circuit is implemented, so that it is applicable to a real-time processing field.

1 to 3 are flowcharts of a real-time feature point matching processing method having rotation invariance for feature points in an image according to an embodiment of the present invention.
4 is a view showing a relationship between a feature point and surrounding pixels;
5 is a diagram showing a shift register generation relationship;
6 is a diagram showing engineer generation and structure;
Figure 7 is a diagram showing descriptor comparison logic;
8 is a diagram showing an example in which a direction characteristic of a feature point having a direction characteristic is changed;

Hereinafter, the present invention will be described in more detail with reference to the drawings. The following drawings are provided by way of example so that those skilled in the art can fully understand the spirit of the present invention. Therefore, the present invention is not limited to the following drawings, but may be embodied in other forms. In addition, like reference numerals designate like elements throughout the specification. It is to be noted that the same elements among the drawings are denoted by the same reference numerals whenever possible. Further, it is to be understood that, unless otherwise defined, technical terms and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Descriptions of known functions and configurations that may be unnecessarily blurred are omitted.

FIGS. 1 to 3 are flowcharts of a real-time feature point matching processing method having rotation invariance for feature points in an image according to an embodiment of the present invention, FIG. 4 is a diagram showing a relationship between a feature point and surrounding pixels, FIG. 6 is a diagram showing a descriptor creation and structure, FIG. 7 is a diagram showing descriptor comparison logic, and FIG. 8 is a diagram showing an example in which a direction characteristic of a minutia having a direction characteristic is changed to be.

The application field for tracking feature points is classified into feature point extraction technique for extracting feature points and feature point matching technique for finding feature points that match characteristics between extracted feature points. In detail, the feature point matching technique consists of a part to generate a descriptor to compare characteristics between extracted features and a part to compare the generated descriptor.

As shown in FIG. 1, a real-time feature point matching processing method having rotation invariance for feature points in an image according to an embodiment of the present invention includes a feature point extracting step (S10), a descriptor generating step (S20) S30).

The feature point extraction step (S10) receives the image information and extracts the feature point (100).

Generally, a feature point 100 is extracted by receiving an image through a camera or the like in real time.

The descriptor generating step S20 is a step of generating a descriptor representing a specific region with respect to the feature point 100 extracted in the feature point extracting step S10 by using a fast circular motion hand- FCS-LBP: Fast Circular Shift - Local Binary Pattern) to generate a descriptor 220 of a rotation invariant characteristic that represents a feature point.

And generates a descriptor for the extracted minutiae. The descriptor has intrinsic property information for each feature point and includes the degree of rotation for the feature points of the rotation invariant feature.

One of the factors that should be considered and considered mainly in the matching technique for checking whether the extracted minutiae can be matched is the rotation invariant characteristic for the minutiae. The rotation invariant is a characteristic that makes the extracted feature points recognized as the same feature points even if they rotate in other input images. In order to consider rotation-invariant matching, the degree of rotation for each feature point must be considered. However, considering the degree of rotation, complex operations are added to consider the rotation such as sin, cos, and tan rather than simple operations such as addition and multiplication in the calculation process. Therefore, In order to enable real-time matching, expensive equipment should be used. In order to simplify such a complicated matching technique, a descriptor 200 of a rotation invariant characteristic is generated based on a fast circular shift-local binary pattern (FCS-LBP).

2, the descriptor generating step S20 includes a region extracting step S21, a pixel defining step S22, a cyclic register generating step S23, a bit shifting step S24, and a descriptor determining step S25 ).

The region extracting step S21 extracts the concentric block 210 having the pixels P spaced apart by a predetermined pixel distance R about the feature point 100. [ The concentric block 210 consists of a circle having a certain pixel distance R from the center of the feature point and pixels P selected according to the user's choice among the pixels that meet the drawn circle. For example, as shown in FIG. 4, when a circle spaced by one pixel (R) from the center of the feature point is drawn, a total of eight pixels are selected, and A, B, C, D, E, F, The pixels of A, C, E, and G may be selected as well as the pixels of G may be selected. As shown in FIG. 4, the concentric block 210 includes eight pixels, which are spaced one pixel away from the center of the feature point, into a first area 211, twelve pixels that are two pixels apart from the center of the feature point, Area 212 and 16 pixels separated by 3 pixels from the center of the feature point.

In other words, the first area 211 includes A, B, C, D, E, F, G, and H in FIG. 3 and the starting point A and the direction of rotation can be arbitrarily determined and used. A, b, c, d, e, f, g, h, i, j, k and l in FIG. 3 and a and the rotation direction can be arbitrarily determined and used. ) Includes 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 and 15 in FIG. 3 and the starting point 0 and the direction of rotation are arbitrarily determined and used .

In addition, although the example of dividing the concentric blocks 210 of the fast recirculation matrixes into three is described, in order to improve the accuracy of the feature point matching, further regions such as the fourth region and the fifth region in which the value of the region (R) It is possible to further expand the application.

In addition, a descriptor including characteristics of pixels around the feature point can be generated by designating surrounding pixels around the feature point as different regions according to the distance.

The pixel defining step S22 compares the color with the surrounding pixels around the feature point 100

The following equation

(Where g (g _p - g _c ) is a value corresponding to the pixel, g _p is the color value of the surrounding pixels, and g _c is the color value of the center pixel)

A value (1 or 0) corresponding to the pixel is determined.

Here, the color value of the pixel defining step S22 may be a gray color.

Generally, a gray color has 256 values, and a value corresponding to a corresponding pixel can be designated as 0 or 1 by comparing the value with a center point (feature point). Also, by using gray color, feature points and surrounding pixels can be compared faster than single color images.

Although gray color examples are given above, real-time operation is possible not only when using gray color but also when using general multi-color image (RGB color, etc.), and can compare real-time feature points and surrounding pixels with only a small implementation cost.

The cyclic register generating step S23 is a step for generating a cyclic register based on the value determined in the pixel defining step S22 for the concentric block 210 divided in the region extracting step S21

Da food

(Where FCS-LBP _{S, P, and R} are the cyclic registers having the number of P bits, and _{S of} FCS-LBP _{S, P, and R} is the amount by which the bit value stored in the cyclic register is to be circulated (G _p - g _c ) is a value corresponding to the pixel, g _p is the color value of the pixel around the p-th feature point, g _c is the color value of the feature point pixel, Shift is the bit address to be stored in the cyclic register with the number of P bits)

A cyclic register 220 having a number of P bits with S being zero is generated.

In other words, with respect to the concentric block 210 divided in the region extracting step S21, 0 or 1 values specified in the pixel defining step S22 are allocated and stored by 1 bit, respectively, Registers can be created.

For example, when three concentric blocks (the first area 211, the second area 212 and the third area 213) are used,

4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15} where P0 = 16 and R0 = generating a FCS-LBP _0,16,3 for and, P1 = 12 and the surrounding pixels in R1 = 2 {a, b, c, d , e, f, g, h, i, j, k, l} for FCS-LBP generates _0,12,2, P2 = 8 and R2 = 1 of {a, B, C, D , E, F, G, H} FCS-LBP 0,8,1 of the neighboring pixel Lt; / RTI >

The bit shifting step S24 shifts the bits of each cyclic register 220,

In addition,

(Where Shift is a bit address to be stored in a cyclic register having a P number of bits, P is the number of pixels in the R region, p is the pth bit of the P pixel, S is the number of bits stored in the cyclic register, The amount to be moved)

Bit shifts each cyclic register 220 generated in the cyclic register generation step S23.

Here, the S value for each concentric block to which to move the bit of each cyclic register 220 in the bit shifting step S24,

Next table

IDX S0 S1 S2 0 0 0 0 One One One 0 2 2 2 One 3 3 2 One 4 4 3 2 5 5 4 2 6 6 5 3 7 7 5 3 8 8 6 4 9 9 7 4 10 10 8 5 11 11 8 5 12 12 9 6 13 13 10 6 14 14 11 7 15 15 11 7

S1 and S2 can be fetched from the reference table by using the value of S0 as in FIG. The S0 value of the concentric block used as a reference is obtained through various methods in the feature point extraction step.

The values of the table are determined according to various feature point extraction techniques and other descriptor generation techniques. The generated table has a value for matching the reference directions of extracted minutiae in the same direction for the rotation invariant minutia matching. That is, it is used to rotate the descriptors as much as the table value to remove the rotation characteristic.

For example, FIG. 9 shows that a feature point having a directional characteristic of? ₁ degrees from the reference line in the (t-1) -th image is shifted in the t-th image while changing the directional characteristic by? ₂ degrees.

In this case, to make the feature points rotate, the feature points of the (t-1) th image should be rotated by the degree of θ ₁ , and the feature points of the t th image should be rotated by θ ₂ degrees.

Here, in the case of the third area 213 concentric block, since the selected pixels P are composed of 16 pixels, each time the one pixel is moved, 360 degrees is divided into 16 equal parts, which is about 22.5 degrees.

If, θ ₁ is 30 ° and θ ₂ is 120 Dora surface for θ _1, the third area 213 moves ssikeul concentric block 1 bit and the case of θ _2, the third area 213 ssikeul concentric blocks 5 bits Move it.

A table is defined by dividing the number of bits to be shifted by the number of bits P held by each of the 360-degree concentric blocks 210.

The defined values are defined differently according to user-defined reference points of feature points, feature extraction techniques, and other descriptor generation techniques.

Using "Features from Accelerated Segment Test" as a feature point extraction method, the reference direction for feature points is divided into 16 equal parts based on 360 degrees. That is, when the feature points are matched using three concentric blocks (the first area 211, the second area 212, and the third area 213), the S0 value to be entered into the table corresponds to the reference direction information obtained from the feature point extraction And the values of S1 and S2 set values such that the concentric blocks of the second area and the third area rotate similarly to the concentric blocks of the first area.

For example, when three concentric blocks (the first area 211, the second area 212 and the third area 213) are used, as shown in FIG. 5, the FCS-LBP _S0,16,3 , FCS-LBP _S1,12,2, and FCS-LBP _S2,8,1 .

When three concentric blocks 210 are used, the number of S values is three (S0, S1 and S2). However, if the number of concentric blocks 210 is increased, it is natural that the number of S increases.

If the number of concentric blocks is increased to improve the accuracy of matching, it is increased by the sum of the bit sizes P of the concentric blocks used, but this is a combination of multiple numbers using floating or fixed decimal points to represent a single number The implementation cost is lower than other matching schemes using structured descriptors.

In the present invention, it is assumed that S0 can be determined by a well-known feature extraction technique, and is defined as an input variable of the present invention suggestion. Therefore, the method of generating S0 is not described in the present invention proposal.

In other words, a descriptor including a rotation invariant property can be generated faster than a complicated calculation such as a trigonometric function or the like by using only a small number of bits, by obtaining a value for moving the bits of the circular register using the corresponding table.

The descriptor determining step S25 determines the descriptor by grouping the respective cyclic registers 220 bit shifted in the bit shifting step S24.

For example, when three concentric blocks (the first area 211, the second area 212, and the third area 213) are used, as shown in Fig. 6, three registers are combined to form a total of 36 bits Can be generated. The generated descriptor can generate descriptors more simply and easily by adding, subtracting, and comparing operations of integer type.

As described above, the descriptors are generated for all the minutiae points generated by the feature extraction technique.

In the descriptor comparison step S30, another image is input and the feature point 100 having the same descriptor 200 is searched.

The descriptors are compared with each other to find feature points having the same descriptor.

As shown in FIG. 3, the descriptor comparison step S30 may include a first calculation step S31, a second calculation step S32, and a same feature point determination step S33.

In the first calculation step S31, the descriptors 200 for the minutiae points 100 generated in one input image are compared with the descriptors 200 for the minutiae points 100 generated in the other image, Perform an exclusive or (XOR) operation.

The second calculation step S32 is a step of calculating the bits generated by '1' for the calculation result of the first calculation step S31

Da food

는 배타적 논리합(XOR), s_t는 현재연상, s_{t -1}은 이전영상)(Where, e is the error, the R region, P is the number of pixels of the region R, g _p is the color value of the peripheral pixels, and g _c is the color value of the center pixel,

(XOR), s _t is the current association, and s _{t -1} is the previous image)

And so on.

In the same feature point determination step S33, all of the error results when the results of the second calculation step S32 are divided by the concentric blocks are equal to or smaller than the first threshold value, and the result of the second calculation step S32 If the sum of the error values of all the concentric blocks is less than the second threshold value, it is determined that the selected two feature points 100 are the same feature point 100.

For example, when three concentric blocks (the first area 211, the second area 212 and the third area 213) are used, e _16,3 is equal to or smaller than the first threshold value, and e _{9 , 2} is equal to or smaller than the first threshold value, e _8,1 is equal to or smaller than the first threshold value, and the sum of e _16,3 , e _9,2 , and e _8,1 is less than the second threshold value , It is determined that the two selected minutiae are the same minutiae.

For example, the first threshold value may use 2, and the second threshold value may use 4.

In other words, feature point matching can be performed only by addition, subtraction, comparison, and exclusive OR (XOR) operations of an integer type, and the low resolution of the degree of rotation by the first threshold value is compensated. It is possible to correct other errors caused by the resolution compensation to the extent that the rotation is rotated by the rotation angle?

) 연산을 통해 기술자를 비교하고 에러 값이 임계값을 벗어나지는 않는 경우 특징점으로 간주하도록 하였다.
A real-time feature point matching method having rotation invariance for minutiae points in an image according to an embodiment of the present invention is called "Fast Circular Shift-Local Binary Pattern (FCS-LBP)" In this method, the sin, cos, and tan operations considering the degree of rotation are removed, and the descriptor is generated only by the bit shift of the circular register based on the lookup table to obtain the rotation invariant feature point matching. In the engineer comparison stage, the engineer performs simple XOR (XOR:

) Operation to compare the descriptors and to treat them as feature points if the error value does not exceed the threshold value.

As a result, according to an embodiment of the present invention, a real-time feature point matching method having rotation invariance for feature points in an image uses a simple descriptor having a rotation invariant characteristic for a feature point, Can be performed more quickly, the calculation structure is simple, it is easy to design with a digital circuit, and it is easy to detect when it is implemented with a circuit, so that it can be applied at a low cost in real time processing field.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

100: Feature point 200: Technician
210: concentric block 211: first region
212: second area 213: third area
220: Cyclic register 230: Control table
S10: minutia extraction step S20: descriptor creation step
S21: Extracting region step S22: Pixel definition step
S23: Cycle register generation step S24: Bit shift step
S25: Technician determination step S30: Technician comparison step
S31: First calculation step S32: Second calculation step
S33: Identification of the same minutiae

Claims (6)

A feature point extracting step (S10) of extracting the feature point 100 by receiving the image information,
(FCS-LBP: Fast Circular Shift (FFT)) which improves an LBP (Local Binary Pattern) for generating a descriptor representing a specific region with respect to the feature point 100 extracted in the feature point extracting step (S10) A descriptor generating step S20 of generating a descriptor 220 of a rotation invariant characteristic representing a minutia point using a local binary pattern
A descriptor comparison step (S30) of receiving another image and finding a feature point (100) having the same descriptor (200)
And,
In the descriptor comparison step S30,
The descriptors 200 for the feature points 100 generated in one input image are subjected to XOR operation for descriptors 200 and descriptors 200 for feature points 100 generated in another image, A first calculation step (S31) of performing an operation;
The bits in which '1' is generated for the result of the operation in the first calculation step (S31)
Da food

(Where, e is the error, the R region, P is the number of pixels of the region R, g _p is the color value of the peripheral pixels, and g _c is the color value of the center pixel,

Is the exclusive OR, st is the current association, and st-1 is the previous video)
A second computation step (S32) of computing as
The error results obtained when the results of the second calculation step S32 are divided by the concentric blocks are all equal to or smaller than the first threshold value and the result of the second calculation step S32, If the sum is smaller than the second threshold value, determining that the two selected feature points 100 are the same feature point 100;
Wherein the real-time feature point matching process has rotation invariance for the feature points in the image.
The method according to claim 1,
The descriptor creation step (S20)
A region extracting step (S21) of dividing the region into two or more concentric blocks (210) spaced a certain distance around the feature point (100) depending on the region (R) of the fast cyclic shift ambience;
By comparing the color with surrounding pixels around the minutiae 100
The following equation

(Where g (g _p - g _c ) is a value corresponding to the pixel, g _p is the color value of the surrounding pixels, and g _c is the color value of the center pixel)
A pixel defining step (S22) of determining a value (1 or 0) corresponding to the pixel by the pixel defining step
For the concentric blocks 210 divided in the region extracting step S21, based on the values determined in the pixel defining step S22
Da food

(Where FCS-LBP _{S, P, and R} are the cyclic registers having the number of P bits, _{S of} FCS-LBP _{S, P, R} is the amount by which the bit value stored in the cyclic register should be shifted, R is a region, P is the number of pixels of the R _{domain, s (g p - g c} ) is corresponding to the pixel value, g _p is the color value of the peripheral pixel, g _c is the color value of the center pixel, Shift the P bit Bit address to be stored in the cyclic register having the number)
A cyclic register generating step (S23) for generating a cyclic register (220) having a number of P bits with S being 0 by the cyclic register generating step (S23);
The bits of each cyclic register 220,
In addition,

(Where Shift is a bit address to be stored in a cyclic register having a P number of bits, P is the number of pixels in the R region, p is the pth bit of the P pixel, S is the number of bits stored in the cyclic register, The amount to be moved)
A bit shifting step (S24) of bit shifting each cyclic register (220) generated in the cyclic register generating step (S23) by a bit shifting step (S24); And
A descriptor determining step S25 of determining a descriptor by grouping each cyclic register 220 bit shifted in the bit shifting step S24;
Wherein the real-time feature point matching process has rotation invariance for the feature points in the image.
3. The method of claim 2,
The concentric block 210 of the region extracting step S21
8 pixels adjacent to the center point are referred to as a first region 211, 12 pixels adjacent to the first region 211 as a second region 212, and 16 pixels adjacent to the second region 212 And a third area (213). The real-time feature point matching processing method according to claim 21,
3. The method of claim 2,
The color value of the pixel defining step (S22)
Wherein the gray-scale image is a gray-colored gray-scale image.
3. The method of claim 2,
Characterized in that the S value for each concentric block to which to move the bit of each cyclic register (220) in the bit shifting step (S24) fetches S1 and S2 from the predetermined reference table (230) Time feature point matching processing method.
delete

Images