WO2014175481A1

WO2014175481A1 - Method for generating descriptor and hardware appartus implementing same

Info

Publication number: WO2014175481A1
Application number: PCT/KR2013/003521
Authority: WO
Inventors: 최병호; 이상설; 황영배; 장성준; 김정호
Original assignee: 전자부품연구원
Priority date: 2013-04-24
Filing date: 2013-04-24
Publication date: 2014-10-30

Abstract

Disclosed are a method for generating a descriptor and a hardware apparatus implementing same. Herein, a method for generating a descriptor, which is a method for a hardware apparatus for generating a descriptor on the basis of a feature of an image, comprises the steps of: configuring a region of interest in which integral image points are uniformly arranged with the feature as the center; calculating all integral image points arranged in the region of interest by means of memory patching; selecting, from among the integral image points, integral image points included in a descriptor generation area; and generating a descriptor on the basis of the selected integral image points.

Description

Descriptor generation method and hardware device implementing the same

The present invention relates to a method for generating a descriptor and a hardware device for implementing the same. Specifically, a hardware structure design technique for generating a descriptor of Speeded Up Robust Features (SURF), which is an algorithm for object extraction using an input image. It is about.

The Speeded Up Robust Feature (SURF) algorithm extracts feature points and descriptors of an object. Fast-Hessian detectors are used to extract feature points that are resistant to size changes, and the extracted feature points are given descriptors that are resistant to rotational changes.

In this case, the conventional SURF algorithm generates a descriptor by dividing the peripheral region of the feature point into 16 regions based on the main direction in order to give a strong descriptor to the rotational change. The SURF algorithm determines the size of the surrounding area to be sampled using the scale information extracted from the feature points and the size of the principal component. The coordinates for rotating the rectangle to generate the descriptor are calculated using the main direction of the feature point. In addition, as the angle of the main axis changes, the pixel that is the center of the calculation does not have a constant pattern, but the calculation is performed at non-uniform intervals according to the rotation.

1 illustrates an integrated image point necessary for generating a descriptor in the related art, FIG. 2 illustrates an integrated image point for a conventional haar-wavelet response operation, and FIG. 3 illustrates a conventional memory. An illustration of patching.

Referring to FIG. 1, the horizontal axis represents a pixel address in the x-axis direction of the integrated image, and the vertical axis represents a pixel address in the y-axis direction of the integrated image.

At this time, in order to calculate the descriptor required for the feature point, all the integrated image points P1 around the feature point should be calculated.

The integrated image point P1 is obtained by approximating the integral image point required by the calculation based on the origin of the feature point. Therefore, the integrated image point P1 is a value that changes depending on the origin of the feature point and the amount of rotation of the main axis. Therefore, as shown in FIG. 1, the integrated image point P1 is not uniform.

Here, in order to calculate a point of interest among the integrated image points P1, a total of eight peripheral points P3 adjacent to the point of interest P1 ′ are required as shown in FIG. 2. That is, FIG. 2 shows the integral image point required for one ha-wavelet calculation.

In this case, when the peripheral point P3 performs memory patching three times in the x-axis direction, all of the peripheral points P3 arranged in the y-axis direction may be acquired.

However, since the integrated image point P1 is not uniform as in FIG. 1, in order to calculate a total of eight points of interest P1 ′ as shown in FIG. 3, a total of 24 times for each point of interest P1 ′ is calculated. Memory patching is required.

As described above, the descriptor generation unit reads the integrated image point in the y-axis direction by applying an address value in the x-axis direction in which the integrated image point is desired to be obtained from the memory in which the integrated image is stored. That is, the operation of storing the necessary data by reading a large amount of integrated image points in the y-axis direction according to one address value in the x-axis direction is memory patching.

Conventionally, as described above, the memory patch can be performed only after the calculation of each accumulated image point. Therefore, the accumulated image points are unpredictable and cannot be memory patched in advance. This delays the processing speed of the memory.

In addition, when patching an integrated image memory corresponding to a coordinate value to be rotated, since the integrated image must be accessed every time according to the change of the angle of the main axis after the feature point extraction of the object, the integrated image cannot be processed at high speed.

This is because hardware patching is required every time to patch the integrated image memory corresponding to the coordinate value to be rotated in the hardware implementation, and thus high speed processing cannot be performed.

Accordingly, an aspect of the present invention is to provide a method and an apparatus capable of reducing the number of times of memory patching for acquiring data of an integrated image required for a descriptor operation.

According to one aspect of the present invention, a descriptor generation method is a method in which a hardware device generates a descriptor based on a feature point of an image, setting a region of interest in which integral image points are evenly disposed around the feature point, and memory patching. Calculating all of the integrated image points disposed in the ROI, selecting integral image points included in a descriptor generation region among the integrated image points, and generating a descriptor based on the selected integrated image points It includes a step.

The setting step,

The ROI in the form of a circle may be set based on the origin of the feature point.

The calculating step,

The operation of calculating a point of interest based on integrated image points acquired through memory patching in one basic unit may be repeated to calculate all integrated image points disposed in the ROI.

The sorting step,

Setting all integrated image points disposed in the ROI acquired through the memory patching as candidate points, coordinate values of each of the candidate points, origin coordinates of the feature points, slopes of the main axes, and scale information of the feature points Determining the descriptor generation region based on the method, determining whether coordinate values of each of the candidate points are included in the descriptor generation region, and interested points necessary for generating the descriptors based on candidate points included in the descriptor generation region. Screening may include.

The descriptor generation region includes a plurality of regions consisting of l and k determined through the following equation,

The determining step,

A descriptor generation method for determining whether the coordinates of each of the candidate points are included in the descriptor generation region by applying the following equation;

Here, Sample X and sample Y are coordinate values indicated by the candidate points, x and y represent the coordinates of the origin of the feature point, co, si are cosine values (cos (θ)) corresponding to the inclination (θ) of the main axis, each represents a sine value (sin (θ)), Scal represents scale information of a feature point, l determines a horizontal region of the descriptor generating region, and k determines a vertical region of the descriptor generating region. .

According to another feature of the present invention, a hardware device is a hardware device for calculating a feature point and a descriptor of an image, an integrated image memory for storing an integrated image for the image, a feature point extracting unit for extracting the feature point based on the integrated image, And setting a region of interest in which integral image points are evenly disposed around the feature point, and performing memory patching to calculate all of the integral image points disposed in the region of interest from the integral image memory, and among the integrated image points. And a descriptor generator configured to select integral image points included in the descriptor generation area and generate a descriptor based on the selected integrated image points.

The descriptor generation unit,

A region of interest in a circular shape is set based on the origin of the feature point, and the feature point obtained from the feature point extractor is located within the region of interest based on scale information from which the feature point is extracted, the origin of the feature point, and the principal axis direction of the feature point Descriptor generation area can be determined.

The descriptor generation unit,

The descriptor generation region is determined based on coordinate values of all candidate points disposed in the ROI acquired through the memory patching, origin coordinates of the feature point, slope of a main axis, and scale information of the feature point, and the candidate point. By determining whether each of the coordinate values is included in the descriptor generation region, points of interest necessary for generating the descriptor may be selected from the candidate points.

According to the embodiment of the present invention, since the data of the integral image required for the operation is obtained in one access in a large amount and the number of memory accesses is reduced, the processing speed of the descriptor operation can be improved.

In addition, although the pixel of interest that changes in accordance with the main axis direction is conventionally used, according to the embodiment of the present invention, since the fixed pixel structure is used, it is suitable for the parallel hardware structure.

In addition, the hardware structure of the same structure can be applied irrespective of the position of various scales and object feature points, it is possible to design a flexible hardware block.

In addition, since the shape of the region of interest is considered as a prototype, the point of interest for the descriptor operation is reduced, and thus, the computational amount of the descriptor is reduced, thereby improving the computational processing speed of the descriptor.

1 shows an integral image point conventionally required for descriptor generation.

2 is an exemplary diagram of computing an integrated image point for a conventional haar-wavelet response operation.

3 is an exemplary diagram of conventional memory patching.

4 is a schematic configuration diagram of a speeded up robust feature (SURF) hardware device according to an embodiment of the present invention.

5 illustrates components of a descriptor to which an embodiment of the present invention is applied.

6 and 7 illustrate indexing of sixteen regions to which embodiments of the present invention are applied.

8 is a flowchart illustrating a descriptor generation method according to an embodiment of the present invention.

FIG. 9 is an exemplary diagram of integral image points required when patching memory for one haar-wavelet response operation according to an embodiment of the present invention.

10 illustrates a memory patching process according to an embodiment of the present invention.

11 illustrates a descriptor generation region according to an embodiment of the present invention.

12 illustrates points of interest required when generating descriptors according to an embodiment of the present invention.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

Throughout the specification, when a part is said to "include" a certain component, it means that it can further include other components, except to exclude other components unless specifically stated otherwise.

In addition, the terms "… unit", "... module" described in the specification means a unit for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software.

Hereinafter, a descriptor generating method and a hardware device implementing the same according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

Referring to FIG. 4, the SURF hardware device 100 includes an image storage unit 101, an integrated image generator 103, an integrated image memory 105, a feature point extractor 107, and a descriptor. The generation unit 109 is included.

The image storage unit 101 stores the input black and white image.

The integrated image generator 103 generates an integrated image of the black and white image stored in the image storage unit 101 and stores the integral image in the integrated image memory 105.

The feature point extractor 107 generates image pyramids and extracts feature points from the generated image pyramid to represent a scale space for extracting feature points from the input black and white image. At this time, the integral image used is the core of the SURF algorithm.

When the main direction of the feature point is determined, the descriptor generator 109 generates a descriptor by dividing the area around the feature point into 16 areas based on the main direction.

The descriptor generator 109 includes an integrated image obtained from the integrated image memory 105, an image size including height and width, origin (x, y) coordinates, and scale of feature points. Generates a descriptor by receiving SURF information.

Here, the image size is obtained when the initial SURF hardware device 100 is implemented, and the SURF information is obtained from the feature point extractor 107.

SURF information is the origin (x, y) coordinates and scale information of a feature point determined to have a large amount of change or maintain a constant pattern when compared with the surrounding area on the integrated image. Here, the feature point depends on the threshold of the feature point extractor 107.

That is, the descriptor generator 109 calculates an integrated image point necessary for generating a descriptor from feature information, that is, x, y coordinates, scale information, and main direction information, to which an integral image point of a pixel required for descriptor generation is input.

Referring to FIG. 5, the descriptor generator 109 may sample using the scale information from which the feature point P7 is extracted, the origin point (x, y) of the feature point P7 of the main axis, and the magnitude of the component of the main axis direction P9. The size of the peripheral area P11 is determined.

That is, the descriptor generator 109 rotates the integrated image based on the main axis direction P9 and divides the image into 16 areas P11. Then, Σdx, Σ | dx |, Σdy, and Σ | dy | are extracted using the difference between adjacent pixel values in each region P11. And ∑dx, ∑ | dx |, ∑dy, and ∑ | dy | extracted from each region P11. Create a total of 64 descriptors (= 4 서술 4 ㅧ 4) using the components.

6 and 7, the descriptor output order of the 16 regions is sequentially from Region 0_1 to Region_3_4, that is, Region 0_1-> Region 0_2-> Region 0_3->. -> Region 3_3-> Region 3_4

At this time, even if the main shaft P9 rotates, as shown in FIG. 7, 16 areas P11 remain constant. That is, it can be seen that the shape P11 of the sixteen regions is maintained even when the main direction P9 is rotated based on the feature point P7.

8 is a flowchart illustrating a descriptor generation method according to an embodiment of the present invention, and FIG. 9 is an integral image point required when patching a memory for one haar-wavelet response operation according to an embodiment of the present invention. 10 is an exemplary diagram, and FIG. 10 illustrates a memory patching process according to an embodiment of the present invention, and FIG. 11 illustrates a descriptor generation area according to an embodiment of the present invention.

First, referring to FIG. 8, the descriptor generator 109 sets a region of interest (ROI) having a radius of a predetermined size around a feature point (S101), which is the ROI. Is the same as FIG. In this case, the radius of the circle may be a value determined by the operator.

Referring to FIG. 9, the integrated image points P15 are evenly distributed in the region of interest ROI P13. All of the integrated image points P15 distributed in the ROI P13 are candidate points for descriptor generation. Hereinafter, all the integrated image points P15 distributed in the region of interest ROI P13 are described as candidate points P15.

Referring again to FIG. 8, the descriptor generator 109 acquires candidate points P15 from the integrated image memory 105 through memory patching S103, and the method of obtaining the descriptor points P15 is the same as FIG. 10. Here, FIG. 10 illustrates a portion of candidate points P15.

Referring to FIG. 10, a process of acquiring an arbitrary candidate point P15, that is, a first point of interest 1, a second point of interest 2, and a third point of interest 3 will be described.

In this case, in order to calculate the first point of interest 1, the second point of interest 2, and the third point of interest 3, the first point of interest 1 and the second point of interest 2, as described with reference to FIG. 2. A total of eight peripheral points arranged in the left, right, and up and down directions with respect to the third point of interest 3 are required. Therefore, the peripheral point must be obtained through the basic three memory patching in the x-axis direction.

At this time, the candidate points P15 acquired through the first first (①) memory patching are stored in the first buffer.

The candidate points P15 acquired through the next second (2) memory patching are stored in the second buffer.

Next, candidate points P15 obtained through the next third memory patching are stored in the third buffer.

As such, the three first points of interest 1 may be calculated based on the candidate points P15 obtained through the three times of memory patching.

Since the candidate points P15 acquired through the second (②) memory patching and the third (③) memory patching become peripheral points, the second point of interest 2 becomes a peripheral point through the fourth (④) memory patching. The candidate points P15 may be obtained. Therefore, the descriptor generator 109 may empty the first buffer and store peripheral candidate points P15 obtained through the fourth memory patching in the first buffer. In this manner, nine first points of interest 1, second points of interest 2, and third points of interest 3 may be calculated through a total of five times of memory patching.

In this way, since 3 memory patching patches a total of 1296 to calculate one integrated image point, an integrated image necessary for generating a descriptor is required by 37 memory patching processes, unlike a total of 3888 memory patching processes. You can compute points.

On the other hand, after the descriptor generator 109 acquires all the candidate points P15, 16 descriptor generation regions are determined in the ROI based on the contents described with reference to FIGS. 6 and 7 (S105).

At this time, the descriptor generation unit 109 determines 16 descriptor generation regions based on the feature point P7 through the values of l and k. And l, k is calculated through the following equation (1).

Equation 1

Here, Sample X and sample Y mean coordinate values indicated by integrated image points (or candidate points) disposed in the ROI. x and y represent the origin coordinates of the feature point P7. co and si represent cosine values (cos (θ)) and sine values (sin (θ)) corresponding to the slope θ of the main axis (P9 of FIGS. 5, 6 and 7), respectively. Scal represents scale information of a feature point.

In this case, the region in the horizontal direction among the 16 regions is determined through the k value based on the feature point P7, and the region in the vertical direction is determined by the l value.

That is, referring to FIG. 11, the sixteen areas P11 are 12 from the feature point 7 to the left and 12 to the right and 12 up and down, respectively, that is, l: -12 to 12, based on the feature point P7. k: It is classified into the range from -12 to 12. The center coordinates (xs, ys) P17 of each region P11 are calculated to perform a haar-wavelet operation for each of the sixteen regions P11 to generate a descriptor.

In this case, since the candidate points P15 in the ROI are already acquired in step S103, the descriptor generator 109 may include candidates included in the descriptor generation area determined in step S105, that is, the 16 areas P11. The points P15 are selected (S107).

Here, the selection process applies the coordinate values (Sample X, sample Y) of the candidate points (P15) to the equation (1) to determine whether the candidate points (P15) are included in the 16 areas (P11) consisting of l, k You can judge.

Next, the descriptor generation unit 109 performs a descriptor generation operation based on the candidate points P15 selected in step S107 (S109).

At this time, in step S109, a descriptor generation operation is performed using the above-described equations (Σdx, Σ | dx |, ∑dy, ∑ | dy |).

That is, a dx buffer that stores the amount of the direction component on the x-axis of the region in which the candidate points are included among the 16 areas, a dy buffer that stores the direction component on the y-axis, and a | dx | Buffer, | dy | which stores the absolute value of the direction component of the y-axis. Stores the buffer again by adding the value currently stored in the previous buffer with the currently calculated value. Therefore, when all candidate points are computed, dx, | dx |, dy, | dy | The sum of the values calculated from the candidate points of the regions is stored in the buffer, and the final descriptor is generated through normalization of the stored values.

As described above, according to the exemplary embodiment of the present invention, the point of interest necessary for generating the descriptor may be obtained by distinguishing the candidate point in the ROI corresponding to the actual descriptor generating region. Therefore, since candidate points P15 are unevenly distributed in the related art, there is no problem that an operation must be performed for each candidate point P15. This allows the descriptor generator 109 to be suitable for hardware and to operate at high speed.

In addition, by setting the ROI in a circular shape, it is possible to bring about a fast processing speed by not performing calculation of unnecessary image points, that is, accumulated image points arranged outside the ROI.

Referring to FIG. 12, assuming that the scale is 2 and a maximum of 11 integrated image points are computed at one time, the integrated image points existing in the circular ROI required for the descriptor generation are illustrated in the pixel P19. will be. At this time, the integrated image point is represented by numbers (1, 2, 3, 4, 5, 6, 7).

In this case, when the descriptor generator 109 performs a memory patch for each horizontal line to patch the first five lines, seven descriptor points may be calculated at a time. That is, the wavelet calculation is performed through the box P21 having the peripheral point set to 1, and the point of interest 3 located in the center is calculated. In this way, box P21 sets the

peripheral points

2, 3, 4,... If set to, 7, the points of

interest

1, 2, 3, 4, 5, 6, and 7 located in the center of the box P21 may be calculated. The next one line of memory patching computes nine points of interest, and each subsequent line of patch can compute 11 points of interest. In this way, seven candidate points are calculated through the first five patches, and once one patch (the sixth patch) is added, a total of nine candidate points are calculated, and another patch (the seventh patch) generates 11 Candidate points are computed. Thereafter, 11 candidate points are calculated for each patch. Therefore, after a total of seven patching is performed, a total of 27 candidate points are calculated.

This can result in a 33x speedup over the existing method, which requires three patches for every point of interest.

The embodiments of the present invention described above are not only implemented through the apparatus and the method, but may be implemented through a program for realizing a function corresponding to the configuration of the embodiments of the present invention or a recording medium on which the program is recorded.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concepts of the present invention defined in the following claims are also provided. It belongs to the scope of rights.

Claims

A hardware device generates a descriptor based on an image feature point.

Setting an ROI in which integral image points are evenly arranged around the feature point;

Computing all of the integrated image points disposed in the region of interest via memory patching,

Selecting integral image points included in a descriptor generation region among the integrated image points, and

Generating a descriptor based on the selected integral image points

Descriptor generation method comprising a.
The method of claim 1.

The setting step,

Descriptor generation method for setting the ROI in the form of a circle based on the origin of the feature point.
The method of claim 1.

The calculating step,

And calculating all the integral image points arranged in the ROI by repeating the operation of calculating the ROI based on the integral image points acquired through the memory patching of one basic unit.
The method of claim 1,

The sorting step,

Setting all integrated image points disposed in the ROI acquired through the memory patching as candidate points,

Determining the descriptor generation region based on coordinate values of each of the candidate points, origin coordinates of the feature point, slope of a main axis, and scale information of the feature point;

Determining whether a coordinate value of each of the candidate points is included in the descriptor generation region, and

Selecting candidate points included in the descriptor generation region as interest points required for generating the descriptor;

How to create a descriptor.
The method of claim 4, wherein

The descriptor generation region includes a plurality of regions consisting of l and k determined through the following equation,

The determining step,

A descriptor generation method for determining whether the coordinates of each of the candidate points are included in the descriptor generation region by applying the following equation;

Here, Sample X and sample Y are coordinate values indicated by the candidate points, x and y represent the coordinates of the origin of the feature point, co, si are cosine values (cos (θ)) corresponding to the inclination (θ) of the main axis, each represents a sine value (sin (θ)), Scal represents scale information of a feature point, l determines a horizontal region of the descriptor generating region, and k determines a vertical region of the descriptor generating region. .
A hardware device for calculating feature points and descriptors of an image,

An integrated image memory for storing an integrated image for the image;

A feature point extracting unit extracting the feature point based on the integrated image, and

Set a region of interest in which integral image points are evenly arranged around the feature point, and perform memory patching to calculate all of the integral image points disposed in the region of interest from the integrated image memory, and describe the descriptors from the integral image points. Descriptor generation unit that selects integral image points included in the generation area and generates a descriptor based on the selected integrated image points

Hardware device comprising a.
The method of claim 6,

The descriptor generation unit,

A region of interest in a circular shape is set based on the origin of the feature point, and the feature point obtained from the feature point extractor is located within the region of interest based on scale information from which the feature point is extracted, the origin of the feature point, and the principal axis direction of the feature point; Hardware device that determines the descriptor generation area.
The method of claim 7, wherein

The descriptor generation unit,

The descriptor generation region is determined based on coordinate values of all candidate points disposed in the ROI acquired through the memory patching, origin coordinates of the feature point, slope of a main axis, and scale information of the feature point, and the candidate point. Determining whether coordinate values are included in the descriptor generation region, and selecting points of interest necessary for generating the descriptor among the candidate points.