KR100691855B1 - Apparatus for extracting features from image information - Google Patents

Abstract

An apparatus and a method for extracting features of image information are provided to set a plurality of local regions from which features of an image are extracted and thereby improve performance in recognition of the image. A feature point detection unit(100) calculates a curvature value of image information and detects feature points for feature extraction by using the curvature value. A feature extraction unit(110) detects vectors with respect to affine transformation in a plurality of regions centering around the detected feature points and determines a set of the detected vectors as a feature vector. The feature point detection unit includes a filter unit for carrying out Gausian filtering regarding to the image information, a differential operation unit for calculating differential values of the image information and a curvature calculation unit for calculating the curvature value of the image information by using the differential values.

Description

Apparatus for extracting features from image information}

1 is a block diagram of the present invention.

FIG. 2 is a detailed block diagram of FIG. 1.

3 is a flowchart of the present invention.

4 is a detailed flowchart of FIG. 3.

FIG. 5 illustrates a process of generating local windows in the local region setting process of FIG. 4.

FIG. 6 illustrates a process of setting a feature vector of FIG. 4.

FIG. 7 is a graph illustrating a recognition rate performance evaluation result according to scale transformation and translation transformation of input image information with respect to the conventional feature extraction method (SIFT) and the feature extraction method according to the present invention (AIFE). .

8 is a graph illustrating a recognition rate performance evaluation result according to a depth direction transformation of input image information for a conventional feature extraction method (SIFT) and AIFE of the present invention.

The present invention relates to image signal processing, and more particularly, to a feature extraction apparatus and method thereof.

In general, it is important to select the recognition technique because the result of the feature extracted from the image passes through the recognition stage and estimates the position of the robot accordingly. Recognition stage can be divided into two types of training method and non-training method. In the case of driving robot, non-training recognition algorithm is generally applied because new data need to be created and updated according to environment change.

The scale invariant feature transform (SIFT) method, which is one of the typical vision-based feature extraction methods for the estimation of the position of a traveling robot, was developed around 2003, and is robust due to the transformation of the scale of image information. The Gaussian Pyramid technique is applied to maintain the stiffness, and a histogram of the angular components of the image information derivatives is used to maintain the robustness in 2D rotation of the image information.

However, the Scale Invariant Feature Transform (SIFT) method is more commonly used in 3D-rotation, that is, deformation in the depth direction than in two-dimensional rotation transformation in left and right in an indoor environment in which a robot travels. It does not consider rotational transformation in the dimensional depth direction.

Recently, research on feature extraction that is robust to affine transformation in the field of vision recognition has been a major issue, and active research is being conducted. Krystian Mikobajczyk of France has proposed a Harris-Affine Invariant Detector that transforms the Corner Detector. However, no research has been conducted on what features to extract from the extracted points and the amount of computation that is impossible to solve the real-time problem of the detector.

However, since the feature extraction method of the conventional image information occurs depending on the location where the image is captured, it is difficult to apply it to the real environment by extracting the affine moments from a given whole image. For example, the recognition performance is reduced even when the environment is slightly changed, and when the verified small moment vector is used, the number of features is insufficient and the performance of the recognition stage is degraded.

An object of the present invention is to provide a feature extraction apparatus for image information that detects feature points including a feature of an image from the image information and forms a feature vector around the detected feature points.

Another object of the present invention is to provide a feature extraction method of image information applied to the feature extraction apparatus of the image information.

In order to solve the above technical problem, the present invention calculates a curvature value of the image information and uses a feature point detection unit for detecting feature points for feature extraction using the curvature value and in a plurality of areas around the feature points And a feature extractor for detecting vectors for affine transformation and setting the set of vectors as a feature vector.

In order to solve the above other technical problem, the present invention calculates a curvature value of the image information, detecting the feature points for feature extraction using the curvature value and in a plurality of areas centered on the feature points Detecting vectors for affine transformation and setting the set of vectors as a feature vector.

Hereinafter, with reference to the drawings will be described a preferred embodiment of the present invention.

1 is a block diagram of the present invention.

The feature point detector 100 calculates a curvature value of the image information and detects feature points for feature extraction by using the calculated curvature value. Preferably, the feature points may be pixels selected for feature extraction of the image among a plurality of pixels constituting the image. Here, the image information refers to a vector or matrix in which digital signal or digitally converted image pixel values from an image input device (camera, camcorder, computer image cam, etc.) are stored.

The feature extractor 110 detects vectors for affine transformation in a plurality of regions centered on the detected feature points, and sets the detected vector as a feature vector. Preferably, the feature extractor 110 outputs the set feature vectors. In this case, the vector for the affine deformation, that is, the affine moment, is a vector composed of a total of M elements proposed by Tomas-suk.

FIG. 2 is a detailed block diagram of FIG. 1.

The feature point detection unit 100 calculates a curvature value of the image information and detects feature points for feature extraction using the calculated curvature value.

In FIG. 2, the feature point detector 200 includes a filter 201, a derivative calculator 202, a curvature calculator 203, a corner detector 204, a point selector 205, and a point remover 206. .

The filter unit 201 performs Gaussian filtering on the image information. In this case, the reason for performing Gaussian filtering is to prevent performance degradation due to the sensitivity of the input image information.

The derivative operation unit 202 calculates differential values of the image information filtered by the filter unit 201.

The curvature calculator 203 calculates a curvature of the image information by using the derivatives calculated by the differential calculator 202.

The corner detector 204 detects corners of the image information by using a local maximum value of the curvature calculated by the curvature calculator 203. The parts of the image that form the boundary of the image are the pixels having the maximum curvature. Therefore, pixels whose curvature is locally maximum in the image are determined as corners that are boundaries of the image. Preferably, the corner detector 204 detects corners of the image information by applying a curvature value to the Harris Corner Detector.

The point selector 205 selects feature points from the points constituting the corners detected by the corner detector 204. Preferably, vertices of the corners are selected as feature points.

The point removing unit 206 removes, from among the feature points, local points to be set out of the image information area. This is a process of removing points whose horizontal and vertical distances on the boundary line between the feature points and the image information are less than or equal to a preset reference size. The reason for removing such points is to make the size of the feature vector extracted for each feature point the same and maintain the performance of the recognition stage. At this time, the preset reference size is a size that a person skilled in the art can arbitrarily determine in consideration of the edge length of the local area. Preferably, the preset reference size at this time may be set to a size corresponding to 1/2 of the edge length of the local area.

The feature extractor 110 detects vectors for affine transformation in a plurality of regions centered on the detected feature points, and sets the detected vector as a feature vector.

The feature extractor 110 may include a local region setter 211, an affine moment detector 212, a feature vector setter 213, a volatile memory 215, and a nonvolatile memory 216.

The local area setting unit 211 sets local points having feature points as the center of each area and having a predetermined reference size. Preferably, the local area setting unit 211 may generate local windows by dividing the local area into a size smaller than the reference size. In this case, the reason for generating the local window by dividing the local area is to construct a high-dimensional feature vector.

The affine moment detector 212 detects vectors for affine deformation for each local region. Preferably, the affine moment detecting unit 212 detects vectors for affine transformation for each local window when the local region setting unit 211 generates local windows, and detects the set of detected vectors in the local region. Set to the detected vector for the phosphorus deformation.

The vectors for the affine deformation of the local region, that is, the affine moment, may be expressed as a function Z normalizing the function F for image information having the same value regardless of the affine deformation to the zero-order moment of the function F. When this is expressed as an equation, Equations 1, 2, and 3 are as follows.

F '= G

Where F is the output value of the function F in the affine transformed image, and G is a constant value composed of parameters related to the affine transformation representing the relationship between the two.

The function F that satisfies the above expression can be expressed as a function consisting of cross product and image pixel values. Therefore, if we remove only the G value, we can find the function F irrelevant to the affine transformation. In order to remove the G value, a characteristic such as the following Equation 2 of the zero-order moment μ ₀₀ is used.

Therefore, if the function F is normalized to μ ₀₀ , the function Z having the same value even after the affine transformation can be finally obtained as shown in Equation 3 below.

At this time, the function Z is transformed into a moment shape is an affine moment (Affine Moment).

The feature vector setting unit 213 sets a set of detected vectors for each local area as a feature vector for image information.

3 is a flowchart of the present invention.

First, the curvature value of the image information is calculated, and feature points for feature extraction are detected using the calculated curvature value (step 300). Preferably, the feature points may be pixels selected for feature extraction of the image among a plurality of pixels constituting the image. Here, the image information refers to a vector or matrix in which digital pixel signals or digitally converted image pixel values from the image input apparatus are stored.

Next, vectors for affine transformation are detected in the plurality of regions centered on the feature points, and the set of detected vectors is set as the feature vector (step 310). Preferably, the step 310 of setting the feature vectors includes outputting the set feature vectors.

4 is a detailed flowchart of FIG. 3.

First, Gaussian filtering the image information (400). In this case, the reason for performing Gaussian filtering is to prevent performance degradation due to sensitivity of the input image information.

Next, differential values are calculated from the Gaussian filtered image (step 401). Preferably, the derivative value may include a first derivative value and a second derivative value.

When the derivative values are calculated, a curvature value is calculated using the calculated derivatives (step 402). If the curvature value is large, the image changes abruptly in the image. If the curvature value is small, the image changes slowly in the image. In this case, a portion in which the image changes rapidly in the image mainly means an outline of the image, that is, a boundary.

Next, corners are detected using the local maximum of the calculated curvature value (step 403). In this case, the corners are corners that form a boundary of the image in the image.

If corners are detected, feature points for detecting the affine moment are selected and unnecessary points are removed (step 404). Preferably, vertices of the corners are selected as feature points. In the process of removing unnecessary points (step 404), points from which the local area to be set out of the image information area are removed from the feature points. This is a process of removing points whose horizontal and vertical distances on the boundary line between the feature points and the image information are less than or equal to a preset reference size. The reason for removing the points as described above is to equalize the size of the feature vector extracted for each feature point and maintain the performance of the recognition stage. At this time, the preset reference size is a size that can be arbitrarily determined by the person in consideration of the edge length of the local area.

When the feature points are selected, a local area is set around each feature point (step 410). Preferably, a local window may be generated that divides the local region into several.

Next, the affine moment is detected in the local area (step 411). In this case, the affine moments are vectors for affine deformation. Preferably, when a local window is generated in a process of setting a local area (step 410), the affine moment is detected in each local window. In this case, the affine moment for the local area is the set of affine moments detected in the local window.

When the affine moment is detected, a set of affine moments for all local regions is set as a feature vector (step 412). Preferably, the step of setting to a feature vector (step 412) includes the step of outputting the set feature vector. Accordingly, the process 412 is a process of constructing the feature vector using the affine moment.

The feature vector configured as described above may be commonly used as reference information for identifying an object through an input image or for reading spatial information of a location at which a current image is captured.

FIG. 5 illustrates a process of generating local windows in the local region setting process (410) of FIG.

First, a local region of size T × T for extracting a feature vector based on the selected points is set. Next, local windows of size X × X are generated for each set local area. One local region is composed of P local windows as shown in FIG. If the size of the local area is T × T and the local window size is X × X, the number P of local windows derived from one local area is expressed by Equation 4 below.

P = (T / X) ²

When local windows are generated, M affine moments are extracted for each local window. At this time, M is a constant that can be arbitrarily determined according to the needs of those skilled in the art.

FIG. 6 illustrates a process of setting a feature vector of FIG. 4 (operation 412).

As can be seen in FIG. 6, the feature vector setting process is a process of constructing a K-dimensional feature vector by integrating M-dimensional vectors detected for each of a plurality of local windows.

A feature vector of P × M dimension is constructed by using M affine moments extracted for each local window. At this time, one local region is composed of P local windows, and M affine moments are obtained from each local window. Therefore, a feature vector of size K = M × P is generated for one local area. If N point values are extracted from the image data, the number of feature vectors resulting from one image data becomes N and the structure of the output value is a matrix having a size of N by K or an N × K dimensional vector.

In conclusion, since the K-dimensional feature vector is detected per feature point, the output becomes a matrix or dimension vector having a size of N by K when the number of detected points is N.

Performance evaluation for the present invention was based on the recognition rate based on the Nearest Neighbor Rule (NNR) by the Euclidian Distance value.

Referring to Table 1, Table 2, Figure 7 and Figure 8 describes the evaluation results.

Table 1 shows the results of the recognition rate performance evaluation according to the scale transformation and translation transformation of the input image information with respect to the conventional feature extraction method (SIFT) and the feature extraction method (AIFE) according to the present invention.

Class (C #) C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 sum SIFT 35.3 81.2 81.2 37.5 50.0 62.5 66.7 66.7 33.3 81.3 68.6 61.5 63.5 AIFE 82.3 87.5 87.5 62.5 75.0 56.3 77.8 88.9 77.8 75.0 81.3 92.3 77.6

For the performance evaluation of Table 1, 180 test images were used for a total of 12 natural marks (classes). The size conversion images were randomly generated such that they moved up and down by a predetermined distance from the reference image and had a maximum distance of 1.5 m. The left and right shift transformed images are randomly generated to move up to 50% from the reference image at regular intervals, and generate a maximum of 50% conversion from the reference image.

Referring to Table 1, it can be seen that the feature extraction method (AIFE) according to the present invention shows an average performance improvement of about 14% (about 1.3 times) than the conventional feature extraction method (SIFT).

FIG. 7 is a graph illustrating a recognition rate performance evaluation result according to scale transformation and translation transformation of input image information with respect to a conventional feature extraction method (SIFT) and a feature extraction method according to the present invention (AIFE). .

7 graphically illustrates the results of Table 1. As in Table 1, according to the present invention, the size transformation and the left-right shift transformation show similar or improved recognition performance as the conventional feature extraction method (SIFT).

Table 2 shows the results of the recognition rate performance evaluation according to the 3D rotation, that is, the conversion of the depth direction, of the input image information with respect to the SIFT method and the AIFE method according to the present invention. Table shown.

Class (C #) C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 sum SIFT 22.2 11.1 55.6 44.4 55.6 11.1 11.1 22.2 22.2 55.6 31.1 AIFE 66.7 55.5 77.8 88.9 66.7 33.3 44.4 88.9 66.7 55.6 64.4

In the performance evaluation of Table 2, 90 test images of 10 classes (natural marks-C #) were used (9 of each class). The three-dimensional rotation conversion test used images generated to move up to an angle from the reference image with an angle difference of up to 80 °.

As can be seen from Table 2, it can be seen that the feature extraction method (AIFE) according to the present invention has improved performance by about 33% (about 2 times) on average.

FIG. 8 is a graph illustrating a recognition rate performance evaluation result according to a depth direction transformation of input image information with respect to the conventional feature extraction method (SIFT) and the feature extraction method according to the present invention (AIFE).

8 graphically illustrates the results of Table 2. Referring to FIG. 8, the conventional feature extraction method (SIFT) shows an average recognition rate of less than 50% for the depth transformation. Therefore, the conventional feature extraction method (SIFT) cannot cope with the conversion in the depth direction. On the other hand, the feature extraction method (AIFE) according to the present invention shows an average recognition rate of 64.4% for the depth transformation. Therefore, according to the present invention, it is possible to recognize the depth direction transformation that has not been recognized in the past.

In conclusion, as a result of the comparison experiment by the feature extraction method (AIFE) and the SIFT method according to the present invention, about 1.3 times in the case of the scale (Scale) and translation (Translation), rotation in the three-dimensional depth direction In the case of (3D-Rotation), the recognition performance is improved by about 2 times or more.

Preferably, a program for executing the feature extraction method of the image information of the present invention on a computer can be recorded on a computer-readable recording medium.

The invention can be implemented via software. When implemented in software, the constituent means of the present invention are code segments that perform the necessary work. The program or code segments may be stored on a processor readable medium or transmitted by a computer data signal coupled with a carrier on a transmission medium or network.

Although the present invention has been described with reference to one embodiment shown in the drawings, this is merely exemplary and will be understood by those of ordinary skill in the art that various modifications and variations can be made therefrom. However, such modifications should be considered to be within the technical protection scope of the present invention. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

As described above, according to the present invention, by detecting the feature points including the feature of the image from the image information and constructing a feature vector around the detected feature points, it is easy to apply to the real environment, the feature of the image The extraction process is not sensitive to changes in the surrounding environment and is robust to affine deformation, and it is effective to improve the recognition performance of the image by setting a plurality of local regions for feature extraction of an image.

Claims (21)

A feature point detector for calculating a curvature value of the image information and detecting feature points for feature extraction using the curvature value; And And a feature extractor for detecting vectors for affine transformation in a plurality of regions centered on the feature points and setting the set of vectors as a feature vector. The method of claim 1, The feature point detection unit And a filter unit for performing Gaussian filtering on the image information. The method of claim 1, The feature point detection unit A derivative operator for calculating differential values of the image information; And And a curvature calculating unit configured to calculate a curvature value of the image information by using the derivative values. The method of claim 1, The feature point detection unit A corner detector for detecting corners of the image information by using a local maximum of the curvature value; And And a point selector which selects the feature points from the points constituting the detected corners. The method of claim 4, wherein The corner detection unit And extracting the corners of the image information by applying the curvature value to a Harris corner detector. The method of claim 1, The feature point detection unit And a point removing unit for removing the arbitrary feature points when a distance between any one of the feature points and a boundary of the image information is equal to or less than a preset reference size. The method of claim 1, The feature extraction unit A local area setting unit which sets the feature points as the center of each area and sets local areas having a predetermined reference size; An affine moment detector for detecting vectors for affine transformation for each local region; And And a feature vector setting unit for setting a set of detected vectors for each local region as a feature vector for the image information. The method of claim 7, wherein The local area setting unit Local parts are generated by dividing the local area into a size smaller than the reference size. The affine moment detector And detecting the vectors for the affine transformation for each local window, and setting the set of the detected vectors as the detected vectors for the affine transformation of the local region. The method of claim 7, wherein The affine moment detector Image information, characterized in that the vectors for the affine deformation of the local region are represented by a function Z for normalizing the function F of the image information having the same value regardless of the affine deformation with the zeroth moment of the function F. Features of the extraction device. The method of claim 1, The feature extraction unit And a volatile memory device for storing the feature vector. The method of claim 1, The feature extraction unit And a nonvolatile memory device for storing the feature vector. Calculating curvature values of the image information and detecting feature points for feature extraction using the curvature values; And Detecting vectors for affine transformation in a plurality of regions centered on the feature points, and setting the set of vectors as a feature vector. The method of claim 12, Detecting the feature point And performing Gaussian filtering on the image information. The method of claim 12, Detecting the feature point Calculating differential values of the image information; And And calculating a curvature value of the image information by using the differential values. The method of claim 12, Detecting the feature point Detecting corners of the image information by using a local maximum of the curvature value; And And selecting the feature points from among the points constituting the detected corners. The method of claim 15, Detecting the corners And detecting corners of the image information by applying the curvature value to a Harris corner detector. The method of claim 12, Detecting the feature point And removing any of the feature points if the distance between any one of the feature points and a boundary of the image information is equal to or less than a preset reference size. The method of claim 12, Setting the feature vector Setting local areas having the feature points as the center of each area and having a predetermined reference size; Detecting vectors for affine transformation for each local region; And And setting a set of detected vectors for each local region as a feature vector for the image information. The method of claim 18, Setting the local areas is Generating local windows by dividing the local area into a size smaller than the reference size; Detecting vectors for the affine transformation Detecting vectors for affine transformation for each local window, and setting the set of detected vectors as a detected vector for affine transformation of the local region. . The method of claim 18, Detecting vectors for the affine transformation Image information characterized in that the vectors for the affine deformation of the local region are represented by a function Z for normalizing the function F for the image information having the same value irrespective of the affine deformation with the zero-order moment of the function F. Feature extraction method. A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 12 to 20.

Images