JP6234107B2 - Feature value generation device, feature value generation method, and program - Google Patents

Description

  The present invention relates to a technique for generating feature quantities of feature points of an image.

  A feature point on an image is generally a point with a unique pixel structure and easy correspondence. For example, intersections of corners and lines are feature points. Tracking of moving objects using matching information between feature points between a plurality of images, camera position estimation, stereo vision technology, and the like are widely used. The feature amount is a unique value of each feature point, and is used for calculation of similarity between feature points and image recognition during matching.

  As a feature amount calculation method, Patent Document 1 describes a method called SIFT (Scale Invariant Feature Transformation). This is a method in which a 128-dimensional vector indicating the distribution of luminance gradient vectors of pixels in a certain region around a feature point is used as a feature amount. The feature amount calculated by this method is strong against image rotation, scale change, and illumination change.

US Pat. No. 6,711,293

  In the conventional method, a feature amount is always generated from pixel information of a fixed rectangular area near the feature point. For this reason, the generated feature quantity is weak to the movement of the subject, and the feature quantity at the same feature point changes greatly when the pixel structure in the rectangular area changes. In camera position estimation and moving object tracking, subject movement and viewpoint changes frequently occur. For this reason, in the conventional method, the feature point of the same subject in a plurality of images may not be regarded as the same point due to a slight change in the surrounding structure such as the background, and matching accuracy may be lowered.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a technique capable of generating a feature quantity that is not easily influenced by the movement of a subject in an image and the movement of surroundings.

The present invention has been made to solve the above-described problem, and includes a feature point detection unit that detects a feature point from an input image, and a pixel that generates pixel information from pixel values around the feature point in the input image. An information generation unit, a feature point pixel information distribution generation unit that generates a first distribution of the pixel information of the first region including the feature point, and a sub feature point that is a feature point around the feature point A sub-feature point detection unit, a sub-feature point pixel information distribution generation unit that generates a second distribution of the pixel information in the second region including the sub-feature point, the first distribution, and the second distribution A feature amount generation unit that generates a feature amount at the feature point based on the distribution, and the sub feature point pixel information distribution generation unit is configured to detect the sub feature point detected by the sub feature point detection unit. The number of divisions of the second area is determined according to the number, and The feature quantity generation device is characterized in that the second area is divided so that the number of divided areas is equal to the determined division number, and the second distribution is generated for each divided area .

The present invention also includes a step of detecting a feature point from the input image, a step of generating pixel information from pixel values around the feature point in the input image, and the pixel in the first region including the feature point Generating a first distribution of information, detecting a sub-feature point from the periphery of the feature point, and generating a second distribution of the pixel information of the second region including the sub-feature point And generating a feature quantity at the feature point based on the first distribution and the second distribution, and in the step of generating the second distribution, The number of divisions of the second area is determined according to the number, the second area is divided so that the number of divided areas is the determined number of divisions, and the second area is divided for each divided area. A feature generation method characterized by generating a distribution .

The present invention also includes a step of detecting a feature point from the input image, a step of generating pixel information from pixel values around the feature point in the input image, and the pixel in the first region including the feature point Generating a first distribution of information, detecting a sub-feature point from the periphery of the feature point, and generating a second distribution of the pixel information of the second region including the sub-feature point And generating a feature amount at the feature point based on the first distribution and the second distribution, and causing the computer to execute the second distribution. In the step, the number of divisions of the second region is determined according to the number of the sub feature points, the second region is divided so that the number of divided regions becomes the determined number of divisions, and after the division The second distribution is generated for each area of It is a program to be formed.

  According to the present invention, based on the first distribution of pixel information of the first region including the feature points and the second distribution of pixel information of the second region including the sub-feature points, the feature at the feature points By generating the amount, it is possible to generate a feature amount that is not easily influenced by the movement of the subject in the image and the surrounding movement.

1 is a block diagram showing a configuration of a feature quantity generating device according to a first embodiment of the present invention. 5 is a flowchart showing a processing procedure of the feature quantity generation device according to the first embodiment of the present invention. FIG. 5 is a reference diagram showing a concept of generating a gradient histogram in the first embodiment of the present invention. FIG. 3 is a reference diagram showing a feature point of interest and sub feature points in the first embodiment of the present invention. 1 is a block diagram showing a configuration of a feature quantity generating device according to a first embodiment of the present invention. 5 is a flowchart showing a processing procedure of the feature quantity generation device according to the first embodiment of the present invention. FIG. 10 is a reference diagram showing local regions when the number of local regions is changed according to the number of sub feature points in the second embodiment of the present invention. FIG. 10 is a reference diagram showing a local region when the size of the local region is changed according to the number of sub feature points in the second embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
First, a first embodiment of the present invention will be described. FIG. 1 shows the configuration of the feature quantity generating apparatus according to this embodiment. 1 includes a feature point detection unit 10, a main direction adjustment unit 20, a pixel information generation unit 30, a feature point pixel information distribution generation unit 40, a sub feature point detection unit 50, and a sub feature point pixel information distribution. A generation unit 60 and a feature amount generation unit 70 are included.

  Hereinafter, the function of each block will be described. In the feature quantity generation device, an input image that is a target of the feature quantity generation process is input from the outside and temporarily stored in a memory (not shown) in the apparatus. Each block performs processing using the input image stored in the memory as appropriate.

  The feature point detection unit 10 detects feature points from the input image. More specifically, the feature point detection unit 10 calculates a feature evaluation value indicating the strength of a corner or an edge using the input image, detects a point having a high calculated feature evaluation value as a feature point, and detects it. The coordinates of the feature points (hereinafter referred to as feature point coordinates) are output.

  The main direction adjustment unit 20 calculates the main direction of the feature point based on the gradient angle of the pixel value of each pixel around the feature point in the input image, rotates the input image according to the main direction, and rotates the rotated image. Is generated. More specifically, the main direction adjustment unit 20 calculates the angle of the gradient of the pixel values near the feature point coordinates detected by the feature point detection unit 10, and uses the angle that occupies the majority of the calculated angles as the main direction. To detect. Further, the main direction adjustment unit 20 cuts out an area of a predetermined size centered on the feature point coordinates from the input image, and rotates the image of the cut out area by rotating the image of the cut out area according to the detected angle of the main direction. Is generated. The region to be cut out is a region including data used when the feature point pixel information distribution generation unit 40 and the sub feature point pixel information distribution generation unit 60 in the subsequent stage generate pixel information distribution.

  The pixel information generation unit 30 generates pixel information from pixel values around feature points in the rotated image generated by the main direction adjustment unit 20. More specifically, the pixel information generation unit 30 generates, as pixel information, a gradient vector including the gradient angle and length of the pixel values around the feature points in the rotated image generated by the main direction adjustment unit 20. Pixel information is information unique to each pixel, such as the angle and length of the gradient of the pixel value. The pixel information generation unit 30 may generate pixel information using at least one of the luminance value and color information (for example, color difference information Cr, Cb) of the rotated image generated by the main direction adjustment unit 20 as a pixel value. Further, the pixel information generation unit 30 may generate pixel information using a RAW image that is an image output from the image sensor as an input image.

  The feature point pixel information distribution generation unit 40 generates a distribution (first distribution) of pixel information in a region including the feature points (first region). More specifically, the feature point pixel information distribution generation unit 40 generates a distribution of pixel information at the feature points based on the pixel information in the small area including the feature point coordinates. For example, the feature point pixel information distribution generation unit 40 includes a gradient vector of pixels in a region including a feature point among the gradient vectors generated by the pixel information generation unit 30 as an element, and a histogram indicating a cumulative distribution of length with respect to an angle As the first distribution.

  The sub feature point detector 50 detects sub feature points around the feature point. More specifically, the sub feature point detection unit 50 is a sub point that is a characteristic point other than the feature point from a predetermined range centered on the feature point in the rotated image generated by the main direction adjustment unit 20. The feature points are detected, and the coordinates of the detected sub feature points (hereinafter referred to as sub feature point coordinates) are output. At this time, the sub feature point detection unit 50 sets a point whose feature evaluation value is lower than the feature evaluation value of the feature point detected by the feature point detection unit 10 as a sub feature point candidate. Further, the sub feature point detection unit 50 determines one or a plurality of points having a feature evaluation value higher than a predetermined value among the sub feature point candidates as sub feature points.

  The sub feature point pixel information distribution generation unit 60 generates a distribution (second distribution) of pixel information in a region (second region) including the sub feature points. More specifically, the sub-feature point pixel information distribution generation unit 60 generates a distribution of pixel information at each sub-feature point based on the pixel information in the small area including the sub-feature point coordinates. For example, the sub-feature point pixel information distribution generation unit 60 uses the gradient vector of the pixel including the sub-feature point in the gradient vector generated by the pixel information generation unit 30 as an element, and calculates the cumulative distribution of the length with respect to the angle. The histogram shown is generated as the second distribution.

  The feature amount generation unit 70 generates a feature amount at the feature point based on the distribution of the pixel information of the feature point and the distribution of the pixel information of the sub feature point. More specifically, the feature amount generation unit 70 outputs a set obtained by combining the elements of the pixel information distribution of feature points and the pixel information distribution of sub feature points as the feature amount of the feature points. For example, the feature quantity generation unit 70 arranges the pixel information distribution of the feature points and the distribution of the pixel information related to each of the plurality of sub feature points in order based on the feature evaluation value of each of the plurality of sub feature points. Is output as a feature value at the feature point.

  Next, with reference to FIG. 2, the feature value generation processing by the feature value generation device will be specifically described. FIG. 2 shows the procedure of the feature quantity generation process. When the input image is supplied and the generation of the feature quantity is instructed, the feature quantity generation device starts the feature quantity generation process.

  When the process is started in step S1, the feature point detection unit 10 detects the position of the feature point from the input image in step S2. For example, the feature point detection unit 10 uses a method of calculating a feature evaluation value indicating the strength of the feature (HarrisCornerDetecter, etc.), and the feature evaluation value is a predetermined number of points from which the feature evaluation value is equal to or higher than a predetermined threshold. Points within a few are detected as feature points.

  Subsequently, in step S3, the feature point detection unit 10 selects one of the detected feature points, and outputs the coordinates of the selected feature point (hereinafter referred to as the feature point of interest) to the main direction adjustment unit 20 in the subsequent stage. To do.

  Subsequently, processing for obtaining the main direction of the feature point of interest is performed. The main direction is a dominant angle that occupies a majority of the gradient angles of the pixels near the feature point of interest.

  More specifically, in step S4, the main direction adjustment unit 20 calculates the angle of the gradient of the luminance value for each of the plurality of pixels near the feature point of interest. Note that the data used to generate the gradient angle is not limited to the luminance value, but may be color information or a pixel value of the RAW image. The gradient angle θ (x, y) of the pixel at the coordinate position (x, y) is calculated by the following equation (1). In equation (1), the luminance values of the pixels adjacent to the pixel at the coordinate position (x, y) are I (x−1, y), I (x + 1, y), I (x, y−1), I ( x, y + 1). The reference direction of the angle is the upward direction (Y direction) of the image, and the angle increases counterclockwise.

  Subsequently, in step S5, the main direction adjustment unit 20 generates a histogram in which the gradient angle generated in step S4 is a bin (horizontal axis of the histogram) and the number of gradient angles is a frequency (vertical axis of the histogram). Then, the angle of the gradient having the maximum frequency in the generated histogram is determined as the main direction of the feature point of interest.

  Subsequently, in step S6, the main direction adjusting unit 20 rotates the input image based on the main direction obtained in step S5. More specifically, the main direction adjusting unit 20 cuts out a region of a predetermined size centered on the target feature point of the input image, and centered on the target feature point in the direction opposite to the main direction angle. A rotated image is generated by rotating the cut-out region by the size of the angle. The main direction of the feature point of interest in the rotated image after rotation is the upward direction of the image. The main direction adjustment unit 20 outputs the rotated image to the subsequent pixel information generation unit 30 and the sub feature point detection unit 50. The size of the region from which the image is cut out is a size that includes a region used for generating pixel information distribution in the subsequent stage. For example, in the present embodiment, the region cut out from the input image to generate the rotated image is a rectangular region of 32 pixels × 32 pixels.

  By performing the main direction adjustment including the processes in steps S4 to S6, the pixel information generation unit 30 in the subsequent stage can generate pixel information that is invariant to the rotation of the subject. By rotating the image based on the angle of the main direction of the feature point, the main direction of the rotated feature point always faces the same direction regardless of the rotation of the subject. Since the luminance gradient vector of the pixel value generated by the pixel information generation unit 30 at the subsequent stage is generated from an image rotated according to the angle of the main direction, the angle of the luminance gradient vector is an angle formed with the main direction (the main direction is The angle of the reference direction). For this reason, even if the subject rotates, if the subject is the same, the angles constituting the brightness gradient vector always indicate the same angle. Thereby, a change in the feature amount due to the rotation of the subject can be reduced. In the above description, the rotation of the subject is assumed. However, when the device that captured the image itself is rotated, the main direction is adjusted according to the rotation as described above. The change in quantity can be reduced.

  Subsequently, in step S7, the pixel information generation unit 30 generates a luminance gradient vector that is a gradient vector of the luminance value based on the luminance value of the rotated image generated in step S6. The length Norm (x, y) and the angle Gradient (x, y) of the luminance gradient vector at the coordinate position (x, y) are calculated by the following equations (2) and (3), respectively. In Equations (2) and (3), the rotated image is represented as R, and the luminance values of adjacent pixels are R (x−1, y), R (x + 1, y), R (x, y−1) , R (x, y + 1). In addition, the angle is quantized to a multiple of 45 degrees and is represented by 8 angles in increments of 45 degrees from 0 degrees to 315 degrees.

  Subsequently, in step S8, the feature point pixel information distribution generation unit 40 generates a histogram (hereinafter referred to as a gradient histogram) representing the distribution of luminance gradient vectors in a small region including the target feature point. The size of the small region is preferably a small size within a range where the feature points can be identified. FIG. 3 illustrates the concept of generating a gradient histogram. FIG. 3 (a) shows an example of a small region R0 for generating a gradient histogram in the rotated image generated in step S6. Small circles in FIG. 3 (a) indicate pixels. In the present embodiment, for example, as shown in FIG. 3A, the small region R0 for generating the gradient histogram is a rectangular region of 12 × 12 pixels centered on the target feature point P0. In the present embodiment, a 3 × 3 pixel region obtained by dividing a small region into 16 is defined as a local region.

  One histogram (hereinafter referred to as a local histogram) is generated for each local region. This local histogram represents the cumulative length with respect to the angle of the luminance gradient vector in the local region. FIG. 3 (b) shows an example of a local histogram generated from the luminance gradient vector of each pixel in one local region. As shown in FIG. 3 (b), the bin of the local histogram is an angle, and the frequency is the total length of the luminance gradient vector. Since the angle of the brightness gradient vector is represented by an angle in 8 directions, the number of bins in one local histogram is 8. When the lengths of a plurality of luminance gradient vectors correspond to the same angle, the frequency of the local histogram at that angle is the total (sum) of the lengths of the plurality of luminance gradient vectors.

  FIG. 3 (c) shows an example of a gradient histogram. The feature point pixel information distribution generation unit 40 generates a local histogram for each of the 16 local regions, and generates a gradient histogram of the feature point of interest by connecting the 16 generated local histograms. Since the number of bins in one local histogram is 8, as shown in FIG. 3 (c), a histogram with 8 × 16 = 128 bins is the gradient histogram of the feature point of interest. Through the above processing, the gradient histogram of the feature point of interest is completed.

  Subsequently, in step S9, the sub feature point detection unit 50 detects the sub feature point. The sub feature points are characteristic points around the feature point of interest, and the number N of sub feature points is 2 in the example of the present embodiment.

  More specifically, the sub feature point detection unit 50 detects the sub feature point within a predetermined range centered on the feature point of interest in the rotated image generated by the main direction adjustment unit 20. As the detection method, a feature evaluation value calculation method such as Harris Corner Detector is used. The sub-feature point detection unit 50 sets a point whose calculated feature evaluation value is equal to or greater than a predetermined threshold as a sub-feature point candidate, and has a higher feature evaluation value (one or more points) among the sub-feature point candidates. ) As a sub-feature point.

  The threshold value set at this time is, for example, a threshold value lower than the threshold value of the feature evaluation value used by the feature point detection unit 10 in step S2 described above. Thereby, even a point with a low feature evaluation value can be a sub-feature point. This is because if the threshold value is high, the sub feature points may not be detected. In this case, the number of dimensions of the final feature amount decreases, and the characteristics of the feature points are difficult to appear. However, if the threshold value is not specified or the threshold value is too low, many points with the same feature evaluation value are generated, and different sub-feature points are selected when generating the feature amount of the same feature point of another image. Therefore, it is desirable that a threshold value greater than or equal to a predetermined value is provided because the feature amount is likely to be unstable. In the example of the present embodiment, there are two sub feature points, and numbers 1 to 2 are assigned to the sub feature points in descending order of feature evaluation values.

  Subsequently, in step S10, a control unit (not shown) initializes the sub feature point number i to be processed. That is, the value of the number i is set to 1.

  Subsequently, in step S11, a control unit (not shown) determines whether or not there is an i-th sub feature point. If the i-th sub feature point exists, the process proceeds to step S12.

  In step S12, the sub feature point pixel information distribution generation unit 60 uses the luminance gradient vector generated by the pixel information generation unit 30 to generate a gradient histogram of the i-th sub feature point. The gradient histogram is generated by a method similar to the method of generating the gradient histogram in step S8 described above for a small region centered on the i-th sub feature point coordinate.

  FIG. 4 shows examples of feature points of interest and sub-feature points on the rotated image. Small circles in FIG. 4 indicate pixels. An attention feature point P0 is set at the center of the rotated image Img1, and two sub feature points P1 and P2 are set around the attention feature point P0. A 12 × 12 pixel rectangular region including the target feature point P0 is a small region R0 used for generating a gradient histogram. Similarly, 12 × 12 pixel rectangular regions including the sub-feature points P1 and P2 are small regions R1 and R2 used to generate a gradient histogram, respectively. A region indicated by a broken line obtained by dividing each of the small regions R0, R1, and R2 into 16 is a local region. In step S12, for each sub feature point, a gradient histogram of the luminance gradient vector of the small region including the sub feature point is generated in the same manner as the target feature point.

  In step S11, if there is no point whose feature evaluation value is equal to or greater than the threshold within the range in which the sub feature point is detected, and the i-th sub feature point does not exist, the process proceeds to step S13. In step S13, the sub-feature point pixel information distribution generation unit 60 generates a gradient histogram whose frequencies are all zero. Alternatively, the gradient histogram of the feature point generated in step S8 may be used as the final feature amount of the feature point of interest without generating the gradient histogram of the non-existing sub feature point. After the gradient histogram of the i-th sub feature point is completed, the process proceeds to step S14.

  In step S14, a control unit (not shown) determines whether or not the processing of all sub feature points has been completed. If the sub-feature point processing has not ended, the processing proceeds to step S15. In step S15, the control unit (not shown) increments (increases by 1) the number i of the sub feature point to be processed. Thereafter, the process returns to step S11, and the process of generating the gradient histogram of the next sub feature point is performed. When the processing of all the sub feature points is completed, the processing proceeds to step S16.

  In step S16, the feature quantity generation unit 70 arranges the gradient histogram of the feature point of interest and the gradient histogram of the two sub-feature points and connects them to generate a histogram with 128 × 3 = 384 bins. At this time, for example, the gradient histograms of two sub-feature points are arranged after the gradient histogram of the feature point of interest. Further, the gradient histograms of the sub feature points are arranged in an order based on the feature evaluation values of the sub feature points (for example, in descending order of the feature evaluation values). By arranging the histograms in descending order of the feature evaluation values, when generating feature amounts of the same feature points from different images, the feature amounts are stable because the histograms are easily arranged in the same order.

  The histogram with 384 bins generated by the above method becomes the feature amount of the feature point of interest, and the feature amount generation processing for one feature point is completed. Subsequently, in step S17, a control unit (not shown) determines whether or not the processing in steps S2 to S16 has been performed on all feature points. If there is a feature point that has not been subjected to the processes in steps S2 to S16, the process returns to step S3, and the processes in steps S2 to S16 are performed on the next feature point. If all feature points have been processed, the feature value generation process ends in step S18.

  In the above example, two sub feature points are detected, but only one sub feature point may be detected, or three or more sub feature points may be detected.

  As described above, by combining multiple feature values generated using data in a narrow range into one feature value, even if there is a change around the feature point of interest, the change amount of the feature value is suppressed. be able to.

  The function of each block constituting the feature quantity generation device according to the present embodiment is such that, for example, a program (CPU) stored in a ROM (not shown) is read and executed by the computer (CPU) of the feature quantity generation device. It may be realized as a function. The program may be provided by a “computer-readable recording medium” such as a flash memory. Further, the above-described program is input to the feature quantity generation device by being transmitted to the feature quantity generation device via a transmission medium or by a transmission wave in the transmission medium from a computer storing the program in a storage device or the like. May be. Here, the “transmission medium” for transmitting the program is a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the above-described program may realize a part of the functions described above. Furthermore, the above-described program may be a so-called difference file (difference program) that can realize the above-described function in combination with a program already recorded in the computer.

  In the above description, the feature quantity generation device has the main direction adjustment unit 20, but the feature quantity generation device may not have the main direction adjustment unit 20. FIG. 5 shows a configuration of a modification of the feature quantity generation device according to the present embodiment. 5 includes a feature point detection unit 10, a pixel information generation unit 30, a feature point pixel information distribution generation unit 40, a sub feature point detection unit 50, a sub feature point pixel information distribution generation unit 60, a feature amount. A generation unit 70 is included.

  The feature point coordinates (feature point coordinates) detected by the feature point detection unit 10 are output to the pixel information generation unit 30 and the sub feature point detection unit 50. The pixel information generation unit 30 generates pixel information from pixel values around feature points in the input image. The sub feature point detection unit 50 detects a sub feature point from the periphery of the feature point in the input image. Except that the pixel information generation unit 30 and the sub feature point detection unit 50 perform processing on the input image instead of the rotated image, the processing performed by the feature amount generation device illustrated in FIG. This is the same as the processing performed by the apparatus.

  FIG. 6 shows a procedure of feature quantity generation processing performed by the feature quantity generation apparatus shown in FIG. The processes of steps S4 to S6 in the process shown in FIG. 2 are deleted, and the process of step S7 is changed to the process of step S7 '. In the process shown in FIG. 6, after the feature point is selected in step S3, in step S7 ', the pixel information generation unit 30 generates a luminance gradient vector from the luminance value of the input image. Other than the above, the processing is the same as that shown in FIG.

  The feature quantity generation device shown in FIG. 5 does not perform the process of reducing the change in the feature quantity due to the rotation of the subject or the rotation of the apparatus itself (the processes in steps S4 to S6 in FIG. 2). It is possible to generate a feature quantity that is not easily affected by the movement of the subject and the surrounding movement.

  As described above, according to the present embodiment, a gradient histogram of feature points, which is a distribution (first distribution) of pixel information of a small region (first region) including feature points, and a small region including sub feature points. Based on the gradient histogram of the sub-feature points, which is the distribution (second distribution) of pixel information in the region (second region), by generating feature quantities at the feature points, It is possible to generate robust feature values that are not easily affected by movement.

  In addition, by selecting a point having a feature evaluation value lower than the feature evaluation value of the feature point as a candidate for the sub feature point, the sub feature point can be detected even in a region having a few strong feature points. It can be prevented from decreasing.

  In addition, by determining one or a plurality of points whose feature evaluation values are higher than a predetermined value among the sub feature point candidates as a sub feature point, a stronger sub feature point is selected. The individuality of the feature value becomes stronger. For this reason, when generating the feature quantity of the same feature point in another image, the possibility that the same sub-feature point is detected increases, and the stability of the feature quantity increases.

  In addition, a feature amount at a feature point is a set obtained by combining a gradient histogram of feature points and an array of gradient histograms related to each of a plurality of sub feature points in order based on the feature evaluation values of each of the plurality of sub feature points. Thus, even when the positional relationship of the sub-feature points changes in another image, the order of the gradient histograms in the feature amounts is the same if the captured points are the same. For this reason, it is possible to generate a feature amount that is robust to surrounding movement.

  In addition, by generating a gradient vector including the gradient angle and length of pixel values around feature points in the input image as pixel information, it is possible to generate a feature value that is robust to changes in illumination.

  Further, among the gradient vectors generated by the pixel information generation unit 30, the gradient vector of the pixel in a small area centered on the sub-feature point is used as an element, and a gradient histogram indicating the cumulative distribution of length with respect to the angle is generated. Thus, it is possible to generate a feature amount including information on sub-feature points that is robust to illumination changes.

  Further, among the gradient vectors generated by the pixel information generation unit 30, the gradient change of the pixel in the region centered on the feature point is used as an element, and a histogram showing the cumulative distribution of length with respect to the angle is generated, thereby changing the illumination. It is possible to generate a feature quantity including information on robust feature points.

  In addition, by using only luminance information as pixel values when generating pixel information, it is possible to generate a feature amount that reflects the characteristics of feature points having different brightness even with the same color with a small amount of data. . In addition, by using only color information as pixel values when generating pixel information, it is possible to generate a feature amount that reflects the characteristics of feature points having a color change with a small amount of data even when the luminance is the same. . Further, by using the luminance information and the color information as the pixel values when generating the pixel information, it is possible to generate a feature amount that takes into account both the color information and the luminance information.

  In addition, by generating pixel information using a RAW image as an input image, it is possible to search for a set that takes into account both color information and luminance information with a small amount of data.

  In addition, the main direction of the feature point is calculated based on the gradient angle of the pixel value of each pixel around the feature point in the input image, and the input image is rotated in accordance with the main direction, thereby rotating the subject or the device. It is possible to generate a feature quantity that is robust to its own rotation.

(Second embodiment)
Next, a second embodiment of the present invention will be described. The configuration of the feature quantity generation device according to the present embodiment is the same as the configuration of the feature quantity generation device shown in FIG. In the present embodiment, the number and size of local regions when generating a gradient histogram are variable.

  For example, the sub-feature point pixel information distribution generation unit 60 determines the number of sub-regions centered on the sub-feature point according to the number of sub-feature points detected by the sub-feature point detection unit 50, A small region is divided so that the number of regions becomes the determined number of divisions, and a gradient histogram of sub feature points is generated for each divided local region. Alternatively, the sub feature point pixel information distribution generation unit 60 determines the division size of the small region centered on the sub feature point according to the number of sub feature points detected by the sub feature point detection unit 50, The small area is divided so that the size of the area becomes the determined division size, and a gradient histogram is generated for each divided local area.

  In addition, the feature point pixel information distribution generation unit 40 determines the number of divisions of the small region centered on the feature point according to the number of divisions determined by the sub feature point pixel information distribution generation unit 60. More specifically, the feature point pixel information distribution generation unit 40 determines the number of divisions of the small area centering on the feature point to be the same as the number of divisions determined by the sub feature point pixel information distribution generation unit 60.

  Alternatively, the feature point pixel information distribution generation unit 40 determines the division size of the small region centered on the feature point according to the division size of the small region determined by the sub feature point pixel information distribution generation unit 60. More specifically, the feature point pixel information distribution generation unit 40 determines the division size of the small area centered on the feature point to the same size as the division size determined by the sub feature point pixel information distribution generation unit 60.

  FIG. 7 shows an example of local regions when the number of local regions is changed according to the number of sub feature points. Small circles in FIG. 7 indicate pixels. An attention feature point P10 is set at the center of the rotated image Img11, and three sub feature points P11, P12, and P13 are set around the attention feature point P10. A 12 × 12 pixel rectangular region including the target feature point P10 is a small region R10 used to generate a gradient histogram. Similarly, 12 × 12 pixel rectangular regions including the sub-feature points P11, P12, and P13 are small regions R11, R12, and R13 used to generate a gradient histogram, respectively. A region indicated by a broken line obtained by dividing each of the small regions R10, R11, R12, and R13 into 12 is a local region.

  If the number of sub feature points is 3 and the number of local regions is 12, and if the number of bins in one local histogram is 8, the number of bins in one gradient histogram is 12 × 8 = 96. Since the number of bins is required for a total of four points including three sub feature points and one feature point of interest, the final feature quantity dimension of the feature point of interest is 96 × 4 = 384 dimensions. For this reason, even when the number of sub-feature points is increased from 2 in the first embodiment, the number of dimensions of the feature amount can be suppressed to the same number of dimensions as that in the first embodiment. Since the number of feature dimensions is proportional to the product of the total number of feature points and sub-feature points and the number of local regions, if the number of sub-feature points is increased and the number of local regions is decreased, the feature amount An increase in the number of dimensions can be suppressed.

  In the memory (not shown) of the feature quantity generation device, information on the size of the small area (small areas R10, R11, R12, R13 in FIG. 7), information on the dimension number of the feature quantity, and one local histogram bin Stores information about numbers. The sub-feature point pixel information distribution generation unit 60 includes the number of sub-feature points detected by the sub-feature point detection unit 50, the size of the small area (12 × 12 pixels), the number of feature dimensions (384 dimensions), The number of local regions is determined according to the number of bins (8) of one local histogram. For example, when the number of sub feature points is 2, the number of bins for a total of three points including two sub feature points and one feature point of interest is 384 ÷ 3 = 128. Since the number of bins in one local histogram is 8, the number of local regions is 128 ÷ 8 = 16. When the number of sub feature points is 3, the total number of bins for 4 points including 3 sub feature points and 1 feature point of interest is 384 ÷ 4 = 96. Since the number of bins in one local histogram is 8, the number of local regions is 96 ÷ 8 = 12.

  FIG. 8 shows an example of a local region when the size of the local region is changed according to the number of sub feature points. Small circles in FIG. 8 indicate pixels. An attention feature point P20 is set at the center of the rotated image Img21, and four sub feature points P21, P22, P23, and P24 are set around the attention feature point P20. A rectangular region of 8 × 8 pixels including the target feature point P20 is a small region R20 used to generate a gradient histogram. Similarly, rectangular regions of 8 × 8 pixels including sub feature points P21, P22, P23, and P24 are small regions R21, R22, R23, and R24 used to generate a gradient histogram, respectively. A region indicated by a broken line obtained by dividing each of the small regions R20, R21, R22, R23, and R24 into 16 is a local region.

  When the number of sub-feature points is 5 and the size of the local area is 2 × 2 = 4 pixels, the number of pixels for obtaining data used to generate the feature amount is 4 × 16 × 5 = 320 pixels. The number of pixels in the first embodiment (FIG. 4) is 9 × 16 × 3 = 432 pixels. For this reason, compared to the first embodiment, it is possible to reduce the number of pixels from which data used to generate the feature amount is obtained. In this way, by changing the size of the local region according to the number of sub feature points, the feature amount with the increased number of sub feature points can be obtained without increasing the amount of calculation and the data transfer amount for generating the feature amount. Can be generated.

  Information on the number of local regions (16) in the small regions (small regions R20, R21, R22, R23, and R24 in FIG. 8) is stored in a memory (not shown) of the feature quantity generation device. The sub-feature point pixel information distribution generation unit 60 determines the size of the local region based on the number of sub-feature points detected by the sub-feature point detection unit 50 and information on the number of local regions (16) in the small region. To do. When the number of sub feature points is 2, for example, the size of the local region is 12 × 12 pixels. In this case, the number of pixels is 432 pixels as described above. When the number of sub feature points is 5, the size of the local region is determined such that the number of pixels does not exceed 432 pixels with respect to 5 sub feature points and 16 local regions. As a result, for example, the size of the local area is determined as 2 × 2 = 4 pixels.

  As described above, according to this embodiment, according to the number of sub feature points, the number of divisions of the small region centering on the sub feature points is determined, and the number of divided regions becomes the determined number of divisions. By dividing a small region into two and generating a gradient histogram of sub feature points for each of the divided local regions, the number of dimensions of the feature amount can be kept constant. For this reason, it is possible to generate a feature amount suitable for an image without changing the use amount or processing time of the storage area.

  In addition, according to the number of sub feature points, the division size of the small region centered on the sub feature point is determined, and the small region is divided so that the size of the divided region becomes the determined division size. By generating a gradient histogram for each local region, it is possible to generate a feature amount without increasing the amount of calculation when generating the feature amount even if the number of sub feature points is increased.

  As described above, the embodiments of the present invention have been described in detail with reference to the drawings. However, the specific configuration is not limited to the above-described embodiments, and includes design changes and the like without departing from the gist of the present invention. .

  10 feature point detection unit, 20 main direction adjustment unit, 30 pixel information generation unit, 40 feature point pixel information distribution generation unit, 50 sub feature point detection unit, 60 sub feature point pixel information distribution generation unit, 70 feature amount generation unit

Claims (13)

A feature point detector for detecting feature points from the input image;
A pixel information generation unit that generates pixel information from pixel values around the feature points in the input image;
A feature point pixel information distribution generation unit for generating a first distribution of the pixel information of the first region including the feature points;
A sub feature point detector that detects a sub feature point that is a feature point around the feature point;
A sub-feature point pixel information distribution generation unit that generates a second distribution of the pixel information of the second region including the sub-feature points;
Based on the first distribution and the second distribution, a feature amount generation unit that generates a feature amount at the feature point;
Equipped with,
The sub feature point pixel information distribution generation unit determines the number of divisions of the second region according to the number of sub feature points detected by the sub feature point detection unit, and determines the number of regions after division. A feature quantity generation device, characterized in that the second region is divided so as to have a division number, and the second distribution is generated for each divided region .   The sub feature point pixel information distribution generation unit determines a division size of the second region according to the number of the sub feature points detected by the sub feature point detection unit, and determines a size of the divided region. The feature amount generation apparatus according to claim 1, wherein the second region is divided so as to have the divided size, and the second distribution is generated for each of the divided regions.   2. The sub feature point detection unit uses the point having a feature evaluation value lower than the feature evaluation value of the feature point detected by the feature point detection unit as a candidate for the sub feature point. Feature quantity generator. The sub-feature point detecting unit, among the candidates of the sub-feature point, to claim 3, wherein evaluation values and determining the one or the sub feature points several points higher than the predetermined value The feature amount generation apparatus described.   The feature amount generation unit includes the first distribution and a distribution in which the second distributions related to each of the plurality of sub feature points are arranged in order based on the feature evaluation values of the plurality of sub feature points. The feature amount generation apparatus according to claim 1, wherein a set obtained by combining the features is used as a feature amount at the feature point.   The feature amount according to claim 1, wherein the pixel information generation unit generates, as the pixel information, a gradient vector including a gradient angle and a length of a pixel value around the feature point in the input image. Generator. The sub-feature point pixel information distribution generation unit is a histogram showing a cumulative distribution of lengths with respect to angles, using the gradient vector of the pixels in the second region among the gradient vectors generated by the pixel information generation unit as elements. Is generated as the second distribution. The feature amount generation apparatus according to claim 6 . The feature point pixel information distribution generation unit includes a histogram indicating a cumulative distribution of length with respect to an angle, using the gradient vector of the pixels in the first region among the gradient vectors generated by the pixel information generation unit. The feature quantity generation device according to claim 7 , wherein the feature quantity generation apparatus generates the first distribution.   The feature value generation apparatus according to claim 1, wherein the pixel information generation unit generates the pixel information using at least one of a luminance value and color information of the input image as the pixel value.   The feature amount generation apparatus according to claim 1, wherein the pixel information generation unit generates the pixel information using a RAW image as the input image. A main direction adjusting unit that calculates a main direction of the feature point based on a gradient angle of a pixel value of each pixel around the feature point in the input image, and rotates the input image according to the main direction; Equipped,
The feature amount generation apparatus according to claim 1, wherein the pixel information generation unit generates pixel information from pixel values around the feature points in the input image rotated by the main direction adjustment unit. Detecting feature points from the input image;
Generating pixel information from pixel values around the feature points in the input image;
Generating a first distribution of the pixel information of the first region including the feature points;
Detecting sub-feature points that are feature points around the feature points;
Generating a second distribution of the pixel information of a second region including the sub-feature points;
Generating a feature quantity at the feature point based on the first distribution and the second distribution;
Equipped with,
In the step of generating the second distribution, the number of divisions of the second region is determined according to the number of the sub feature points, and the number of divided regions is the determined number of divisions. A feature amount generating method , wherein the second distribution is generated for each of the divided regions . Detecting feature points from the input image;
Generating pixel information from pixel values around the feature points in the input image;
Generating a first distribution of the pixel information of the first region including the feature points;
Detecting sub-feature points that are feature points around the feature points;
Generating a second distribution of the pixel information of a second region including the sub-feature points;
Generating a feature quantity at the feature point based on the first distribution and the second distribution;
In the step of generating the second distribution, the number of divisions of the second region is determined according to the number of the sub-feature points, and the number of regions after the division A program that divides the second area so as to have the determined number of divisions and generates the second distribution for each of the divided areas .

Images