JP2012103758A - Local feature amount calculation device and method therefor, and corresponding point search device and method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a local feature amount calculation device capable of calculating a local feature amount at high-speed.SOLUTION: A local feature amount calculation device 1 comprises: a feature point neighbor region extraction unit 11 for extracting a feature point p and a neighbor region NR set for the neighborhood of the feature point p, from an image TP; a direction intensity calculation unit 12 for calculating an edge gradient direction θ and an edge intensity m of each pixel in the neighbor region NR, respectively; a main axis direction detection unit 13 for detecting a main axis direction v in the neighbor region NR; and a local feature amount calculation unit 16 for calculating a local feature amount d of the feature point p by voting the edge intensity of an attention pixel according to a corrected edge gradient direction acquired by correcting the edge gradient direction θ of the attention pixel according to the main axis direction v, for a vote cell to which the attention pixel belongs among vote cells of a rotated vote cell pattern CP' acquired by rotating a reference vote cell pattern by an angle corresponding to the main axis direction v.

Description

  The present invention relates to a local feature amount calculation device and a local feature amount calculation method for calculating a local feature amount of an image, a corresponding point search device using the local feature amount calculation device, and a corresponding point using the local feature amount calculation method. It relates to a search method.

  Conventionally, in the field of image processing, corresponding point search processing for searching for corresponding points between two images has been frequently used. FIG. 22 is a diagram illustrating corresponding points between two images. In this way, by searching for corresponding points between a plurality of images, an image corresponding to the input image is searched from the image database, or a plurality of images of subjects photographed from different viewpoints are associated with each other. It is possible to restore the three-dimensional shape of the subject.

  Local feature quantities can be used to search for corresponding points between two images. The local feature amount is an amount that characterizes the texture pattern of the peripheral region of the target pixel, and is represented by a D-dimensional vector. Pixels with similar peripheral texture patterns have similar local features. For this reason, for example, a local feature amount is obtained for each pixel of the first and second images, and the local feature amount is compared, whereby the pixel of the first image and the pixel of the second image are compared. Can be determined whether or not the corresponding pixel (corresponding point).

  FIG. 23 is a diagram for explaining image search using local feature amounts. Image search can be performed as follows by the corresponding point search using the local feature amount (see, for example, Non-Patent Document 1). The purpose of image retrieval is to retrieve an image corresponding to the input image from the image database. First, feature points are extracted from each of a plurality of images registered in the image database, and local feature amounts of the feature points are obtained. When an image to be searched (input image) is input, feature points are also extracted from the input image, and local feature amounts are obtained for each feature point. Then, paying attention to one of the plurality of feature points of the input image, the local feature amount closest to the local feature amount of the feature point of interest is searched from the image database. Then, a vote is cast on the image to which the found local feature amount belongs. This process is repeated for all feature points in the input image, and the image that received the most votes is used as the search result.

  In addition, a three-dimensional shape can be restored from a plurality of images taken from different viewpoints by searching for corresponding points using local feature amounts (see, for example, Non-Patent Document 2). First, two are selected from a plurality of images obtained by shooting a shooting target from a plurality of shooting positions, and corresponding points between the two images are obtained. This corresponding point search is performed for all combinations of a plurality of images. Next, using the information on the corresponding points as a clue, the shooting position parameter of each image and the shape parameter of the shooting target are obtained by bundle adjustment.

  As typical methods for obtaining local feature amounts, SIFT (Scale Invariant Feature Transform) (for example, see Non-Patent Document 3 and Patent Document 1) and SURF (Speeded Up Robust Features) (for example, see Non-Patent Document 4) It has been known. In SIFT and SURF, even if the image is rotated or the scale of the image is different, it is possible to calculate local feature amounts that are invariant (not affected) by these changes. Therefore, when searching for corresponding points between two images, the corresponding points between the images can be appropriately obtained even if the images are rotated or the scales of the images are different.

  Hereinafter, a technique for calculating a local feature amount will be described using SIFT as an example. FIG. 24 is a flowchart of a local feature amount calculation process using SIFT. 25 to 28 are diagrams for explaining the calculation process of the station feature amount by SIFT. In the local feature amount calculation process, first, as shown in FIG. 25A, a feature point p and a neighboring region NR set around the feature point are extracted from the image TP (step S71). When there are a plurality of feature points p, the neighborhood region NR is set for each feature point p. In the extraction process of the feature point p and the neighboring region NR, the scale information of the image is output together with the position of the feature point p by the DoG filter. Scale invariance is realized by cutting out the neighboring region NR centered on the feature point p in accordance with the scale information.

  Next, the direction of the spindle is obtained (step S72). In the calculation processing of the main axis, the following processing is applied to each neighboring region NR set for each feature point p. First, as shown in FIGS. 25C and 25D, the differential values in the x direction and the y direction are calculated for each pixel in the neighboring region NR, and the edge intensity m (x, y) and the edge gradient direction are calculated. θ (x, y) is obtained. Here, the edge strength m (x, y) is weighted by a Gaussian window G (x, y, σ) (see FIG. 25B) centered on the feature point p, and the weighted edge strength mhat ( x, y). Thereby, it is considered that the central pixel in the neighboring region NR has important information.

  Next, an edge gradient direction histogram is generated. Specifically, 36 edge gradient directions of each pixel are quantized, and the weighted edge strength mhat (x, y) is voted in the corresponding direction. In this way, the edge gradient direction histogram as shown in FIG. 25E is obtained by quantizing the edge gradient direction and voting the weighted edge strength for all the pixels in the neighboring region NR. Next, the maximum value is detected from the gradient direction histogram, and a quadratic function is fitted using the left and right values in the direction having the maximum value, and the direction corresponding to the maximum is set as the direction v of the main axis.

  Next, the rotation of the texture pattern in the vicinity region is corrected (step S73). In this rotation correction, the texture pattern in the neighboring region NR is rotated so that the principal axis direction v obtained in step S72 matches the reference direction RD. FIG. 26 shows an example in which the reference direction is the horizontal right direction. The value of each pixel after rotating the neighboring region NR is obtained by linearly interpolating the surrounding pixels. A new neighboring region NR ′ is set for the rotated texture pattern.

  Next, a local feature amount is calculated (step S74). In order to calculate the local feature amount, first, the differential values in the x direction and the y direction are calculated again for each pixel in the neighborhood region NR ′ newly set with respect to the texture pattern subjected to the rotation correction. As shown in (a), the edge gradient direction and the edge strength are obtained. In FIG. 27, the arrow direction represents the edge gradient direction, and the arrow length represents the edge strength.

  Next, the edge gradient directions of each pixel are quantized into eight ways, and the neighborhood region is divided into 4 × 4 to define 16 voting cells VC as shown in FIG. In the example of FIG. 27B, one voting cell VC is formed by 2 × 2 pixels. Then, eight gradient direction histograms are obtained for each voting cell VC. From the above calculation, a feature vector of 16 × 8 = 128 dimensions is obtained. When the length of this feature vector is normalized to 1, a local feature amount is obtained.

  As described above, a local feature amount having rotation invariance and scale invariance can be obtained. Although the above is the local feature amount calculation processing by SIFT, SURF is also based on the same concept.

U.S. Pat. No. 6,711,293 "Improving bag-of-features for large scale image search", International Journal of computer vision, 2010 "Unsupervised 3D object recognition and reconstruction in unordered datasets," International Conference on 3-D Digital Imaging and Modeling (3DIM 2005) "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, 60, 2 (2004) Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool, "SURF: Speeded Up Robust Features", Computer Vision and Image Understanding (CVIU), Vol. 110, No. 3, pp. 346-359, 2008

  SIFT has a problem that the execution speed of local features is slow. SURF is a technique that improves the calculation speed of SIFT, but it still does not provide performance sufficient to operate with low-spec hardware such as a portable terminal. It can be considered that there are the following two factors for the slow execution speed.

  The first factor is that in the local feature amount calculation processing by SIFT or SURF, in order to obtain the principal axis direction v, differential calculation is performed for each pixel in the neighboring region NR, and the texture pattern is rotationally corrected based on the obtained principal axis direction v. Then, the differential calculation is performed again to calculate the local feature amount for each pixel in the new neighboring region NR ′ after the rotation correction. That is, the differential calculation for determining the main axis and the differential calculation for determining the local feature amount are performed independently, although they are similar processes.

  The second factor is a huge amount of calculation for rotationally correcting the texture pattern. Calculation of texture pattern rotation correction requires linear interpolation processing in units of sub-pixels and involves a huge amount of floating-point calculations, which increases the amount of calculation.

  Therefore, an object of the present invention is to provide a local feature amount calculating apparatus and method capable of calculating a local feature amount at high speed, and a corresponding point searching apparatus and method using them.

  The local feature amount calculation apparatus according to the present invention is a local feature amount calculation device that calculates a local feature amount of an image, and extracts a feature point and a neighborhood region set in the vicinity of the feature point from the image. For each pixel, a region extraction unit, a direction intensity calculation unit that calculates the direction of each pixel in the neighboring region and the intensity of that direction, a principal axis direction detection unit that detects the principal axis direction of the neighboring region, and the respective pixels The pixel calculated by the direction intensity calculation unit for the voting cell to which the pixel belongs among the voting cells of the rotated voting cell pattern obtained by rotating the voting cell pattern at an angle corresponding to the principal axis direction. In accordance with the corrected direction obtained by correcting the direction according to the main axis direction, voting the intensity of the direction of the pixel calculated by the direction intensity calculation unit, It has a configuration in which a local feature amount calculation unit for obtaining the local feature quantity of feature points.

  With this configuration, in order to achieve rotation invariance, the voting cell pattern is rotated instead of voting on the newly set voting cell pattern cell after rotating the texture pattern in the vicinity of the feature point. Thus, voting is performed on the voting cells of the rotated voting cell pattern (rotated voting cell pattern), so that sub-pixel linear interpolation processing with enormous floating point calculation for rotating the texture pattern is not necessary, The calculation of the feature amount can be speeded up. Note that the feature point neighboring region extraction unit may extract only pixels satisfying a predetermined condition among pixels constituting the image as feature points, or may use all pixels of the image as feature points. Further, the main axis direction detection unit may detect based on detection values of a gyro sensor, a geomagnetic sensor, a gravity sensor, and the like provided in the imaging device when generating an image.

  Further, in the above-described local feature amount calculation device, the principal axis direction detection unit is configured to determine the neighborhood region based on the direction of the plurality of pixels in the neighborhood region calculated by the direction strength calculation unit and the intensity in the direction. Detects the main shaft direction.

  As described above, the local feature amount calculation apparatus according to the present invention corrects the direction of each pixel by rotating the voting cell pattern instead of rotating the texture pattern in order to realize rotation invariance. Since voting is performed, it becomes unnecessary to calculate the direction of each pixel and the intensity of the direction again for the rotated texture pattern. The direction of each pixel and the intensity of the direction calculated by the direction intensity calculation unit are the main axes. It can be used for direction detection, and can also be used for calculation of local features. Thereby, the differential calculation for calculating the direction of each pixel and the intensity of the direction is sufficient, and the calculation of the local feature amount can be speeded up.

  The local feature amount calculation apparatus further includes a direction quantization unit that quantizes the direction of the pixel, and the principal axis direction detection unit follows the direction of the pixel quantized by the direction quantization unit. By voting the intensity of the direction of the pixel, a direction histogram is generated, the vicinity of the peak of the direction histogram is approximated by a curve, and the direction corresponding to the maximum of the curve is detected as the main axis direction.

  With this configuration, the intensity is accumulated for each quantized direction, and the direction with the largest accumulated intensity value can be detected as the principal axis direction. Further, since the peak of the histogram is approximated by a curve and the maximum is set as the main axis direction, a real main axis direction can be obtained. The principal axis direction may be expressed by a quantization number (integer) when the pixel direction is quantized, or may be expressed in radians.

  In addition, the local feature amount calculation device further includes a voting cell pattern generation unit that generates the rotated voting cell pattern by rotating the reference voting cell pattern by an angle corresponding to the main axis direction.

  According to this structure, the rotated voting cell pattern rotated corresponding to the main axis direction can be generated.

  The local feature amount calculation device further includes a direction correction pattern generation unit that generates a direction correction pattern for correcting the direction of the pixel calculated by the direction intensity calculation unit according to the principal axis direction. The local feature amount calculation unit obtains the corrected direction by correcting the direction of each pixel using the direction correction pattern.

  With this configuration, since the direction of the pixel is corrected corresponding to the main axis direction, an appropriate direction for voting on the rotated voting cell pattern can be acquired.

  The local feature amount calculation apparatus further includes a direction quantization unit that quantizes the direction of the pixel, and the principal axis direction detection unit follows the direction of the pixel quantized by the direction quantization unit. A direction histogram is generated by voting the intensity of the direction of the pixel, and a direction having a maximum value is detected as the main axis direction in the direction histogram.

  According to this configuration, the intensity in each direction is accumulated for each quantized direction, and the direction having the largest accumulated intensity value can be detected as the main axis direction. Also, since the direction having the maximum value in the histogram is the main axis direction, the discretized main axis direction can be obtained. The principal axis direction may be expressed by a quantization number (integer) when the pixel direction is quantized, or may be expressed in radians.

  Further, the local feature amount calculation device further includes a plurality of votes obtained by rotating the reference voting cell pattern at a plurality of angles respectively corresponding to a plurality of principal axis directions that can be detected by the principal axis direction detection unit. A voting cell pattern table in which cell patterns are stored, wherein the local feature amount calculation unit includes voting cell patterns corresponding to the main axis direction among the voting cell patterns stored in the voting cell pattern table as the rotated voting cells; The local feature amount is calculated using the pattern.

  As described above, when the spindle direction is obtained as a discretized value, a rotated voting cell pattern corresponding to each spindle direction can be prepared in advance and tabulated. With the above configuration, when a spindle direction is obtained using such a table, a voting cell pattern is obtained by calculation each time by selecting a rotated voting cell pattern corresponding to the spindle direction from the table. Since it is not necessary, the processing can be speeded up. Further, if the rotated voting cell pattern stored in the voting cell pattern is the same size as the neighboring area, each voting cell to be voted by each pixel is obtained only by one memory access (table lookup). be able to.

  Further, the local feature amount calculation device further includes a plurality of votes obtained by rotating the reference voting cell pattern at a plurality of angles respectively corresponding to a plurality of principal axis directions that can be detected by the principal axis direction detection unit. Among the cell patterns, a representative voting cell pattern table storing a plurality of representative voting cell patterns whose patterns do not overlap each other, and a cell number conversion table for converting the cell numbers of the representative voting cell patterns according to the main axis direction And the local feature amount calculation unit calculates a cell number of the representative voting cell pattern corresponding to the main axis direction among the representative voting cell patterns stored in the voting cell pattern table, and the cell number conversion table. Referring to the voting cell for voting the strength of the direction by converting according to the direction of the main axis.

  If the reference voting cell pattern has a point-symmetric shape, when it is rotated, rotated voting cell patterns having the same pattern shape appear periodically. The above configuration uses the periodicity to compress the voting cell pattern table. That is, for a plurality of rotated voting cell patterns having overlapping pattern shapes, only one of them is stored as a representative voting cell pattern in the voting cell pattern table, and the pattern shape is the same as the representative voting cell pattern. For the spindle directions having different numbers, the rotated cell pattern in the spindle direction can be obtained by converting the voting cell number of the representative voting cell pattern.

  Further, in the local feature amount calculation device, the corrected direction with respect to the direction quantized by the direction quantization unit is defined for each of a plurality of principal axis directions that can be detected by the main axis direction detection unit. The local feature amount calculation unit refers to the direction correction table, and the corrected direction corresponding to the direction quantized by the direction quantization unit according to the principal axis direction by referring to the direction correction table. To get.

  As for the pixel direction, a table can be prepared if the principal axis direction can be obtained discretely. With the above configuration, since the direction of each pixel is corrected using such a table, the corrected direction can be obtained for each pixel by only one memory access (table lookup). Speed can be increased.

  In the local feature amount calculation device, the corrected direction with respect to the direction quantized by the direction quantization unit among the plurality of main axis directions that can be detected by the main axis direction detection unit is the same. A representative direction correction table in which a representative corrected direction with respect to the direction quantized by the direction quantization unit is defined for each group, and the representative direction correction according to the main axis direction. A direction conversion table for converting the representative corrected direction of the table, and the local feature amount calculation unit include the main axis direction and the direction quantum of the representative corrected directions defined in the representative direction correction table. By converting a representative corrected direction corresponding to the direction quantized by the conversion unit according to the main axis direction with reference to the direction conversion table, the corrected To get the direction.

  Also in the direction correction table, when the corrected direction has periodicity, the table can be compressed. With the above configuration, when the main axis direction having the same period of the corrected direction is regarded as one group, only the representative corrected direction is stored for each group, and when the direction is actually corrected, the representative correction direction table Referring to FIG. 5, the corrected direction can be obtained by obtaining the representative correction direction according to the main axis direction and converting the obtained representative correction direction according to the main axis direction.

  Further, in the above-described local feature quantity calculation device, the local feature quantity calculation unit, for each pixel, when the pixel belongs to a plurality of voting cells, according to the proportion of each voting cell in the pixel, Vote the intensity in the direction of the pixel to the plurality of voting cells.

  In the voting cell pattern, it is possible to set the voting cell so that each pixel belongs to only one voting cell. However, in the vicinity of the voting cell boundary due to the difference in the rotation angle of the rotated voting cell pattern. An error will occur. With the above configuration, by making one pixel belong to a plurality of voting cells, an accurate voting value can be obtained regardless of the rotation angle of the rotated voting cell pattern.

  In the local feature amount calculation apparatus, the direction is a pixel edge gradient direction, and the intensity in the direction is a pixel edge intensity.

  With this configuration, the direction and its intensity can be obtained by differentiating the pixels.

  Further, in the above-described local feature quantity calculation device, the local feature quantity calculation unit weights the direction strength according to the size of the voting cell for voting the direction strength and votes.

  As described above, in the voting cell pattern, it is possible to set the voting cell so that each pixel belongs to only one voting cell, but the area of the same voting cell is rotated by the rotated voting cell pattern. Different things occur due to the difference in corners. According to the above configuration, the voting strength is weighted according to the size of the voting cell, so that the error of the cumulative value of voting to the same voting cell due to the difference in the rotation angle of the rotated voting cell pattern as described above can be obtained. Can be relaxed.

  In the local feature amount calculation device, the direction intensity calculation unit calculates the intensity in the direction by weighting a pixel closer to the feature point with a higher weight.

  With this configuration, a pixel closer to a feature point is regarded as a more important pixel in the vicinity region.

  Another aspect of the present invention is a corresponding point search device that searches for corresponding points between a plurality of images, using the local feature amount calculation device described above and the local feature amount calculation device. And a corresponding point search unit that searches for corresponding points between the plurality of images by comparing the calculated local feature amounts of the plurality of images with each other.

  With this configuration, the local feature amount can be calculated at high speed in order to search for corresponding points between a plurality of images.

  Still another aspect of the present invention is a local feature amount calculation method for calculating a local feature amount of an image, the feature point extracting step for extracting a feature point from the image, and setting a neighborhood region in the vicinity of the feature point A neighboring area setting step, a direction calculating step for calculating the direction of each pixel in the neighboring area, an intensity calculating step for calculating the intensity in the direction of each pixel in the neighboring area, and a plurality of areas in the neighboring area Based on the direction of the pixel and the intensity of the direction, a main axis direction detecting step for detecting the main axis direction of the neighboring region, and rotating the reference voting cell pattern by an angle according to the main axis direction A voting cell pattern rotation step to be acquired, a direction correction step of correcting the direction of each pixel according to the principal axis direction to acquire a corrected direction, and the vicinity For each pixel in the area, voting the intensity of the pixel in the direction according to the corrected direction with respect to the voting cell to which the pixel belongs among the voting cells of the rotated voting cell pattern. And a local feature amount calculating step for obtaining the local feature amount.

  Even with this configuration, in order to realize rotation invariance, the voting cell pattern is rotated instead of voting on the newly set voting cell pattern cell after rotating the texture pattern in the vicinity of the feature point. Then, since voting is performed on the voting cells of the rotated voting cell pattern (rotated voting cell pattern), linear interpolation processing in units of subpixels with a huge floating point calculation for rotating the texture pattern becomes unnecessary. The local feature amount can be calculated at high speed. Note that the above steps need not be executed in the order described. Some of the above steps may be performed in parallel, or some of the above steps may be performed in a different order than described.

  Still another aspect of the present invention is a local feature amount calculation method for calculating a local feature amount of an image, the feature point extracting step for extracting a feature point from the image, and setting a neighborhood region in the vicinity of the feature point A neighboring region setting step, a direction calculating step for calculating a direction of each pixel in the neighboring region, a direction quantization step for quantizing the direction of each pixel in the neighboring region, and each of the neighboring region in the neighboring region A direction histogram is generated by voting the intensity of the direction of the pixel according to the direction of the pixel quantized in the intensity calculation step for calculating the intensity of the pixel direction and the direction quantization step, and the direction histogram A main axis direction detecting step for detecting a direction having a maximum value as a main axis direction of the neighboring area, and for each pixel in the neighboring area, the main axis Referring to a voting cell pattern table storing a plurality of voting cell patterns obtained by rotating a reference voting cell pattern at a plurality of angles respectively corresponding to a plurality of main axis directions that can be detected in the direction detecting step, the main axis A voting cell determination step for determining a voting cell to which the pixel belongs in a voting cell pattern corresponding to a direction, and for each of a plurality of principal axis directions that can be detected in the principal axis direction detection step for each pixel in the neighboring region. , Corrected with respect to the direction of the principal axis and the direction quantized in the direction quantization step with reference to the direction correction table in which the corrected direction with respect to the direction quantized in the direction quantization step is defined A direction correction step for obtaining a direction, and a voting cell determination step for each pixel in the neighboring region. A local feature amount calculating step for determining a local feature amount of the feature point by voting the intensity of the pixel in the direction according to the corrected direction. is doing.

  Even with this configuration, in order to realize rotation invariance, the voting cell pattern is rotated instead of voting on the newly set voting cell pattern cell after rotating the texture pattern in the vicinity of the feature point. Then, since voting is performed on the voting cells of the rotated voting cell pattern (rotated voting cell pattern), linear interpolation processing in units of subpixels with a huge floating point calculation for rotating the texture pattern becomes unnecessary. The local feature amount can be calculated at high speed. In addition, since the main axis direction is obtained as a discrete value, the rotated voting cell pattern can be obtained by using a table, and the direction of each pixel can also be corrected by using the table. It can be faster. The voting cell pattern table and the direction correction table may be a combination of a compressed table and a conversion table. Further, the above steps need not be executed in the order described. Some of the above steps may be performed in parallel, or some of the above steps may be performed in a different order than described.

  Still another aspect of the present invention is a corresponding point search method for searching for corresponding points between a plurality of images, wherein the local feature amounts of the plurality of images are calculated by the local feature amount calculation method described above. A feature amount calculating step, and a corresponding point searching step of searching for corresponding points between the plurality of images by comparing each local feature amount of the plurality of images calculated in the local feature amount calculating step. It has the composition which includes.

  Even with this configuration, in order to search for corresponding points between a plurality of images, the local feature amount can be calculated at high speed. Note that the above steps need not be executed in the order described. Some of the above steps may be performed in parallel, or some of the above steps may be performed in a different order than described.

  Yet another embodiment of the present invention is a program for causing an arithmetic processing device to execute any one of the above methods.

  According to the present invention, in order to realize rotation invariance, instead of voting on a cell of the newly set voting cell pattern after rotating the texture pattern in the vicinity of the feature point, the voting cell pattern Rotating and voting on the voting cell of the rotated voting cell pattern eliminates the need for sub-pixel linear interpolation with enormous floating-point calculations to rotate the texture pattern, and calculates local features Speed can be increased.

The block diagram which shows the structure of the local feature-value calculation apparatus of the 1st Embodiment of this invention. The figure which shows the feature point and vicinity area | region in the input image of the 1st Embodiment of this invention The figure which shows the pyramid image which reduced the input image of the 1st Embodiment of this invention in multiple steps The graph explaining the quantization of the edge gradient direction according to the first embodiment of the present invention The graph which shows the histogram of the edge gradient direction of the 1st Embodiment of this invention The figure which shows the voting cell pattern (0 degree of main axis directions) of the 1st Embodiment of this invention The figure which shows the voting cell pattern (main axis direction 22.5 degree | times) of the 1st Embodiment of this invention The figure which shows a mode that the voting cell pattern of the 1st Embodiment of this invention was superimposed on the input image The figure for demonstrating the vector which shows the local feature-value of the 1st Embodiment of this invention The flowchart of the local feature-value calculation method of the 1st Embodiment of this invention The block diagram which shows the structure of the local feature-value calculation apparatus of the 2nd Embodiment of this invention. The figure which shows each voting cell pattern corresponding to each direction of the main axis | shaft of the 2nd Embodiment of this invention. The figure which shows the voting cell pattern table of the 2nd Embodiment of this invention. The figure which shows the compressed voting cell pattern table of the 2nd Embodiment of this invention The figure which shows the conversion table for compression for determining the voting cell pattern of the 2nd Embodiment of this invention The figure which shows the direction correction table of the 2nd Embodiment of this invention. The figure which shows the compressed direction correction table of the 2nd Embodiment of this invention. The figure which shows the conversion table for compression for performing the direction correction of the 2nd Embodiment of this invention. Flow chart of the local feature amount calculation method of the second embodiment of the present invention The figure explaining the process which votes for several voting cells about one pixel of the 2nd Embodiment of this invention The block diagram which shows the structure of the corresponding point search apparatus of the 3rd Embodiment of this invention. The flowchart of the corresponding point search method of the 3rd Embodiment of this invention Diagram showing corresponding points between two images Diagram for explaining image search using local features Flow chart of calculation process of station feature value by SIFT The figure for demonstrating the calculation process of the station feature-value by SIFT The figure for demonstrating the calculation process of the station feature-value by SIFT The figure for demonstrating the calculation process of the station feature-value by SIFT

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiments described below are merely examples and do not limit the invention.

(First embodiment)
FIG. 1 is a block diagram showing a configuration of a local feature quantity calculation device 1 according to the first embodiment of this invention. The local feature amount calculation device 1 includes a feature point neighborhood region extraction unit 11, a direction strength calculation unit 12, a spindle direction detection unit 13, a voting cell pattern generation unit 14, a direction correction pattern generation unit 15, and a local feature amount. And a calculation unit 16. The local feature quantity calculation device 1 may implement the configuration shown in FIG. 1 as hardware, or may implement the configuration shown in FIG. 1 by installing software (computer program) in the arithmetic processing device. . The local feature quantity calculation device 1 inputs an image and outputs a local feature quantity in the image.

  An image TP is input to the feature point neighboring region extraction unit 11. The feature point neighboring region extraction unit 11 extracts a pixel satisfying a predetermined condition from the pixels constituting the target image as the feature point p. The conditions for extracting feature points are determined so that extraction with high reproducibility can be performed. For example, feature points can be extracted using an algorithm (corner detector) that extracts corner points (corners).

  In this way, by extracting only some of the plurality of pixels constituting the input image as feature points, local feature amounts can be obtained for only some of the pixels in the input image. In the corresponding point search using the quantity, the corresponding point can be searched at high speed. On the other hand, all pixels of the input image may be extracted as feature points. In this case, it can be expected that close correspondence between images can be obtained.

The feature point neighborhood region extraction unit 11 sets a predetermined region near the extracted feature point p as the neighborhood region NR and extracts it. In the present embodiment, a square region of W × W pixels centering on the feature point is set as the neighborhood region NR. 2, the feature point neighboring region extraction unit 11, the extracted region in the vicinity of the feature point _p 1 ~p ₆ and the feature point _{p 1} is a diagram showing an NR _1. Note that the shape of the neighborhood region is not limited to a square, and may be another shape such as a circle or a rectangle.

  As shown in FIG. 3, the feature point neighboring region extraction unit 11 obtains a pyramid image obtained by reducing the image TP in multiple stages, and extracts feature points independently from each other under the same conditions for each scale image. A neighborhood region may be defined. FIG. 3 shows a pyramid image reduced in multiple stages with a scale of 1 / s. By doing so, it is possible to obtain a local feature amount that has scale invariance and is not affected by the difference in scale. Moreover, the feature point vicinity area | region extraction part 11 may acquire scale information with a feature point with a DoG filter, and may cut out a vicinity area according to this scale information. This also makes it possible to obtain local feature quantities having scale invariance.

As described above, the feature point neighboring region extraction unit 11 generally extracts a plurality of feature points p _{1 to} p _N from one input image, and sets the neighborhood region NR for each feature point. And the local feature-value calculation apparatus 1 calculates a local feature-value about each feature point, respectively. Hereinafter, the direction intensity calculation unit 12, the spindle direction detection unit 13, the voting cell pattern generation unit 14, the direction correction pattern generation unit 15, and the local feature quantity calculation unit 16 will be described. When the feature points p _{1 to} p _N are extracted, the respective units 12 to 16 perform the processing described below for each of the plurality of feature points p _{1 to} p _N. In the following description, a process for obtaining a local feature amount of one feature point will be described.

The direction intensity calculator 12 calculates the direction of each pixel in the neighborhood region NR and the intensity in that direction. Here, in the present invention, the “direction” of a pixel represents a bias in the distribution of pixel values in a small region including the periphery of the pixel. The “direction strength” represents the degree of bias, that is, the reliability of the “direction”. The “direction” and the “direction intensity” can typically be obtained based on the edge information obtained by differentiating the pixel value of the pixel to obtain the edge information of the pixel. That is, by differentiating the pixel in the x direction and the y direction, the edge gradient direction and the edge strength can be obtained as edge information. The edge gradient direction is set as the “direction” of the pixel, and the edge strength is set to the pixel. It can be “direction strength”. In the present embodiment, the edge gradient direction and the edge strength are adopted as the “direction” and “direction strength” of the pixel, respectively. Specifically, it is as follows. Hereinafter, (x, y) ^T represents coordinates on the image.

First, the x-direction derivative f _x (x, y) and the y-direction derivative f _y (x, y) of the pixel at the coordinates (x, y) are obtained by Expression (1) and Expression (2), respectively.

Then, using these x-direction derivative f _x (x, y) and y-direction derivative f _y (x, y), the edge gradient direction θ (x, y) and the edge intensity m (x, y) are respectively determined. It calculates | requires by Formula (3) and Formula (4).
However, according to Equation (3), the edge gradient direction cannot distinguish the first quadrant and the third quadrant, and the second quadrant and the fourth quadrant. Thus, with reference to the sign of the x-direction differential _f x (x, y) and y directional derivative _f y (x, y), by appropriately corrected, obtaining the direction of 0~2π as the edge gradient direction. Hereinafter, in order to simplify the description, all arctangents output values of 0 to 2π.

The direction strength calculation unit 12 obtains the weighted edge strength mhat (x, y) by the equation (5) using the edge strength obtained by the equation (4). Note that p = (p _x , p _y ) ^T is the position of the feature point. “Hat” represents “^” in the formula.
Here, the function w (x, y) is a weight, for example, the average is (0, 0) ^T ,
A two-dimensional Gaussian distribution can be used. This means that the center pixel is considered as important.

  The weighted edge strength mhat (x, y) obtained by the direction strength calculation unit 12 is used by the main axis direction detection unit 13 and the local feature amount calculation unit 16 in the subsequent stage, respectively. The weighted edge strength for detecting the principal axis direction and the weighted edge strength for calculating the local feature amount may be separately obtained by using different weights w (x, y) for the amount calculation. Of course, it is also possible to detect the principal axis direction using the unweighted edge strength m (x, y) as it is to calculate the local feature amount.

  The direction intensity calculation unit 12 calculates the edge gradient direction θ (x, y) and the weighted edge intensity mhat (x, y) for each pixel in the neighboring region NR as described above, and uses them to calculate the principal axis direction detection unit 13. And output to the local feature amount calculation unit 16.

The main shaft direction detector 13 detects the main shaft direction. For this purpose, the principal axis direction detection unit 13 first quantizes the edge gradient direction θ (x, y) according to equation (6).
Here, the right side of Expression (6) is a function that performs scalar quantization of the edge gradient direction θ (x, y) from 0 to N−1. For example, Expression (7) can be employed.
If this equation (7) is represented in a graph for N = 4, for example, it is as shown in FIG. As a result of experiments, it has been found that N = 40 is preferable.

When the principal axis direction detection unit 13 quantizes the edge gradient direction θ (x, y) of each pixel in the neighboring region NR, the quantized edge gradient direction θhat (x, y) and the direction intensity calculation unit 12 Using the weighted edge strength mhat (x, y) obtained from the above, the histogram h (k) in the edge gradient direction is obtained according to the equation (8).
In Equation (8), S _k is given by Equation (9).
As shown in Expression (8), the weighted edge strength hat (x, y) is used as the vote value of the edge gradient direction histogram. FIG. 5 is a graph showing an example of the edge gradient direction histogram obtained by Expression (8).

The main axis direction detection unit 13 obtains the main axis from the edge gradient direction histogram h (k). In the present embodiment, the main axis direction detection unit 13 approximates the vicinity of the peak of the histogram h (k) as a parabola, and sets the parabola peak in radians as the main axis v. The main axis v is a real number, and is therefore expressed as v _real . As will be described later, in another embodiment, the peak of the histogram is directly used as the main axis v. In this case, since the main axis v is a discrete value, that is, an integer value of 0 to N−1 or a value obtained by converting it into radians, this is expressed as v _integer and distinguished from v _real .

The main axis direction detector 13 may detect a plurality of main axis directions when there are a plurality of peaks in the edge gradient direction histogram h (k). In this case, local feature amounts are given by the number in the main axis direction, and search performance is improved in the corresponding point search using the local feature amounts. The main axis direction detection unit 13 outputs the detected main axis direction v _real to the voting cell pattern generation unit 14 and the direction correction pattern generation unit 15.

The voting cell pattern generation unit 14 rotates the reference cell pattern CP at an angle corresponding to the principal axis direction _vreal . In other words, the voting cell pattern generation unit 14 rotates the voting cell pattern CP so that the reference direction ED of the reference voting cell pattern CP matches the direction of the principal axis direction _vreal . The rotated voting cell pattern is called a rotated voting cell pattern. As the voting cell pattern, as in the case of the SIFT described in the background art above, a pattern composed of voting cells divided on a 4 × 4 grid can be adopted. As shown in FIGS. 6A and 6B, the voting cell is formed by dividing the circle into three regions along the circumference, and further dividing the two donut-shaped regions so divided into eight equal parts in the radial direction. The voting cell pattern formed is used. Each voting cell VC of the voting cell pattern CP is given a voting cell number in order.

  The reference voting cell pattern is set so that the center thereof is the position of the feature point, and the rotated voting cell pattern is obtained by rotating the reference voting cell pattern with the center as the viewpoint. A voting cell number “0” is given to the center cell, and voting cell numbers “1” to “8” are sequentially counterclockwise from the voting cell on the right side of the reference voting cell pattern in the outer donut-shaped region. In the outermost donut-shaped region, voting cell numbers “9” to “16” are given in order counterclockwise from the voting cell on the right side of the reference voting cell pattern. In other words, the voting cell numbers in the outer donut-shaped region are “1” to “8” in order from the right voting cell in the counterclockwise direction, and the voting cell number is the right side of the reference voting cell pattern in the outermost donut-shaped region. The voting cell patterns that are “9” to “16” in order from the voting cells in the counterclockwise direction are the reference voting cell patterns of the present embodiment, and the rotated voting cell patterns are the rotated ones. The reference direction RD of the reference voting cell pattern is a direction passing through the centers of the voting cell 0, the voting cell 1, and the voting cell 9.

FIG. 6A shows a rotated voting cell pattern CP ′ (ie, not rotated from the reference voting cell pattern) in which the reference direction RD overlaps the main axis v _real when the main axis direction v _real is the horizontal right direction. FIG. 6B shows a rotated voting cell pattern CP ′ that is rotated so that the reference direction RD coincides with the main axis direction v _real when the main axis direction v _real is 22.5 degrees.

FIG. 7 is a diagram showing a state where the rotated voting cell pattern CP ′ is overlaid on the image TP. In the example of FIG. 7, the main axis direction v _real is 20 degrees, and the rotated voting cell pattern CP ′ is rotated so that the reference direction RD coincides with the direction of the main axis v _real . When voting for each pixel, the voting cell to which the pixel belongs is voted.

When the principal axis direction v _{real is} radian representation rotated rotated already voted cell patterns CP to the reference direction RD of the reference voting cell pattern CP coincides with the direction of the main axis v _real 'has the formula (10) It is represented by That is, the vote cell pattern generation unit 14, by substituting the principal axis _{v real} in equation (10), to produce a by voting cell pattern CP 'rotates in accordance spindle _{v real.} Note that (p _x , p _y ) ^T is the position of the feature point p.

By substituting the coordinates (x, y) of the pixel into this equation (10), the voting cell that the pixel should vote is determined. In Equation (10), r ₁ , r ₂ , and r ₃ are respectively the radius of the voting cell 0 and the radius of the outer periphery of the voting cells 1 to 8 (voting cells) in the voting cell patterns shown in FIGS. 6A and 6B. 9-16, the radius of the outer periphery of the voting cells 9-16. For example, when the size of the neighborhood region NR is 25 × 25, r ₁ = 3, r ₂ = 8, r ₃ = 12, etc. may be selected. The voting cell pattern generation unit 14 outputs the generated rotated voting cell pattern CP ′ to the local feature amount calculation unit 16.

The direction correction pattern generation unit 15 generates a direction correction pattern for correcting the edge gradient direction θ (x, y) according to the main axis direction _vreal . The direction correction pattern is expressed by Expression (11).
As is clear from Equation (11), the direction correction pattern is a correction pattern for correcting the edge gradient direction θ (x, y) so as to cancel the inclination of the main axis direction v _{real from} the reference direction RD. The direction correction pattern generation unit 15 outputs the generated direction correction pattern to the local feature amount calculation unit 16.

  The local feature quantity calculation unit 16 includes an edge gradient direction θ (x, y) and weighted edge strength mhat (x, y) of each pixel in the neighboring area calculated by the direction strength calculation unit 12, and a voting cell pattern generation unit. The rotated voting cell pattern CP ′ generated in 14 and the direction correction pattern generated in the direction correction pattern generation unit 15 are input, and the local feature amount d is obtained using these. Specifically, the local feature amount calculation unit 16 corrects the edge gradient direction θ (x, y) of each pixel in the neighboring region NR with the direction correction pattern to obtain a corrected edge gradient direction. Then, according to the corrected edge gradient direction, the local feature amount d is obtained by voting the weighted edge strength mhat (x, y) on the rotated voting cell pattern CP ′.

  When voting, the local feature quantity calculation unit 60 first corrects the edge gradient direction θ (x, y) of each pixel with a direction correction pattern to obtain a corrected edge gradient direction, and quantizes it in eight ways. . As shown in FIGS. 6A and 6B, the rotated voting cell pattern CP ′ of the present embodiment has a total of 17 voting cells. Therefore, if a histogram in eight directions is generated in each voting cell and connected to form a local feature amount d, the local feature amount is 17 × 8 = 136 dimensions. Note that the weighted edge strength mhat (x, y) of each pixel is voted in the generation of the histogram.

If the local feature amount is described by a vector notation d = (d ₀ , d ₁ ,..., D ₁₃₅ ) ^T , d _i is obtained by Expression (12).
This S _i is given by equation (13).
Further, the index (x, y) of the equation (13) is obtained by the equation (14).

In addition, voting_cell_pattern _value (x, y) in Expression (14) is represented by Expression (10). Further, direction_offset _real (θ (x, y)) in equation (14) is shown in equation (11), and equation (7) is adopted as function quantize (θ, 8).

According to the equation (14), d ₀ , d ₁ ,..., D ₇ are voted to the voting cell 0 in the local feature amount d = (d ₀ , d ₁ ,..., D ₁₃₅ ). The voting cumulative values in the eight directions are d ₈ , d ₉ ,..., D ₁₅ are the voting cumulative values in the eight directions voted to the voting cell 1, and d ₁₆ , d _17. , · · ·, d ₂₃ are each voting cumulative value of the eight directions found in the voting cell 2, and so forth. This will be described with reference to FIG.

  Next, a local feature quantity calculation method executed by the local feature quantity calculation apparatus 1 will be described. The local feature quantity calculation method may be executed by the local feature quantity calculation apparatus 1 configured by hardware, or software (computer program) that is implemented in the arithmetic processing device and configures the local feature quantity calculation apparatus 1. ) May be executed. FIG. 9 is a flowchart of the local feature amount calculation method of the present embodiment. The local feature amount calculation method according to the present embodiment calculates a local feature amount from an input image.

  First, the feature point neighborhood region extraction unit 11 extracts a feature point p from the image TP (step S11). Next, the feature point neighborhood region extraction unit 11 sets a neighborhood region NR in the vicinity of the feature point p (step S12). Then, the feature point neighborhood region extraction unit 11 outputs the feature point p and the neighborhood region NR to the direction strength calculation unit 12. The direction intensity calculation unit 12 calculates the edge gradient direction θ (x, y) for each pixel in the neighboring region NR by differentiating each pixel in the neighboring region NR in the x direction and the y direction (step) S13) The weighted edge strength mhat (x, y) is calculated (step S14). Note that either step S13 or step S14 may be performed first or in parallel.

Next, the principal axis direction detection unit 13 quantizes the edge gradient direction θ (x, y) (step S15). Then, the principal axis direction detection unit 13 votes the edge gradient direction histogram by voting the weighted edge strength mhat (x, y) to the quantized edge gradient direction θhat (x, y) of the pixels in the neighboring region NR. Is generated (step S16). Main axis direction detecting unit 13 performs curve fitting near the peak of the gradient direction histogram to detect the direction (real number) corresponding to the maximum of the curve as a main axis direction v _real proximity area NR (step S17).

The voting cell pattern generation unit 14 acquires the main axis direction v _real from the main axis direction detection unit 13, and rotates the reference voting cell pattern PC at an angle corresponding to the main axis direction v _real to generate a rotated voting cell pattern PC. (Step S18). Further, the direction correction pattern generation unit 15 generates a direction correction pattern corresponding to the main axis direction _vreal (step S19). Note that either step S18 or step S19 may be performed first or in parallel.

  The local feature amount calculation unit 16 receives the edge gradient direction θhat (x, y) and the weighted edge strength mhat (x, y) from the direction strength calculation unit 12, and the rotated voting cell from the voting cell pattern generation unit 14. The pattern PC ′ is input, and the direction correction pattern is input from the direction correction pattern generation unit 15. The local feature amount calculation unit 16 corrects the edge gradient direction θhat (x, y) according to the direction correction pattern for each pixel in the neighboring region NR, and acquires a corrected edge gradient direction (step S20). For each pixel in the neighboring region NR, the local feature amount calculation unit 16 determines the pixel according to the corrected edge gradient direction with respect to the voting cell to which the pixel belongs among the voting cells of the rotated voting cell pattern PC ′. The weighted edge strength mhat (x, y) is voted (step S21). Thereby, the local feature amount d of the feature point p is obtained.

  As described above, in this embodiment, in order to realize rotation invariance, a voting cell pattern newly set after rotating the texture pattern in the vicinity region NR of the feature point p as in the prior art. In order to rotate the texture pattern, the reference voting cell pattern CP is rotated and the voting cell of the rotated voting cell pattern (rotated voting cell pattern CP ′) is voted. This eliminates the need for linear interpolation processing in units of subpixels accompanied by a huge amount of floating point calculations, and can speed up the calculation of local features.

In addition, as described above, in order to realize the rotation invariance, instead of rotating the texture pattern, the reference voting cell pattern is rotated and the voting is performed after correcting the direction of each pixel. It is not necessary to calculate the direction of each pixel and the intensity of the direction again for the rotated texture pattern as in the prior art, and the edge gradient direction and edge intensity of each pixel calculated by the direction intensity calculation unit 12 are It can be used for detecting the principal axis direction _vreal and for calculating the local feature amount. Thereby, the differential calculation for calculating the direction of the edge gradient and the edge strength of each pixel is only required once, and the calculation of the local feature amount can be speeded up.

(Second Embodiment)
FIG. 10 is a block diagram showing a configuration of a local feature quantity calculating apparatus according to the second embodiment of this invention. The local feature quantity calculation device 2 of the present embodiment includes a feature point vicinity region extraction unit 21, a direction strength calculation unit 22, a main axis direction detection unit 23, and a local feature quantity calculation unit 26. The local feature quantity calculation device 2 may implement the configuration shown in FIG. 10 as hardware, or may implement the configuration shown in FIG. 10 by installing software in the arithmetic processing device.

  The feature point neighborhood region extraction unit 21 extracts feature points p from the input image TP in the same manner as the feature point neighborhood region extraction unit 11 of the first embodiment, and the neighborhood region for each feature point p. Set NR. Similarly to the direction intensity calculation unit 12 of the first embodiment, the direction intensity calculation unit 22 differentiates each pixel in the neighboring region NR in the x direction and the y direction, thereby obtaining “direction” and “direction direction”. As the “strength”, the edge gradient direction θ (x, y) and the edge strength m (x, y) are calculated.

  In the first embodiment, the principal axis direction detection unit 13 quantizes the edge gradient direction θ (x, y) of the pixels in the neighboring region NR obtained by the direction intensity calculation unit 22 using Equation (6). However, in this embodiment, the direction intensity calculation unit 22 performs this quantization, and the direction intensity calculation unit 22 uses the quantized edge gradient direction θhat (x, y) as the main axis direction detection unit 23 and the local feature. It outputs to the quantity calculation part 26. For the edge strength, the direction strength calculation unit 22 obtains the weighted edge strength mhat (x, y) as in the direction strength calculation unit 12 of the first embodiment, and the main axis direction detection unit 23 and the local feature amount. It outputs to the calculation part 26.

The principal axis direction detection unit 23 obtains the quantized edge gradient direction θhat (x, y) and the weighted edge strength mhat (x, y) from the direction intensity calculation unit 22, and uses the equations (8) and (9). A histogram h (k) in the edge gradient direction is obtained. Then, the peak of the histogram h (k) is set as the main axis direction v _integer (see FIG. 5). The main axis direction v _integer is an integer value of 0 to N−1, that is, a discrete value. The main shaft direction detection unit 30 may convert this integer value into radians. Also in this case, radians as the main axis v _integer take discrete values. The main axis direction detection unit 23 outputs the main axis v _integer that is a discrete value to the local feature amount calculation unit 26.

  Compared with the local feature quantity calculation device 1 of the first embodiment, the local feature quantity calculation device 2 of the present embodiment has a voting cell pattern generation unit 14 and a direction correction pattern generation unit 15 of the first embodiment. Not equipped. In this embodiment, since the main shaft direction is detected as a discrete value by the main shaft direction detector 23, the rotated voting cell pattern and the direction correction pattern can be tabulated. This is because it is not necessary to generate (calculate) a rotated voting cell pattern or a direction correction pattern each time based on the detected main axis direction as in the embodiment. The local feature amount calculation unit 26 of the present embodiment includes a voting cell determination unit 24 and a direction correction unit 25 therein.

First, the voting cell determination unit 24 will be described. The voting cell determination unit 24 determines a voting cell pattern according to the main axis v _integer obtained from the main axis direction detection unit 23, refers to the determined voting cell pattern, and acquires a voting cell number corresponding to the position of the target pixel.

FIG. 11 shows a reference voting cell pattern PC and a rotated voting cell pattern PC ′ corresponding to each main axis direction v _integer . The voting cell pattern has the same shape as the voting cell pattern shown in FIGS. 6A and 6B, and there are 17 voting cells from 0 to 16. The method of assigning cell numbers is the same as in FIGS. 6A and 6B.

As described above, the main axis direction v _integer is discretized from 0 to N-1, and the types thereof are finite (0 to N-1). Therefore, the types of rotated vote cell patterns corresponding thereto are also finite ( 0 to N-1). Therefore, if these finite rotated voting cell patterns are tabulated, when the main axis direction v _integer is given, the voting cell pattern is read from the table and rotated at an angle corresponding to the main axis direction v _integer. The rotated voting cell pattern thus obtained can be acquired.

  FIG. 12 is a diagram illustrating an example of a voting cell pattern table. FIG. 12 shows a case where N = 40 and W = 25. Each rotated voting cell pattern PC ′ has the same size as the size W × W (= 25 × 25) of the neighboring region NR. That is, the number of the voting cell to which the pixel belongs is given to each pixel in W × W. The cell number “−1” is given outside the circle, and no voting is performed. In the example of FIG. 12, one voting cell is assigned to one pixel.

By storing the voting cell pattern table shown in FIG. 12 in the voting cell determining unit 24, when the main axis direction v _integer is given from the main axis direction detecting unit 23, the voting cell determining unit 24 _{selects the} main axis direction v _integer. The rotated voting cell pattern corresponding to is selected, and the voting cell is determined according to the coordinates of each pixel.

  When the reference voting cell pattern has a point-symmetric shape, when it is rotated with a predetermined step width, rotated voting cell patterns having the same pattern shape appear periodically. Therefore, in this case, the voting cell pattern table can be compressed by using the cell number conversion table.

In this embodiment, the voting cell pattern is a dot object, and the number of steps is N = 40. Therefore, the rotated voting cell pattern having the same pattern shape is periodically every 45 degrees (every 5 steps). Appear. Therefore, the voting cell pattern table can be compressed. That is, the voting cell pattern with the main axis v _integer = 5 to 39 can be expressed by using the voting cell pattern with the main axis v _integer = 0 to 4, and the voting cell pattern table can be compressed by 1/8. For example, the voting cell pattern of the main axis v _integer = 5 to 9 can be expressed by the formula (15) using the voting cell pattern of the main axis v _integer = 0 to 4.

Therefore, in the present embodiment, the voting cell determining unit 24 is actually provided with a representative voting cell pattern table 241 as shown in FIG. 10, and as shown in FIG. 13, the main axis v _{integer of} FIG. Only voting cell patterns of 0 to 4 are stored as representative voting cell patterns. In other words, the representative cell pattern table 241 stores only a plurality of representative voting cell patterns whose patterns do not overlap each other. In the present embodiment, five types of rotated voting cell patterns corresponding to the main axis direction V _integer = 0, 1, 2, 3, 4 are representative voting cell patterns. The voting cell determination unit 24 further stores a cell number conversion table (compress _A ) 242 shown in FIG.

When the voting cell determining unit 24 is given the main axis direction v _integer , first, the voting cell determining unit 24 refers to the representative voting cell pattern table 241 in FIG. 13 and corresponds to the main axis direction v _integer of the five types of representative voting cell patterns. A representative voting cell pattern is selected. Then, in the representative voting cell pattern, the cell number of the voting cell to which the coordinate of the pixel of interest belongs is obtained. Next, the cell number of the obtained voting cell is converted with reference to the compression conversion table 242 of FIG.

For example, in the vicinity region where the main axis direction v _integer = 18, for the pixel of interest whose coordinates around the feature point are (−4, 2), the main axis direction v _integer is “18”, so FIG. Since the fourth representative voting cell pattern from the left is selected and the coordinates of the pixel of interest are (−4, 2), the cell number “4” is obtained. This cell number “4” is converted to “1” by referring to the fourth row and fifth column of the cell number conversion table 242 shown in FIG. Thereby, the voting cell of this pixel of interest is determined to be voting cell 1.

  Furthermore, if the first quadrant, the second quadrant, the third quadrant, and the fourth quadrant of the voting cell pattern are similar, paying attention to this, the voting cell pattern table shown in FIG. 1 can be compressed.

Next, the direction correction unit 25 will be described. The direction correction unit 25 obtains the quantized edge gradient direction θhat (x, y) for each pixel in the neighboring region obtained from the direction intensity calculation unit 22 and the principal axis direction v obtained from the principal axis direction detection unit 23. The edge gradient direction is corrected using an _integer .

As described above, in the present embodiment, the main axis direction v _integer is discretized from 0 to N−1, the type is finite (0 to N−1), and the edge gradient direction θhat (x , Y) is also a discrete value, so that the corrected edge gradient direction corresponding to the combination of the principal axis direction v _integer and the edge gradient direction θhat (x, y) can be tabulated. By preparing such a table, when the spindle direction v _integer is given, the edge gradient direction is read from the table, so that the corrected edge gradient direction corrected according to the spindle direction v _integer is obtained. You can get it.

FIG. 15 is a diagram illustrating an example of the direction correction table. FIG. 15 shows a case where N = 40. By storing the direction correction table shown in FIG. 15 in the direction correction unit 25, the direction correction unit 25 is given the principal axis direction v _integer and the quantized edge gradient direction θhat (x, y). By referring to the direction correction table in FIG. 15, any one of the eight directions 0 to 7 can be obtained as the corrected edge gradient direction.

As can be seen from FIG. 15, in the direction correction table, the change in the corrected edge gradient direction with respect to the edge gradient direction θhat (x, y) has periodicity. Therefore, the table can be compressed using this periodicity. In this embodiment, since N = 40, the direction correction table of FIG. 15 can be compressed to 1/8 by using the compression conversion table. That is, the edge gradient direction (0 to 7) in the case of the main axis v _integer = 5 to 40 can be obtained from the table of the portion of the main axis v _integer = 0 to 4. For example, when the main axis v _integer = 5 to 9, it can be expressed by Expression (16) using the direction correction table of the main axis v _integer = 0 to 4.

Therefore, in the present embodiment, the direction correction unit 25 is actually provided with a representative direction correction table 251 as shown in FIG. 10, and in this representative direction correction table 251, as shown in FIG. Only the portion of v _integer = 0 to 4 in the table of FIG. 15 is defined as the representative corrected edge gradient direction. In other words, in the representative direction correction table 251, the corrected edge gradient direction with respect to the edge gradient direction θhat (x, y) has the same period (for example, main axis direction v _integer = 0, 5, 10, 15, 20, 25, 30, 35), one group is defined, and only one row of representative corrected edge gradient directions is defined for each group. In the present embodiment, the corrected edge gradient direction (first to fifth rows in FIG. 15) corresponding to the principal axis direction v _integer = 0, 1, 2, 3, 4 is the representative corrected edge gradient direction as the representative direction. It is defined in the correction table 251. The direction correction unit 25 further stores a direction conversion table (compress _B ) 252 shown in FIG.

When the main axis v _integer and the quantized edge gradient direction θhat (x, y) are given, the direction correction unit 25 refers to the representative direction correction table 251 shown in FIG. Next, the corrected edge gradient direction is acquired by converting it using the direction conversion table 252 shown in FIG. For example, “1” is obtained from the fourth row and the eleventh column of the representative direction correction table 251 in FIG. 16 for the pixel of interest with the main axis v _integer = 18 and the edge gradient direction θhat (x, y) = 10. Further, “6” is obtained as the corrected edge direction gradient from the fourth row and second column of the direction conversion table 252 of FIG. As described above, the direction correction unit 25 acquires any one of 0 to 7 as the corrected edge gradient direction for each pixel in the neighboring region NR.

When the voting cell is determined by the voting cell determination unit 24 and the corrected edge gradient direction is obtained by the direction correction unit 25, the local feature amount calculation unit 26 votes the edge strength of the pixel. The local feature quantity calculation unit 26 obtains a local feature quantity d = (d ₀ , d ₁ ,..., D ₁₃₅ ) ^T by performing such voting processing for all the pixels in the neighborhood region NR. The local feature amount d _i is expressed by an equation (17).

Here, S _i in equation (17) is given by equation (18).
Also, index (x, y) in equation (18) is given by equation (19).

  Next, the local feature amount calculation method of the present embodiment will be described. The local feature amount calculation method according to the present embodiment calculates a local feature amount from an input image. The local feature amount calculation method may be executed by the local feature amount calculation device 2 configured by hardware, or software (computer program) that is mounted on the arithmetic processing device and configures the local feature amount calculation device 2. ) May be executed. FIG. 18 is a flowchart of the local feature amount calculation method of the present embodiment.

  The feature point vicinity region extraction unit 21 extracts a feature point p from the image TP (step S31). Next, the feature point neighborhood region extraction unit 21 sets a neighborhood region NR in the vicinity of the feature point p (step S32). Then, the feature point neighborhood region extraction unit 21 outputs the feature point p and the neighborhood region NR to the direction strength calculation unit 22. The direction intensity calculation unit 22 calculates the edge gradient direction θ (x, y) for each pixel in the neighboring region NR by differentiating each pixel in the neighboring region NR in the x direction and the y direction (step) S33), and quantize it to obtain the quantized edge gradient direction θhat (x, y) (step S34). The direction intensity calculator 22 also calculates a weighted edge intensity mhat (x, y) for each pixel in the neighboring region NR (step S35). Note that any of step S33 and subsequent step S34 and step S35 may be performed first or in parallel.

Next, the principal axis direction detection unit 23 votes the edge gradient direction by voting the weighted edge strength mhat (x, y) to the quantized edge gradient direction θhat (x, y) of the pixels in the neighboring region NR. A histogram is generated (step S36). The principal axis direction detection unit 23 detects the edge gradient direction (discretized value) having the maximum value in the gradient direction histogram as the principal axis direction v _integer of the neighboring region NR, and outputs the detected value to the local feature amount calculation unit 26 ( Step S37).

The local feature quantity calculation unit 26 receives the edge gradient direction θhat (x, y) and the weighted edge strength mhat (x, y) quantized from the direction intensity calculation unit 22, and receives the spindle from the main axis direction detection unit 23. A direction v _integer is input. The local feature quantity calculation unit 26 refers to the voting cell pattern table for each pixel in the neighboring region NR and determines a voting cell to which the pixel of interest belongs in the voting cell pattern corresponding to the main axis direction v _integer (step S38). ).

  When the voting cell pattern table shown in FIG. 12 is stored in the voting cell determination unit 24 in the local feature amount calculation unit 26, the voting cell is determined with reference to the voting cell pattern table in step S38. To do. On the other hand, when the representative voting cell pattern table 241 shown in FIG. 13 and the cell number conversion table 242 shown in FIG. 14 are stored in the voting cell determination unit 24 (in the case of FIG. 10), in step S38, the voting cell The voting cell is determined by referring to the representative cell pattern table 241 and the cell number conversion table 242 as the pattern table.

The local feature amount calculation unit 26 refers to the direction correction table for each pixel in the neighborhood region NR, and corrects the edge gradient direction corresponding to the principal axis direction v _integer and the quantized edge gradient direction θhat (x,). Is acquired (step S39).

  If the direction correction table shown in FIG. 15 is stored in the direction correction unit 25 in the local feature amount calculation unit 26, the corrected edge gradient direction is obtained with reference to the direction correction table in step S39. To do. On the other hand, when the representative direction correction table 251 shown in FIG. 16 and the direction conversion table 252 shown in FIG. 17 are stored in the direction correction unit 25 (in the case of FIG. 10), in step S39, the representative direction correction table 251 and the direction are displayed. With reference to the conversion table 252, the corrected edge gradient direction is acquired. Note that either step S38 or step S39 may be performed first or in parallel.

  Next, for each pixel in the neighboring region NR, the local feature amount calculation unit 26 applies the weight of the pixel of interest according to the corrected edge gradient direction obtained in step S39 for the voting cell determined in step S38. The attached edge strength mhat (x, y) is voted (step S40), whereby the local feature amount d of the feature point p is obtained.

  As described above, also in the present embodiment, in order to realize the rotation invariance, the voting cell newly set after rotating the texture pattern in the vicinity region NR of the feature point p as in the prior art. Instead of voting on the cell of the pattern, voting is performed on the voting cell of the rotated voting cell pattern CP ′ rotated in accordance with the main axis direction, so that a sub that involves a huge floating point calculation for rotating the texture pattern The linear interpolation processing for each pixel is not necessary, and the local feature amount can be calculated at high speed.

Also, as described above, in order to realize rotation invariance, instead of rotating the texture pattern, voting is performed after correcting the direction of each pixel with respect to the rotated reference voting cell pattern. It is not necessary to calculate the direction of each pixel and the intensity of the direction again for the rotated texture pattern as in the technique, and the edge gradient direction and the edge strength of each pixel calculated by the direction intensity calculation unit 22 are the main axes. It can be used for detecting the direction v _integer and for calculating the local feature amount. Thereby, the differential calculation for calculating the direction of the edge gradient and the edge strength of each pixel is only required once, and the calculation of the local feature amount can be speeded up.

Further, in the present embodiment, since the main axis direction v _integer is obtained as a discretized value, a rotated voting cell pattern corresponding to each main axis direction can be prepared in advance and tabulated. When the main axis direction v _integer is obtained using a simple table, by selecting the rotated voting cell pattern corresponding to the main axis direction v _integer from the table, it is not necessary to calculate the voting cell pattern each time. Therefore, the processing can be speeded up.

  Furthermore, in this embodiment, since the reference voting cell pattern has a point-symmetric shape, the voting cell pattern table can be compressed using this periodicity, and the required memory capacity can be reduced. If the table is of the extent shown in the present embodiment, it can be put in the CPU cache and the processing can be further speeded up.

  Further, in this embodiment, a table can be prepared for the edge gradient direction of the pixel, and the edge gradient direction of each pixel is corrected using such a table, so that the processing can be speeded up.

  Furthermore, in this embodiment, since the corrected edge gradient direction has periodicity, the direction correction table can also be compressed, and the required memory capacity can be reduced. If the table is of the extent shown in the present embodiment, it can be put in the CPU cache and the processing can be further speeded up.

In the voting cell pattern of the second embodiment, each voting cell is set so that each pixel belongs to only one voting cell. Therefore, depending on the rotation angle of the rotated voting cell pattern, The same voting cell may have different areas. For example, referring to FIG. 12, the area of the voting cell 1 when the main axis direction v _integer = 0 is 21 pixels, but the area of the voting cell 1 when the main axis direction v _integer = 0 is 20 pixels, which is the same. Despite being a voting cell, both areas are slightly different due to the difference in the rotation angle of the rotated voting cell pattern. Then, even if the images are the same, the number of pixels that vote the edge strength for the same cell increases because one of the principal axes is different, and the other decreases. Occurs.

  In consideration of this point, the local feature amount calculation unit 26 may vote by further weighting the weighted edge strength according to the size of the voting cell. For example, the reciprocal of the area can be used as this weight. Thereby, the local feature-value which is not influenced by the difference in the area of a voting cell by the difference in a rotation angle can be calculated | required.

  In the voting cell pattern of the second embodiment, the voting cell is set so that each pixel belongs to only one voting cell. Therefore, depending on the rotation angle of the rotated voting cell pattern, An error occurs near the boundary of the voting cell.

In consideration of this point, the local feature amount calculation unit 26 draws a voting cell pattern with a solid line for pixels located on the boundary of the voting cell, and draws the pixel on the solid line as shown in FIG. For, a plurality of voting cells may be voted according to the area divided by practice. FIG. 19A is a rotated voting cell pattern with the main axis direction v _integer = 0 shown in FIG. 12, and FIGS. 19B and 19C show a part of them (the voting cell 15 and the voting cell 16 It is the figure which expanded the boundary part. FIG. 19B is an example in which no error is taken into account, and FIG. 19C is an example in which the error is relaxed.

  In FIG. 19C, for example, for the upper left pixel, 60% thereof belongs to the voting cell 15 and 40% thereof belongs to the voting cell 16. Therefore, for this cell, 60% of the weighted edge strength mhat (x, y) is voted for the voting cell 15, and 40% is voted for the voting cell 16. Thus, by making one pixel belong to a plurality of voting cells, an accurate voting value can be obtained regardless of the rotation angle of the rotated voting cell pattern.

(Third embodiment)
Next, a corresponding point search device that employs the local feature quantity calculation device of the first or second embodiment will be described as a third embodiment. FIG. 20 is a block diagram illustrating a configuration of the corresponding point search apparatus 3 according to the third embodiment. As illustrated in FIG. 20, the corresponding point search device 3 includes a local feature amount calculation device 100 and a corresponding point search unit 31. The corresponding point search device 3 may implement the configuration shown in FIG. 20 as hardware, or may implement the configuration shown in FIG. 20 by installing software in the arithmetic processing device.

  As the local feature quantity calculation apparatus 100, either the local feature quantity calculation apparatus 1 of the first embodiment or the local feature quantity calculation apparatus 2 of the second embodiment is adopted. The local feature amount calculation apparatus 100 receives the first image and the second image, calculates a local feature amount for each, and outputs the local feature amount to the corresponding point search unit 31. The corresponding point search unit 31 searches for a feature point by comparing the local feature amount of the first image with the local feature amount of the second image.

  Note that the corresponding point search device 3 can be applied as a device for searching for corresponding points between an input image and an image stored in an image database in a system for searching for an image, and a first image captured from a different viewpoint. The system for restoring the three-dimensional shape from the second image can be applied as an apparatus for searching for corresponding points between the first image and the second image, and can also be applied to another system.

  FIG. 21 is a flowchart of the corresponding point search method of the present embodiment. The corresponding point search method may be executed by the corresponding point search device 3 configured by hardware, or executed by software (computer program) that is mounted on the arithmetic processing device and configures the corresponding point search device 3. You may do.

  In the search for corresponding points, first, the local feature quantity calculation device 100 calculates the local feature quantity of the first image (step S51), and then calculates the local feature quantity of the second image (step S52). . In step S51 and step S52, either the local feature value calculation method of the first embodiment or the local feature value calculation method of the second embodiment can be adopted. Next, the corresponding point search unit 31 compares the local feature amount of the first image with the local feature amount of the second image, and searches for a corresponding point (step S53).

  As described above, according to the first to third embodiments, in order to realize the rotation invariance in the calculation of the local feature amount, the texture pattern in the region near the feature point is rotated and newly added. Instead of voting on the cell of the set voting cell pattern, the voting cell pattern is rotated and the voting cell of the rotated voting cell pattern (rotated voting cell pattern) is voted, so the texture pattern is rotated. This eliminates the need for a sub-pixel linear interpolation process with a huge floating-point calculation, thereby speeding up the process and, as a result, speeding up the corresponding point search process.

  In addition, according to the first to third embodiments, the algorithm becomes simpler than the conventional local feature quantity calculation algorithms SIFT and SURF. The scale can be reduced and the cost can be reduced. Also, when implemented as software, the size of the executable binary can be reduced, so that there is an advantage that it can be easily incorporated into various applications. Therefore, whether the local feature amount calculation apparatus of the above-described embodiment is implemented as hardware or software, it can be suitably implemented in a mobile device or a low-end device.

  In the above embodiment, the “direction” and “direction intensity” of a pixel are not limited to those based on edge information. The “direction” and “direction strength” can also be obtained by wavelet transformation or Gabor transformation. Further, when a new image is input by learning a database of edge gradient directions and edge strengths for a large number of images, the edge gradient directions and edge strengths may be obtained using this database.

  In the above embodiment, the weight for obtaining the weighted edge strength mhat (x, y) is obtained by obtaining local feature amounts for a plurality of images with w (x, y) = 1. The weight w (x, y) may be re-determined according to the obtained statistic. For example, when a strong value tends to concentrate on a specific voting cell, the contribution ratio of each voting cell can be adjusted by reducing the weight of that portion.

  In the above embodiment, the principal axis direction is detected based on image information (edge gradient direction and edge intensity image). However, the principal axis direction is not limited to this, and is provided in an imaging apparatus when generating. May be detected based on the detected values of a gyro sensor, a geomagnetic sensor, or a gravity sensor, or when a new image is input by learning a database of the main axis direction for a large number of images. The main shaft direction may be detected using this database.

  The present invention has the effect of speeding up the calculation of local feature values, and uses a local feature value calculation device, a local feature value calculation method, and a local feature value calculation device for calculating a local feature value of an image. This is useful as a corresponding point search method using a corresponding point search device and a local feature amount calculation method.

DESCRIPTION OF SYMBOLS 1 Local feature-value calculation apparatus 11 Feature point vicinity area extraction part 12 Direction intensity | strength calculation part 13 Main axis direction detection part 14 Voting cell pattern generation part 15 Direction correction pattern generation part 16 Local feature-value calculation part 2 Local feature-value calculation apparatus 21 Feature point Neighboring region extraction unit 22 Direction intensity calculation unit 23 Spindle direction detection unit 24 Voting cell determination unit 241 Representative voting cell pattern table 242 Cell number conversion table 25 Direction correction unit 251 Representative direction correction unit 252 Direction conversion table 26 Local feature amount calculation unit 3 Corresponding point search device 100 Local feature amount calculation device 31 Corresponding point search unit

Claims (20)

A local feature amount calculation device for calculating a local feature amount of an image,
A feature point neighborhood extraction unit for extracting feature points and neighborhood regions set in the vicinity of the feature points from the image;
A direction intensity calculator for calculating the direction of each pixel in the neighboring area and the intensity of the direction, respectively;
A main axis direction detector for detecting the main axis direction of the vicinity region;
For each of the pixels, the directional strength calculating unit applies the voting cell to which the pixel belongs among the voting cells of the rotated voting cell pattern obtained by rotating the reference voting cell pattern at an angle according to the principal axis direction. In accordance with the corrected direction obtained by correcting the direction of the pixel calculated in accordance with the principal axis direction, voting the intensity of the pixel direction calculated by the direction intensity calculation unit, the feature point A local feature amount calculation unit for obtaining a local feature amount of
The local feature-value calculation apparatus characterized by having provided.   The main axis direction detection unit detects the main axis direction of the neighboring area based on the direction of the plurality of pixels in the neighboring area calculated by the direction intensity calculation unit and the intensity in the direction. The local feature amount calculation apparatus according to claim 1. Furthermore, a direction quantization unit for quantizing the direction of the pixel is provided,
The main axis direction detection unit generates a direction histogram by voting the intensity of the direction of the pixel according to the direction of the pixel quantized by the direction quantization unit, and approximates a curve near the peak of the direction histogram. The local feature amount calculation apparatus according to claim 1, wherein a direction corresponding to the maximum of the curve is detected as the main axis direction.   4. The voting cell pattern generation unit configured to generate the rotated voting cell pattern by rotating the reference voting cell pattern by an angle corresponding to the main axis direction. 5. The local feature-value calculation apparatus of crab. Furthermore, a direction correction pattern generation unit that generates a direction correction pattern for correcting the direction of the pixel calculated by the direction intensity calculation unit according to the principal axis direction,
The local feature according to claim 1, wherein the local feature amount calculation unit obtains the corrected direction by correcting the direction of each pixel using the direction correction pattern. Quantity calculation device. Furthermore, a direction quantization unit for quantizing the direction of the pixel is provided,
The main axis direction detection unit generates a direction histogram by voting the intensity of the direction of the pixel according to the direction of the pixel quantized by the direction quantization unit, and has a maximum value in the direction histogram. The local feature amount calculation apparatus according to claim 1, wherein the local feature amount is detected as the main axis direction. Furthermore, a voting cell pattern table storing a plurality of voting cell patterns obtained by rotating the reference voting cell pattern at a plurality of angles respectively corresponding to a plurality of main shaft directions that can be detected by the main shaft direction detecting unit is provided. ,
The local feature amount calculation unit calculates the local feature amount by using a voting cell pattern corresponding to the main axis direction among the voting cell patterns stored in the voting cell pattern table as the rotated voting cell pattern. The local feature amount calculation apparatus according to claim 6, wherein: Furthermore, among the plurality of voting cell patterns obtained by rotating the reference voting cell pattern at a plurality of angles respectively corresponding to a plurality of main shaft directions that can be detected by the main shaft direction detecting unit, the shapes of the patterns are mutually A representative voting cell pattern table storing a plurality of representative voting cell patterns that do not overlap, and
A cell number conversion table for converting a cell number of the representative voting cell pattern according to the main axis direction;
With
The local feature amount calculation unit refers to the cell number conversion table for the cell number of the representative voting cell pattern corresponding to the main axis direction among the representative voting cell patterns stored in the voting cell pattern table, The local feature amount calculation apparatus according to claim 6, wherein a voting cell for voting the intensity in the direction is determined by performing conversion according to the main axis direction. Further, for each of a plurality of principal axis directions that can be detected by the principal axis direction detection unit, a direction correction table in which the corrected direction with respect to the direction quantized by the direction quantization unit is defined,
The local feature amount calculation unit obtains the corrected direction corresponding to the direction quantized by the direction quantization unit according to the principal axis direction by referring to the direction correction table. The local feature-value calculation apparatus in any one of Claim 6 thru | or 8. Further, among the plurality of main axis directions that can be detected by the main axis direction detection unit, the main axis directions in which the corrected direction with respect to the direction quantized by the direction quantization unit has the same period are grouped, For each group, a representative direction correction table in which a representative corrected direction with respect to the direction quantized by the direction quantization unit is defined,
A direction conversion table for converting the representative corrected direction of the representative direction correction table according to the main axis direction;
The local feature amount calculation unit calculates a representative corrected direction corresponding to the principal axis direction and the direction quantized by the direction quantization unit among the representative corrected directions defined in the representative direction correction table. The local feature amount calculation apparatus according to claim 6, wherein the corrected direction is acquired by referring to the direction conversion table and performing conversion according to the main axis direction.   The local feature amount calculation unit, for each pixel, when the pixel belongs to a plurality of the voting cells, the direction of the pixel to the plurality of voting cells according to a ratio of each voting cell in the pixel The local feature amount calculation apparatus according to claim 1, wherein the voting strength is voted.   12. The local feature amount calculation apparatus according to claim 1, wherein the direction is an edge gradient direction of a pixel, and the intensity in the direction is an edge intensity of the pixel.   The said local feature-value calculation part weights the intensity | strength of the said direction according to the magnitude | size of the voting cell which votes the intensity | strength of the said direction, and votes. Local feature amount calculation device.   14. The local feature amount calculation apparatus according to claim 1, wherein the direction intensity calculation unit calculates the intensity in the direction by weighting a pixel closer to the feature point with a higher weight. . A corresponding point search device for searching for corresponding points between a plurality of images,
The local feature amount calculation apparatus according to any one of claims 1 to 14,
A corresponding point search unit that searches for corresponding points between the plurality of images by comparing the local feature amounts of the plurality of images calculated using the local feature amount calculating device;
Corresponding point search device characterized by comprising: A local feature amount calculation method for calculating a local feature amount of an image,
A feature point extracting step of extracting feature points from the image;
A neighborhood region setting step for setting a neighborhood region in the vicinity of the feature point;
A direction calculating step for calculating the direction of each pixel in the neighboring region;
An intensity calculating step for calculating the intensity in the direction of each pixel in the neighboring area;
A main axis direction detecting step for detecting a main axis direction of the neighboring area based on the direction of the plurality of pixels in the neighboring area and the intensity of the direction;
A voting cell pattern rotating step for obtaining a rotated voting cell pattern by rotating a reference voting cell pattern at an angle according to the main axis direction;
A direction correction step of correcting the direction of each pixel according to the principal axis direction to obtain a corrected direction;
For each pixel in the neighborhood region, voting the intensity of the pixel in the direction according to the corrected direction for the voting cell to which the pixel belongs among the voting cells of the rotated voting cell pattern, A local feature amount calculating step for obtaining a local feature amount of the feature point;
The local feature-value calculation method characterized by including. A local feature amount calculation method for calculating a local feature amount of an image,
A feature point extracting step of extracting feature points from the image;
A neighborhood region setting step for setting a neighborhood region in the vicinity of the feature point;
A direction calculating step for calculating the direction of each pixel in the neighboring region;
A direction quantization step of quantizing the direction of each pixel in the neighborhood region;
An intensity calculating step for calculating the intensity in the direction of each pixel in the neighboring area;
A direction histogram is generated by voting the intensity of the direction of the pixel according to the direction of the pixel quantized in the direction quantization step, and the direction having the maximum value in the direction histogram is set as the principal axis direction of the neighboring region. A spindle direction detection step to detect as
A plurality of voting cell patterns obtained by rotating a reference voting cell pattern at a plurality of angles respectively corresponding to a plurality of main axis directions that can be detected in the main axis direction detecting step for each pixel in the vicinity region are stored. A voting cell determination step for determining a voting cell to which the pixel belongs in the voting cell pattern corresponding to the principal axis direction with reference to a voting cell pattern table;
A direction correction table in which a corrected direction for the direction quantized in the direction quantization step is defined for each of a plurality of main axis directions that can be detected in the main axis direction detection step for each pixel in the neighborhood region. A direction correction step for obtaining a corrected direction corresponding to the principal axis direction and the direction quantized in the direction quantization step;
For each pixel in the neighboring region, the voting cell determined in the voting cell determination step is voted for the intensity of the pixel in the direction according to the corrected direction, thereby local features of the feature point. A local feature amount calculating step for obtaining a quantity;
The local feature-value calculation method characterized by including.   A program for causing an arithmetic processing device to execute the local feature amount calculation method according to claim 16. A corresponding point search method for searching for corresponding points between a plurality of images,
A local feature amount calculating step of calculating a local feature amount of each of the plurality of images by the local feature amount calculating method according to claim 16 or 17,
Corresponding point search step for searching corresponding points between the plurality of images by comparing each local feature amount of the plurality of images calculated in the local feature amount calculating step;
Corresponding point search method characterized by including.   A program for causing an arithmetic processing unit to execute the corresponding point search method according to claim 19.