JP6126979B2 - Feature selection apparatus, method, and program - Google Patents

Description

  The present invention relates to a feature selection device, method, and program, and more particularly, to a feature selection device, method, and program for performing feature selection.

  In the feature selection method for local features, focus on the appearance frequency of local features in multiple images representing the same concept or object, and select local features with high appearance frequency as useful features for search and recognition. (Non-patent document 1, Non-patent document 2, Non-patent document 3).

  Specifically, a method of selecting a local feature with a high posterior probability (Non-Patent Document 1), taking the posterior probability of an image classification label when a certain local feature appears in an image as the appearance frequency of the local feature, and the posterior probability A measure called information acquisition amount is defined based on the entropy of the image, and it is regarded as the appearance frequency of local features, and a local feature with a high information acquisition amount is selected (Non-Patent Document 2), and the feature selection problem is a noise feature exclusion problem Focusing on the appearance frequency of local features in negative example images, focusing on negative example images that tend to be misrecognized as interest classification labels (positive examples). (Non-patent Document 3).

  In addition, the feature selection method that focuses on the contextual relationship between feature points is a local feature that has strong spatial consistency by performing collation based on spatial context for multiple images that may represent the same concept or object. Is selected as a feature of interest useful for search and recognition (Non-Patent Document 4).

  In the method of Non-Patent Document 4, RANSAC (Non-Patent Document 5), which is one of spatial context matching methods, is used to select a feature point called Inlier, which is an output of RANSAC, with high spatial consistency.

  In addition, the feature expression method that takes into account the spatial context focuses on feature points with large shading changes in the image, focusing on contextual relationships such as the relative positional relationship between the feature points and geometric characteristic differences. Is expressed as an image feature.

  Specifically, a multi-scale Delaunay diagram is proposed and used to detect a triad of feature points that are neighboring in the image space, and a feature representation method (Non-Patent Document 6) using this triad as a basic unit, Using feature-based methods to detect feature point pairs that are close to each other in the image space, a feature expression method (Non-Patent Document 7) based on the difference in geometric characteristics (size and main direction) of this pair, and similarity between images A number of geometric transformations are temporarily set based on feature points, feature points that are evidence of each temporary are defined as Inliers, and an image expression / matching method called RANSAC with the number of Inliers as a scale, and between similar feature points Focusing on geometric characteristics (size and main direction), image representation and collation method called Weak Geometric Consistency by geometric characteristic difference histogram (non- Patent Document 8).

  The above method can also be classified according to whether or not it depends on the geometric characteristics of feature points. Since the method of Non-Patent Document 6 is based only on the neighborhood relationship between feature points, it is classified as a method that does not depend on geometric characteristics. Is considered the basic unit. The method of Non-Patent Document 8 and the method of Non-Patent Document 7 are classified into methods that depend on geometric characteristics in order to perform image representation based on the size and main direction of feature points. Here, the method that does not depend on the geometric characteristics is robust against the error of feature point detection and description, whereas the method that depends on the geometric characteristics has an advantage of high discrimination power against incorrect images.

Fayin Li and Jana Kosecka.Probabilistic location recognition using reduced feature set.In ICRA, pp. 3405 {3410, 2006. Grant Schindler, Matthew Brown, and Richard Szeliski.City-scale location recognition. In CVPR, 2007. Jan Knopp, Josef Sivic, and Tomas Pajdla.Avoiding confusing features in place recognition.In ECCV (1), pp. 748 {761, 2010. P. Turcot and D.G.Lowe.Better matching with fewer features: The selection of useful features in large database recognition problems.In Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on, pp. 2109 {2116, 2009. James Philbin, Ondrej Chum, Michael Isard, Josef Sivic, and Andrew Zisserman.Object retrieval with large vocabularies and fast spatial matching.In CVPR, 2007. Yannis Kalantidis, Lluis Garcia Pueyo, Michele Trevisiol, Roelof van Zwol, and Yannis S. Avrithis.Scalable triangulation-based logo recognition. In ICMR, p. 20, 2011. Zhen Liu, Houqiang Li, Wengang Zhou, and Qi Tian.Embedding spatial context information into inverted le for large-scale image retrieval.In ACM Multimedia, pp. 199 {208, 2012. Herve Jegou, Matthijs Douze, and Cordelia Schmid.Improving bag-of-features for large scale image search.International Journal of Computer Vision, Vol. 87, No. 3, pp. 316 {336, 2010.

  The methods of Non-Patent Documents 1 to 3 focus only on a single feature point and do not consider the context relationship between a plurality of feature points, and thus have a problem of being easily affected by a noise region such as white noise.

  Regarding feature selection based on feature expression in consideration of a spatial context, the method of Non-Patent Document 4 can select only local features having strong spatial consistency in a plurality of input images. Therefore, as a result, although it is useful for search, recognition, etc., there is a tendency that features of interest such as low spatial consistency due to insufficient input multiple images are excessively excluded. .

  In addition, because it depends on the geometric characteristics (shape, etc.) of feature points, it is sensitive to errors in feature point detection / description, and because spatial consistency analysis depends on RANSAC, which has a high computational load, a large database Difficult to adapt to.

  In the method of Non-Patent Document 7, a spatial context feature expression / feature selection method based on neighborhood detection by the K neighborhood method can be considered, but there is a problem that the calculation load of the K neighborhood method itself is high and the processing time is long.

  The present invention has been made to solve the above-described problems, and an object thereof is to provide a feature selection apparatus, method, and program capable of selecting an appropriate feature of interest.

  In order to achieve the above object, a feature selection device according to a first invention includes an input unit that receives a plurality of images representing an object, and a feature point of the image for each of the plurality of images received by the input unit. A local feature extraction unit that extracts each of the local features of the feature points, and for each of the plurality of images, a feature point of the image, and a local feature extracted from another image Detecting each feature point having a common local feature as a common feature point, and for each of the plurality of images, detecting each of a plurality of common feature point pairs among the detected common feature points of the image Generating a spatial context feature representation representing local features of the plurality of common feature points for each of the detected pairs, and for each of the plurality of images, the detected pair, Painting A spatial context feature representation unit that detects each of the pairs of spatial context feature representations that are common to the spatial context feature representation generated from the image, and the images detected in the spatial context feature representation unit for each of the plurality of images A region of interest estimation unit that estimates a region of interest based on the pair of the common spatial context feature representations, and each of the plurality of images is included in the region of interest of the image estimated by the region of interest estimation unit And a feature selection unit that selects each of the local features of the feature points as attention features.

  A feature selection method according to a second invention is a feature selection method in a feature selection device including an input unit, a local feature extraction unit, a spatial context feature expression unit, a region of interest estimation unit, and a feature selection unit. The input unit receives a plurality of images representing an object, and the local feature extraction unit extracts each of feature points of the image for each of the plurality of images received by the input unit, and the feature Each of the local features of the points is extracted, and the spatial context feature representation unit extracts, for each of the plurality of images, a local feature that is a feature point of the image and is common to the local features extracted from other images. Each of the feature points is detected as a common feature point, and for each of the plurality of images, a plurality of common feature point pairs among the common feature points of the detected image are detected, and the detected Each of the pairs On the other hand, a spatial context feature representation representing local features of the plurality of common feature points is generated, and for each of the plurality of images, the detected pair of the spatial context feature representations generated from other images Each of the pair of spatial context feature representations that are in common with each other, and the region of interest estimation unit, for each of the plurality of images, the common spatial context feature of the images detected in the spatial context feature representation unit A region of interest is estimated based on the pair of expressions, and the feature selection unit, for each of the plurality of images, local features of feature points included in the region of interest of the image estimated by the region of interest estimation unit Are selected as features of interest.

  According to the first and second inventions, the input unit receives a plurality of images representing an object, the local feature extraction unit extracts each feature point for each of the images, and the local feature of the feature point is extracted. Each of the plurality of images is extracted by the spatial context feature expression unit, and each of the feature points having the same local features as the local features extracted from other images is common to each of the plurality of images. A plurality of common feature points among the common feature points of the detected image are detected for each of the plurality of images, and the plurality of common features is detected for each of the detected pairs. A spatial context feature representation that represents a local feature of a point is generated, and for each of a plurality of images, a pair of the spatial context feature representation that is detected and is common to the spatial context feature representation generated from another image Detect each of The attention area estimation unit estimates the attention area for each of the plurality of images based on the pair of spatial context feature expressions common to the detected images. Each of the local features of the feature points included in the estimated region of interest of the image is selected as the feature of interest.

  Thus, according to the first and second inventions, each of the local features for each feature point extracted for each of the images representing the object is extracted, and each of the spatial context feature expressions common to the plurality of images is extracted. For each of the plurality of images based on the common spatial context feature expression, and for each of the plurality of images, each of the local features of the feature points included in the estimated region of interest is determined. By selecting as a feature of interest, an appropriate feature of interest can be selected.

  Further, in the first invention, the spatial context feature representation represents a local feature of a pair of a plurality of common feature points existing in the vicinity, obtained based on a multiscale Delaunay diagram for the common feature points of the image. It may be a thing.

  In the first invention, the spatial context feature expression may represent a local feature of a plurality of pairs of common feature points existing in the vicinity obtained from the common feature points of the image by a K-neighbor method. .

  In the first invention, the spatial context feature expression may represent a local feature of a plurality of common feature point pairs without considering a geometric characteristic of the plurality of common feature point pairs. Good.

  In the first invention, the spatial context feature representation may represent a local feature of a plurality of common feature point pairs and a difference in geometric characteristics of the plurality of common feature point pairs.

  Moreover, the program of this invention is a program for functioning a computer as each part which comprises said feature selection apparatus.

  As described above, according to the feature selection device, method, and program of the present invention, each local feature for each feature point extracted for each image representing an object is extracted, and is common to a plurality of images. Detecting each of the spatial context feature representations, estimating a region of interest for each of the plurality of images based on a common spatial context feature representation, and feature points included in the estimated region of interest for each of the plurality of images By selecting each of the local features as the feature of interest, an appropriate feature of interest can be selected.

It is a block diagram which shows the functional structure of the feature selection apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the functional structure of the local feature extraction part which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the functional structure of the spatial context characteristic expression part which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the functional structure of the attention area estimation part which concerns on the 1st Embodiment of this invention. It is a figure which shows the example which detects an attention vicinity. It is a block diagram which shows the functional structure in the code book learning apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart figure which shows the code book learning process routine in the code book learning apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart figure which shows the feature selection process routine in the feature selection apparatus concerning the 1st Embodiment of this invention. It is a block diagram which shows the functional structure of the feature selection apparatus which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the functional structure of the spatial context characteristic expression part which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the functional structure of the attention area estimation part which concerns on the 2nd Embodiment of this invention. It is a flowchart figure which shows the feature selection process routine in the feature selection apparatus concerning the 2nd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Configuration of Feature Selection Device According to First Embodiment>
Next, the configuration of the feature selection device according to the first embodiment of the present invention will be described. As shown in FIG. 1, a feature selection apparatus 100 according to the first embodiment of the present invention includes a CPU, a RAM, a ROM that stores a program and various data for executing a feature selection processing routine described later. , Can be configured with a computer including. Functionally, the feature selection apparatus 100 includes an input unit 10, a calculation unit 20, and an output unit 90 as shown in FIG.

  The input unit 10 receives each of a plurality of images including the same target (concept, object, etc.).

  The calculation unit 20 includes a local feature extraction unit 30, a spatial context feature expression unit 40, an attention area estimation unit 60, and a feature selection unit 70.

  For each of the images received by the input unit 10, the local feature extraction unit 30 extracts feature points from the image, and extracts local feature vectors of the feature points. As shown in FIG. 2, the local feature extraction unit 30 includes a feature point extraction unit 32, a feature point description unit 34, a code book storage unit 36, and a Visual Word allocation unit 38. . A local feature vector is an example of a local feature.

  For each of the images received by the input unit 10, the feature point extraction unit 32 uses an affine invariant feature detector such as Hessian Affine or Harris Affine from the image and an image obtained by inverting the image. , And the center of the elliptical area is extracted as a feature point, and the coordinates of the feature point are obtained.

  For each image, the feature point description unit 34, for each of the feature points of the image extracted by the feature point extraction unit 32, the image obtained by inverting the image, the coordinates of the feature point, Based on the above, the local feature vector of the elliptical region of the feature point is calculated. As an example of the local feature vector, SIFT, SURF, Root SIFT, and the like are calculated.

  The code book storage unit 36 stores the code book output from the code book learning device 200 described later. Here, the code book is a set of unique Visual Words defined corresponding to each local feature vector.

  For each of the images, the Visual Word assigning unit 38, for each feature point of the image extracted by the feature point extracting unit 32, a local feature vector of the feature point calculated by the feature point description unit 34, and a code Based on the code book stored in the book storage unit 36, the Visual Word that is the nearest neighbor of the local feature vector is obtained using the approximate nearest neighbor method, and assigned to the local feature vector of the feature point.

  The spatial context feature representation unit 40 performs feature representation / matching in consideration of the spatial context based on each of the local feature vectors extracted for each of the images by the local feature extraction unit 30, and the spatial context feature with other images. A pair of common feature points with common expressions is detected. As shown in FIG. 3, the spatial context feature expression unit 40 includes a feature point index assigning unit 42, a common feature point detecting unit 44, a neighborhood detecting unit 46, a neighborhood description unit 48, and a neighborhood index assigning unit 52. And a common neighborhood detector 54.

  The feature point index assigning unit 42 uses, for each feature point extracted for each image in the feature point extracting unit 32, transposition index assignment based on the Visual Word assigned to the feature point. Corresponding to the Visual Word assigned to the feature point, the feature point is put into the feature point index to generate a feature point index.

  Based on the feature point index generated by the feature point index assigning unit 42, the common feature point detecting unit 44 extracts the feature points of the image extracted by the feature point extracting unit 32 for each of the images received by the input unit 10. And feature points in which feature points to which the same Visual Word is assigned are extracted from other images are detected as common feature points. Specifically, when there are feature points of a plurality of images as feature points corresponding to Visual Word in the feature point index, the feature points are detected as common feature points.

  The neighborhood detection unit 46 applies multiscale Delaunay triangulation to each of the common feature points of the image extracted by the common feature point detection unit 44 for each of the images received by the input unit 10 to A scale Delaunay diagram is generated, a pair of common feature points existing in the vicinity is detected, and a feature point neighborhood set consisting of a set of pairs of common feature points existing in the detected neighborhood is detected for each image. Specifically, for each of the images received by the input unit 10, the common feature points of the image are rearranged in the order of the size of the elliptical area corresponding to the common feature points, and a plurality of obtained lists are obtained. Are subdivided into subsets, and the Delaunay triangulation is applied to each subset to generate a multi-scale Delaunay diagram. Then, two common feature points connected by the sides of the Delaunay triangle are detected as a pair of common feature points existing in the vicinity.

  For each of the images received by the input unit 10, the neighborhood description unit 48 creates, for each pair of common feature points that exist in the vicinity, based on the feature point neighborhood set of the image detected by the neighborhood detection unit 46. Is generated as a spatial context feature representation of the pair, and assigned to the pair.

  The neighborhood index assigning unit 52 includes, for each pair of common feature points existing in the neighborhood included in each of the feature point neighborhood sets detected by the neighborhood detection unit 46, based on the spatial context feature expression assigned to the pair. Then, using a transposed index assignment or the like, the pair is placed in the neighborhood index in correspondence with the spatial context feature expression assigned to the pair, and a neighborhood index is generated.

  Based on the neighborhood index generated by the neighborhood index assigning unit 52, the common neighborhood detecting unit 54 exists in the neighborhood detected for the image by the neighborhood detecting unit 46 for each of the images received by the input unit 10. A pair of common feature points that are assigned to the same spatial context feature expression and a pair of common feature points existing in the neighborhood is detected from another image, and a spatial context Detect as a pair with common feature expression.

  The attention area estimation unit 60 sets, for each of the images received by the input unit 10, a pair of common feature points that are detected for the image by the spatial context feature representation unit 40 and share the same spatial context feature representation with other images. Based on this, an attention area is estimated. Further, as shown in FIG. 4, the attention area estimation unit 60 includes a common neighborhood extension unit 62, a transition closure unit 64, and a boundary rectangle calculation unit 66.

  The common neighborhood extension unit 62 shares the same multi-scale Delaunay diagram generated by the neighborhood detection unit 46 with respect to each image received by the input unit 10 and the spatial context feature expression detected by the common neighborhood detection unit 54 and other images. Based on each of the pair of common feature points, the pair of common feature points common to other images and the spatial context feature representation is expanded, and the expanded pair of common feature points is detected as the attention neighborhood. Specifically, for each of the images received by the input unit 10, for each pair of common feature points detected by the common neighborhood detection unit 54 and having a common spatial context feature expression with another image, an edge connecting the pair is determined. For a Delaunay triangle to be included, if the diagonal of the side connecting the pair is equal to or greater than a predetermined threshold (for example, an obtuse angle), the triangle is extended to both sides sandwiching the diagonal and detected as the attention neighborhood. An example is shown in FIG.

  For each image received by the input unit 10, the transition closure unit 64 corresponds to each pair of common feature points that are detected for the image by the common neighborhood detection unit 54 and share a spatial context feature expression with other images. The edges and the edges corresponding to each of the attention neighborhoods detected by the common neighborhood extension unit 62 are combined to be regarded as an undirected graph, and at least one or more connected graphs are detected using transitive closure.

  The boundary rectangle calculation unit 66 calculates, for each of the images in the input unit 10, a minimum boundary rectangle that includes each of the connected graphs of the image detected in the transition closure unit 64, and obtained at least one or more obtained The union at the boundary rectangle is calculated, and the area represented by the calculated union is estimated as the attention area of the image.

  The feature selection unit 70 includes, for each of the images received by the input unit 10, local features of the feature points extracted by the feature point extraction unit 32 included in the attention area of the image estimated by the boundary rectangle calculation unit 66. Select a vector as the feature of interest.

  The output unit 90 outputs each feature of interest selected by the feature selection unit 70.

<Configuration of Codebook Learning Device According to First Embodiment>
Next, the configuration of the code book learning device according to the first embodiment of the present invention will be described. As shown in FIG. 1, the code book learning device 200 according to the first embodiment of the present invention stores a CPU, a RAM, a program for executing a code book construction processing routine described later, and various data. It can be composed of a computer including a ROM. Functionally, the code book learning apparatus 200 includes an input unit 210, a calculation unit 220, and an output unit 290 as shown in FIG.

  The input unit 210 receives a plurality of images for codebook learning.

  The calculation unit 220 includes a feature point extraction unit 232, a feature point description unit 234, a code book storage unit 236, and a code book construction unit 238.

  For each of the images received by the input unit 210, the feature point extraction unit 232 uses the affine invariant feature detector such as Hessian Affine or Harris Affine from the image and an image obtained by inverting the image. And the coordinates of each feature point are obtained.

  The feature point description unit 234, for each of the images received by the input unit 210, for each feature point of the image extracted by the feature point extraction unit 232, as in the feature point description unit 34, Compute the local feature vector of the region.

  The code book storage unit 236 stores the code book learned by the code book construction unit 238.

  Based on the local feature vectors of the feature points calculated by the feature point description unit 234, the code book construction unit 238 learns the code book using a method such as approximate K-Means or Vocabulary Tree. The data is stored in the storage unit 236 and output to the output unit 290. Specifically, the Visual Word corresponding to each of the local feature vectors is learned, and a set of pairs of the local feature vector and the Visual Word is used as a code book.

  The output unit 290 outputs the code book learned by the code book construction unit 238.

<Operation of the code book learning device according to the first embodiment>
Next, the operation of the code book learning apparatus 200 according to the first embodiment of the present invention will be described. When the input unit 210 receives a plurality of images for codebook learning, the codebook learning device 200 executes a codebook learning process routine shown in FIG.

  First, in step S100, each feature point is extracted from each of a plurality of images received by the input unit 210.

  Next, in step S104, for each of the plurality of images received by the input unit 210, for each feature point acquired in step S100, a local feature vector of the elliptical region of the feature point is calculated.

  Next, in step S106, the code book is learned based on the local feature vectors of the feature points acquired in step S104, and stored in the code book storage unit 236.

  Next, in step S108, the code book acquired in step S106 is output from the output unit 290, and the process of the code book learning process routine is terminated.

<Operation of Feature Selection Device According to First Embodiment>
Next, the operation of the feature selection apparatus 100 according to the first embodiment of the present invention will be described. The code book learned in advance by the code book learning device 200 is received by the input unit 10 and stored in the code book storage unit 36 of the feature selection device 100. When the input unit 10 receives a plurality of images including the same target (concept, object, etc.), the feature selection device 100 executes a feature selection processing routine shown in FIG.

  First, in step S <b> 200, each feature point is extracted from each image received by the input unit 10.

  Next, in step S204, for each image received by the input unit 10, a local feature vector of each feature point of the image acquired in step S200 is calculated.

  In step S206, the code book stored in the code book storage unit 36 is read.

  Next, in step S208, for each of the images received in the input unit 10, for each feature point of the image acquired in step S200, the local feature vector of the feature point acquired in step S204, and in step S206. Based on the acquired codebook, a Visual Word is assigned to the feature point using an approximate nearest neighbor method.

  Next, in step S210, for each feature point extracted for each of the images received in the input unit 10 acquired in step S200, in step S208, based on the Visual Word assigned to the feature point, The feature point is entered into the feature point index using transposition indexing or the like, and a feature point index is generated.

  Next, in step S212, for each of the images received by the input unit 10 based on the feature point index generated in step S210, the feature points of the image acquired in step S200 and the same Visual Word. A feature point to which a feature point assigned to is extracted from another image is regarded as a common feature point and is detected.

  Next, in step S214, multiscale Delaunay triangulation is applied to each of the common feature points of the image acquired in step S212 for each of the images received by the input unit 10, and a multiscale Delaunay diagram is displayed. A pair of common feature points that are generated and detected in the vicinity are detected, and a feature point neighborhood set that is a set of pairs of common feature points that exist in the detected neighborhood is detected.

  Next, in step S216, for each image received by the input unit 10, the pair is configured for each pair of common feature points existing in the vicinity based on the feature point neighborhood set acquired in step S214. A descriptor obtained by connecting the Visual Word assigned in step S208 to each common feature point is generated as a spatial context feature expression of the pair, and assigned to the pair.

  Next, in step S218, for each of the pairs, based on the spatial context feature expression assigned to each pair in step S216, using the transposed index assignment or the like, the spatial context feature expression assigned to the pair is converted into the spatial context feature expression assigned to the pair. Correspondingly, the pair is placed in the neighborhood index to generate a neighborhood index.

  Next, in step S220, for each of the images received by the input unit 10 based on the neighborhood index acquired in step S218, the common feature points existing in the vicinity detected for the image acquired in step S214. In step S216, a pair of common feature points existing in the neighborhood, to which the same spatial context feature expression is assigned, is detected from another image as a spatial context. Detect as a pair with common feature expression.

  Next, in step S222, for each of the images received by the input unit 10, each of the pairs of common feature points detected for the image acquired in step S220 and having a common spatial context feature expression with the other images, and Based on the multi-scale Delaunay diagram acquired in step S214, a pair of common feature points having a common spatial context feature expression with another image is expanded, and the expanded result is detected as the attention neighborhood.

  Next, in step S224, each of the images received in the input unit 10 corresponds to each pair of common feature points detected in the image acquired in step S220 and having a common spatial context feature expression with other images. The edge corresponding to each of the attention neighborhoods acquired in step S222 is considered as an undirected graph, and at least one connected graph is detected using transitional closure.

  Next, in step S226, for each image received by the input unit 10, the attention area of the image is estimated based on each of the connection graphs of the image acquired in step S224.

  Next, in step S228, for each of the images received by the input unit 10, the local feature vectors of the feature points acquired in step S204 included in the target region of the image acquired in step S226 are selected as the target features. To do.

  Next, in step S230, the feature of interest acquired in step S228 is output from the output unit 90, and the processing of the feature selection processing routine is terminated.

  As described above, according to the feature selection device according to the first embodiment of the present invention, each of the local features for each feature point extracted for each of the images representing the object is extracted, and another image is obtained. Common feature points that are common to local features are detected, and by using multiscale Delaunay triangulation, pairs of common feature points existing in the vicinity are detected, and common feature points having a common spatial context feature representation in multiple images are detected. Detect each pair, estimate a region of interest that does not depend on geometric characteristics based on a common spatial context feature representation for each of multiple images, and include each of the multiple images in the estimated region of interest An appropriate feature of interest can be selected by selecting each of the local features of the feature point to be selected as the feature of interest.

  In addition, multi- Delaunay triangulation makes it possible to more efficiently detect pairs of common feature points existing in the vicinity, and as a result, the efficiency of feature selection can be increased.

  In addition, by indexing, it is possible to more efficiently realize feature expression / collation considering the spatial context, and as a result, feature selection can be applied to a large-scale database.

  In addition, spatial context feature expression that does not depend on geometric characteristics enables feature selection that is robust against feature point detection and description errors.

  In addition, it is possible to estimate a region of interest of an image representing the same concept or object from a plurality of input images, and as a result, although it is useful for search, recognition, etc., the plurality of input images are not valid. Even if the feature of interest has a low spatial consistency because it is sufficient, more complete feature selection can be realized without being excessively excluded.

  Further, by expanding the neighborhood of the feature, a more complete attention area estimation can be performed.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, in the first embodiment, a case has been described in which the spatial context feature representation is a combination of Visual Words assigned to each of feature points included in a pair of common feature points existing in the vicinity. However, the present invention is not limited to this. As in the second embodiment to be described later, a combination of Visual Words assigned to each of feature points included in a pair of common feature points existing in the vicinity, and a pair of common feature points existing in the vicinity A spatial context feature representation that represents a difference in geometric characteristics of each of the feature points included may be generated.

  In the first embodiment, the case has been described in which the feature selection unit 70 selects, as the feature of interest, the local feature vector of each feature point included in the region of interest for each image received by the input unit 10. However, the present invention is not limited to this. For example, the feature selection unit 70 extracts feature points from the attention area for each image received by the input unit 10, calculates a local feature vector for each of the extracted feature points, and calculates the calculated local vector. You may make it select as an attention feature.

  Next, a second embodiment will be described. In addition, about the part which becomes the structure and effect | action similar to 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  In the second embodiment, the spatial context feature expression further represents a difference in geometric characteristics of feature points, and a pair of common feature points existing in the vicinity is detected using the K-neighbor method. The difference from the first embodiment is that a pair of common feature points that share a spatial context feature expression with other images is not expanded.

<Configuration of Feature Selection Device According to Second Embodiment>
Next, the configuration of the feature selection device 300 according to the second embodiment will be described.

  As shown in FIG. 9, the feature selection device 300 according to the second exemplary embodiment of the present invention includes an input unit 10, a calculation unit 320, and an output unit 90.

  The calculation unit 320 includes a local feature extraction unit 30, a spatial context feature expression unit 340, an attention area estimation unit 360, and a feature selection unit 70.

  The spatial context feature representation unit 340 performs feature representation / matching in consideration of the spatial context based on each of the local feature vectors extracted for each of the images by the local feature extraction unit 30, and the spatial context feature with other images. A pair of common feature points with common expressions is detected. Further, as shown in FIG. 10, the spatial context feature expression unit 340 includes a feature point index assigning unit 42, a common feature point detection unit 44, a neighborhood detection unit 346, a neighborhood description unit 348, and a neighborhood index assignment unit 52. And a common neighborhood detector 54.

  The neighborhood detection unit 346 applies the K neighborhood method to each of the common feature points of the image extracted by the common feature point detection unit 44 for each of the images received by the input unit 10 and exists in the neighborhood. A common feature point pair is detected, and a feature point neighborhood set composed of a set of common feature point pairs existing in the detected neighborhood is detected for each image.

  For each of the images received by the input unit 10, the neighborhood description unit 348 determines, for each pair of common feature points existing in the neighborhood, based on the feature point neighborhood set of the image detected by the neighborhood detection unit 346. Connected to each of the common word points constituting the common feature points, and the geometric characteristic difference of each of the common feature points constituting the pair (difference in the size of the ellipse region, difference in the main direction, A spatial context feature representation representing the shortest distance between ellipses is generated and assigned to the pair.

  The attention area estimation unit 360 sets, for each of the images received by the input unit 10, a pair of common feature points that are detected for the image by the spatial context feature expression unit 340 and share a spatial context feature expression with other images. Based on this, an attention area is estimated. Further, as shown in FIG. 11, the attention area estimation unit 360 includes a transition closure unit 364 and a boundary rectangle calculation unit 66.

  For each image received by the input unit 10, the transition closure unit 364 corresponds to each pair of common feature points that are detected for the image by the common neighborhood detection unit 54 and share a spatial context feature expression with other images. The edge is regarded as an undirected graph, and at least one connected graph is detected using transitive closure.

<Operation of Feature Selection Device According to Second Embodiment>
Next, the operation of the feature selection apparatus 300 according to the second embodiment of the present invention will be described. The code book learned in advance by the code book learning device 200 is received by the input unit 10 and stored in the code book storage unit 36 of the feature selection device 300. When the input unit 10 receives a plurality of images including the same target (concept, object, etc.), the feature selection device 300 executes a feature selection processing routine shown in FIG.

  First, in step S <b> 200, each feature point is extracted from each image received by the input unit 10.

  Next, in step S204, for each image received by the input unit 10, a local feature vector of each feature point of the image acquired in step S200 is calculated.

  In step S206, the code book stored in the code book storage unit 36 is read.

  Next, in step S208, for each of the images received in the input unit 10, for each feature point of the image acquired in step S200, the local feature vector of the feature point acquired in step S204, and in step S206. Based on the acquired codebook, a Visual Word is assigned to the feature point using an approximate nearest neighbor method.

  Next, in step S210, for each feature point extracted for each of the images received in the input unit 10 acquired in step S200, in step S208, based on the Visual Word assigned to the feature point, The feature point is entered into the feature point index using transposition indexing or the like, and a feature point index is generated.

  Next, in step S212, for each of the images received by the input unit 10 based on the feature point index generated in step S210, the feature points of the image acquired in step S200 and the same Visual Word. A feature point to which a feature point assigned to is extracted from another image is regarded as a common feature point and is detected.

  Next, in step S300, for each of the images received by the input unit 10, the K feature method is applied to each of the common feature points of the image acquired in step S212, and common feature points existing in the vicinity. And a feature point neighborhood set consisting of a set of common feature point pairs existing in the detected neighborhood.

  Next, in step S302, for each of the images received by the input unit 10, the pair is configured for each common feature point pair existing in the vicinity based on the feature point neighborhood set of the image acquired in step S300. Each of the common feature points connected to the Visual Word assigned in step S208 and the geometric characteristic difference of each of the common feature points constituting the pair (difference in the size of the ellipse region, the main difference) A spatial context feature representation representing a difference in direction, the shortest distance between ellipses, etc.) is generated and assigned to the pair.

  Next, in step S218, for each of the pairs, based on the spatial context feature expression assigned to each pair in step S302, using the transposed index assignment or the like, the spatial context feature expression assigned to the pair Correspondingly, the pair is placed in the neighborhood index to generate a neighborhood index.

  Next, in step S220, for each of the images received in the input unit 10 based on the neighborhood index acquired in step S218, the common feature points existing in the vicinity detected for the image acquired in step S300. And a pair in which a common feature point pair existing in the vicinity, to which the same spatial context feature representation acquired in step S302 is assigned, is detected from another image, and another image and space Detect as a pair with common context feature expressions.

  Next, in step S304, for each of the images received by the input unit 10, each pair of common feature points detected for the image acquired in step S220 and having a common spatial context feature representation with another image is obtained. The corresponding edge is regarded as an undirected graph, and at least one connected graph is detected using transitive closure.

  Next, in step S226, for each image received by the input unit 10, the attention area of the image is estimated based on each of the connection graphs of the image acquired in step S304.

  Next, in step S228, for each of the images received by the input unit 10, the local feature vectors of the feature points acquired in step S204 included in the target region of the image acquired in step S226 are selected as the target features. To do.

  Next, in step S230, the feature of interest acquired in step S228 is output from the output unit 90, and the processing of the feature selection processing routine is terminated.

  As described above, according to the feature selection device according to the second embodiment of the present invention, each local feature for each feature point extracted for each of the images representing the object is extracted, and another image is obtained. Common feature points that are common to local features are detected, a pair of common feature points existing in the neighborhood is detected by the K-neighbor method, and a pair of common feature points having a common spatial context feature representation in a plurality of images is detected. Detecting each of the plurality of images, estimating a region of interest in consideration of geometric characteristics based on a common spatial context feature expression, and including the features included in the estimated region of interest for each of the plurality of images By selecting each local feature of a point as a feature of interest, an appropriate feature of interest can be selected.

  Note that the present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  For example, in the second embodiment, a combination of Visual Words assigned to each of feature points included in a pair of common feature points existing in the vicinity, and a pair of common feature points existing in the vicinity Although the case where the spatial context feature expression representing the difference between the geometric characteristics of each feature point to be generated is described has been described, the present invention is not limited to this. As in the first embodiment, a combination of Visual Words assigned to each feature point included in a pair of common feature points existing in the vicinity may be generated as a spatial context feature expression.

  Further, in the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium or provided via a network. It is also possible to do.

DESCRIPTION OF SYMBOLS 10 Input part 20 Operation part 30 Local feature extraction part 32 Feature point extraction part 34 Feature point description part 36 Codebook storage part 38 Visual Word allocation part 40 Spatial context feature expression part 42 Feature point index addition part 44 Common feature point detection part 46 Neighborhood detection unit 48 Neighborhood description unit 52 Neighborhood indexing unit 54 Common neighborhood detection unit 60 Region of interest estimation unit 62 Common neighborhood extension unit 64 Transition closure unit 66 Boundary rectangle calculation unit 70 Feature selection unit 90 Output unit 100 Feature selection device 200 Codebook Learning device 210 Input unit 220 Calculation unit 232 Feature point extraction unit 234 Feature point description unit 236 Code book storage unit 238 Code book construction unit 290 Output unit 300 Feature selection device 320 Calculation unit 340 Spatial context feature expression unit 346 Neighborhood detection unit 348 Neighborhood Description part 360 Attention area estimation part 364 Transition closure part

Claims (7)

An input unit for receiving a plurality of images representing the object;
For each of a plurality of images received at the input unit, it extracts each characteristic point of the image, as a local feature of the feature point, and the local feature extractor for exiting each extract the Visual Word,
For each of the plurality of images, a feature point of the image, each of the feature points having a local feature common to a local feature extracted from another image is detected as a common feature point,
For each of the plurality of images, a pair of a plurality of common feature points in the image among the common feature points of the detected image is detected, and the plurality of pairs are detected for each of the detected pairs. Generating a spatial context feature representation including a descriptor concatenating Visual Words that are local features of the common feature points of
A spatial context feature representation unit that detects each of the pairs of spatial context feature representations that are the detected pairs and are common to spatial context feature representations generated from other images for each of the plurality of images; ,
A region of interest estimation unit that estimates a region of interest based on the pair of the common spatial context feature representations of the images detected in the spatial context feature representation unit for each of the plurality of images;
For each of the plurality of images, a feature selection unit that selects each of the local features of feature points included in the region of interest of the image estimated by the region of interest estimation unit as a feature of interest;
A feature selection device. An input unit for receiving a plurality of images representing the object;
For each of the plurality of images received in the input unit, each of the feature points of the image is extracted, and a local feature extraction unit that extracts each of the local features of the feature points;
For each of the plurality of images, a feature point of the image, each of the feature points having a local feature common to a local feature extracted from another image is detected as a common feature point,
For each of the plurality of images, each of a plurality of common feature points among the common feature points of the detected image is detected, and the plurality of common feature points is detected for each of the detected pairs. Generate a spatial context feature representation that represents local features of
A spatial context feature representation unit that detects each of the pairs of spatial context feature representations that are the detected pairs and are common to spatial context feature representations generated from other images for each of the plurality of images; ,
A region of interest estimation unit that estimates a region of interest based on the pair of the common spatial context feature representations of the images detected in the spatial context feature representation unit for each of the plurality of images;
For each of the plurality of images, a feature selection unit that selects each of the local features of feature points included in the region of interest of the image estimated by the region of interest estimation unit as a feature of interest;
Only including,
The feature selection apparatus, wherein the spatial context feature representation represents a local feature of a plurality of pairs of common feature points existing in the vicinity, obtained based on a multiscale Delaunay diagram for the common feature points of the image .   The feature selection apparatus according to claim 1, wherein the spatial context feature representation represents a local feature of a pair of a plurality of common feature points existing in the vicinity obtained from a common feature point of the image by a K-neighbor method.   4. The spatial context feature representation represents a local feature of a plurality of common feature point pairs and a difference in geometric characteristics of the plurality of common feature point pairs. The feature selection device according to item. A feature selection method in a feature selection device including an input unit, a local feature extraction unit, a spatial context feature expression unit, a region of interest estimation unit, and a feature selection unit,
The input unit receives a plurality of images representing an object,
The local feature extraction unit for each of the plurality of images received at the input unit, extracts each characteristic point of the image, as a local feature of the feature point, out each extract the Visual Word,
The spatial context feature expression unit, for each of the plurality of images, is a feature point of the image, each feature point having a local feature common to a local feature extracted from another image as a common feature point Detect
For each of the plurality of images, a pair of a plurality of common feature points in the image among the common feature points of the detected image is detected, and the plurality of pairs are detected for each of the detected pairs. Generating a spatial context feature representation including a descriptor concatenating Visual Words that are local features of the common feature points of
For each of the plurality of images, detecting each of the detected pairs of spatial context feature representations that are in common with spatial context feature representations generated from other images;
The attention area estimation unit estimates an attention area for each of the plurality of images based on the pair of the common spatial context feature representations of the images detected by the spatial context feature representation unit,
The feature selection unit selects, for each of the plurality of images, each of local features of feature points included in the attention region of the image estimated by the attention region estimation unit as a feature of interest. A feature selection method in a feature selection device including an input unit, a local feature extraction unit, a spatial context feature expression unit, a region of interest estimation unit, and a feature selection unit,
The input unit receives a plurality of images representing an object,
The local feature extraction unit extracts each of the feature points of the image for each of the plurality of images received in the input unit, extracts each of the local features of the feature point,
The spatial context feature expression unit, for each of the plurality of images, is a feature point of the image, each feature point having a local feature common to a local feature extracted from another image as a common feature point Detect
For each of the plurality of images, each of a plurality of common feature points among the common feature points of the detected image is detected, and the plurality of common feature points is detected for each of the detected pairs. Generate a spatial context feature representation that represents local features of
For each of the plurality of images, detecting each of the detected pairs of spatial context feature representations that are in common with spatial context feature representations generated from other images;
The attention area estimation unit estimates an attention area for each of the plurality of images based on the pair of the common spatial context feature representations of the images detected by the spatial context feature representation unit,
The feature selection unit selects, as attention features, local features of feature points included in the attention region of the image estimated by the attention region estimation unit for each of the plurality of images.
Including
The feature selection method, wherein the spatial context feature representation represents a local feature of a plurality of pairs of common feature points existing in the vicinity obtained based on a multi-scale Delaunay diagram for the common feature points of the image . The program for functioning a computer as each part which comprises the characteristic selection apparatus of any one of Claims 1-4 .

Images