JP2018156544A - Information processing device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing device which can obtain a proper feature point for object determination and other purposes based on a feature amount from an image.SOLUTION: An information processing device includes: a detection unit 2 which detects a feature point from an image and obtains a feature point score of each feature point; a selection unit 3 which outputs the feature point obtained by performing selection based on the feature point score and position of each feature point from the detected feature points as the feature point of the image; a calculation unit 4 which calculates the feature amount at each feature point by referring to the neighboring region of the feature point in the image for each feature point output as the feature point of the image by the selection unit 3; and a determination unit 5 which determines that the object imaged in the image is corresponding to which object out of the plurality of prescribed objects by comparing the feature amount calculated for the image with the feature amount calculated from the photographed image for each of the plurality of prescribed objects by the calculation unit 4.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an information processing apparatus and a program capable of obtaining an appropriate feature point from an image for object determination based on a feature amount and other uses.

  A technique for determining an object from an image can convert analog information described in a medium that can be easily distributed and presented into digital information, and can improve user convenience. For example, Non-Patent Document 1 is disclosed as the technology. In Non-Patent Document 1, a feature point is detected from an image, a feature amount is calculated from the periphery of the feature point, and then compared with a feature amount accumulated in advance. Is identified.

  On the other hand, Patent Documents 1 to 3 are disclosed as techniques for further improving the accuracy with respect to the determination based on the above feature points and feature amounts. In Patent Document 1, in order to reduce erroneous correspondence of feature points, accuracy is improved by Delaunay triangulation using the feature points as vertices and topology evaluation. Patent Document 2 discloses a method for selecting effective feature amounts by evaluating the feature amount discrimination performance. In Patent Document 3, the correspondence between feature points is improved by determining the similarity of the positional relationship between feature points and edges.

JP 2014-127068 A JP 2014-134858 A JP 2014-126893 A

D. G. Lowe, `` Object recognition from local scale-invariant features, '' Proc. Of IEEE International Conference on Computer Vision (ICCV), pp.1150-1157, 1999.

  However, the conventional techniques such as Non-Patent Document 1 and Patent Documents 1 to 3 described above have a problem that the distribution of feature points greatly affects the accuracy.

  Specifically, when there is a complicated pattern in the background, the feature points concentrate on the background, and there is a problem that the determination process does not function at all if a sufficient number of feature points cannot be detected from the imaging target as the foreground. For example, considering the case of adopting a framework in which an upper limit is set for the number of feature points from the viewpoint of processing time and the score is selected from the top of the score (feature point score) that defines the likelihood of feature points, Even when feature points exist, if the feature point score is relatively small, the feature points to be imaged may not be selected. This is called the first case.

  On the other hand, if there is a pattern with a large feature point score on the imaging target itself, the number of feature points reaching the upper limit in only a part of the area is selected, and it is erroneously determined as a similar target that matches only in that area. It can happen. This is referred to as a second case.

  FIG. 1 is a diagram illustrating schematic examples of the first case and the second case as [1] and [2].

  That is, in the first case shown in [1] in FIG. 1, it becomes difficult to distinguish two or more different objects given as images P1 and P2. For example, both images P1 and P2 are the covers of books belonging to the same series of the same publisher (series explaining hobbies), the book of image P1 is about climbing, and the book of image P2 is about the same series explaining about cycling. It is a separate volume. In this case, series title areas R10 and R20 indicating that the images are the same series are present in the upper part of the images P1 and P2, respectively, and these areas R10 and R20 are called completely identical patterns (characters and other patterns are called patterns). In addition, the contents of each volume are expressed because it is a separate volume in the series, and regions R11 and R21 that are configured with different patterns exist in the middle and lower side portions. Shall.

  Here, in the images P1 and P2, it is assumed that the series title regions R10 and R20 having the completely same pattern are configured as regions with high feature point scores, and the regions R11 and R21 configuring different patterns. Is configured as a region having a low feature point score, all or most feature points are detected from regions R10 and R20 having the same pattern, and no feature points are detected from regions R11 and R21 different from each other. Only a few will be detected. Therefore, the images P1 and P2 that should actually be capturing different objects are indistinguishable or difficult to distinguish.

  Further, in the second case shown as [2] in FIG. 1, the same image P1 as described in the first case is registered as a reference image, and the query image obtained by actually photographing the same object is the image P10. If it is configured as described above, it is difficult to determine whether the images P1 and P10 match. That is, the query image P10 forms a region R13 that does not exist in the reference image P1, for example, by giving “band” to the lower part of the book cover for the purpose of sales promotion. The middle region R12 of the query image P10 partially matches the middle and lower region R11 of the reference image P1, and the upper series title regions R10 and R30 are the reference image P1 and the query image P10. Is common.

  In such a case, it is assumed that the “band” region R13 that does not exist in the reference image P1 in the query image P10 is configured as a region having a high feature point score, and all or most of the feature points in the query image P10 are regions. If it is detected from R13, for the same reason as described in the first case of [1], the same determination of images P10 and P1 that should have actually captured the same object is obtained. Becomes difficult.

  In addition, although the book cover was mentioned as a schematic example in FIG. 1, it should be noted that the same or similar problem may occur not only in the book cover but also in any kind of product package and other arbitrary objects. Further, in FIG. 1, as a schematic example, the description has been made by paying attention to only one determination target, which is a book cover, but the same or similar is also true when one or more objects are imaged in a spatial arrangement in the image. Note that the following issues can arise.

  In view of the above-described problems of the prior art, a first object of the present invention is to provide an information processing apparatus that can obtain an appropriate feature point from an image for object determination based on a feature amount and other uses. .

  A second object of the present invention is to provide an information processing apparatus that can determine a target in an image with high accuracy based on the appropriate feature point.

  In order to achieve the above object, the present invention provides an information processing apparatus, which detects a feature point from an image and obtains a feature point score of each feature point, and from each of the detected feature points, It is a first feature that the image processing apparatus includes a selection unit that outputs a feature point obtained by performing selection based on a feature point score and a position of a feature point as a feature point of the image.

  Further, according to the present invention, the information processing apparatus further refers to a neighborhood area of the feature point in the image with respect to each feature point output as the feature point of the image by the selecting unit. In the image, a calculation unit that calculates a feature amount, a feature amount calculated for the image by the calculation unit, and a feature amount calculated from the captured image for each of a plurality of predetermined targets are compared. And a determination unit that determines which of the plurality of predetermined objects corresponds to the object being imaged.

  According to the first feature, an appropriate feature point can be obtained by outputting the feature point obtained by performing the selection based on the feature point score and position of each feature point as the feature point of the image. .

  According to the second feature, it is possible to determine which of the plurality of predetermined targets corresponds to the target imaged in the image, using the obtained appropriate feature point.

It is a figure which shows the model example of the 1st example as a subject of a prior art, and a 2nd example. It is a functional block diagram of the information processor concerning one embodiment. It is a figure which shows the example of the weight map calculated in inverse proportion to depth from a captured image and a captured image. It is a flowchart which shows an example of the process of the selection part in 2nd embodiment. It is a figure which shows the example of the effect of this invention as an application example of the flow of FIG. It is a figure which shows the example of the effect of this invention about the case of many feature points from FIG.

  FIG. 2 is a functional block diagram of the information processing apparatus according to the embodiment of the present invention. As illustrated, the information processing apparatus 10 includes an imaging unit 1, a detection unit 2, a selection unit 3, a calculation unit 4, a determination unit 5, and a storage unit 6. As the information processing apparatus 10, an arbitrary information terminal including the imaging unit 1 can be used, and in addition to a portable terminal, a tablet terminal, a desktop or laptop computer, or the like can be used. In addition, some or all of the components other than the imaging unit 1 may be installed in a server, and information may be exchanged appropriately through communication. The processing content in each part of FIG. 2 is as follows.

  The imaging unit 1 captures an imaging target and outputs the image to the detection unit 2, the selection unit 3, and the calculation unit 4. Here, the user or the like operates the imaging unit 1 to perform imaging so that the imaging target includes an object determined by the determination unit 5 (any of the objects stored in advance in the storage unit 6). Assumed to be performed. As the imaging unit 1, a digital camera or the like that is provided as a standard in a mobile terminal can be used.

  The detection unit 2 detects a plurality of feature points from the image obtained by the imaging unit 1, obtains a feature point score representing the feature point likelihood for each feature point, and selects the plurality of feature points and the feature point score Output to part 3. Existing methods such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features) can be used to detect the feature points, and the feature points score can be calculated according to each method. .

  For example, in SIFT, a feature point candidate is obtained as a point giving the maximum value of a DoG (Difference of Gaussian) image (difference image group), and then a stable feature point is detected based on the principal curvature and contrast. The DoG output value (maximum value) at the detected feature point may be adopted as the feature point score.

  The selection unit 3 selects a predetermined number of feature points based on the feature point score and the position of the feature point in the image from the feature points with the feature point score obtained by the detection unit 2, and the selected feature The point is output to the calculation unit 4. The sorting unit 3 may refer to an image obtained by the imaging unit 1 when sorting. Details of the sorting process by the sorting unit 3 will be described later.

  In the calculation unit 4, with respect to each feature point selected by the selection unit 3, the feature amount of each feature point is calculated by referring to the local neighborhood of each feature point in the image obtained by the imaging unit 1, The feature amount is output to the determination unit 5. For the calculation of the feature amount in the calculation unit 4, as described in the detection unit 2, existing methods such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features) can be used. Note that the feature amount calculation method in the calculation unit 4 and the feature point detection method in the detection unit 2 may be based on different types of methods.

  The determination unit 5 compares the feature amount calculated by the calculation unit 4 with the feature amount stored in the storage unit 6 to determine a match and determines an imaging target. That is, it is specified which of the one or more predetermined imaging targets stored in the storage unit 6 is the target in the captured image.

  Here, when comparing feature quantities, various known methods can be used. For example, the similarity between histograms of BoVW (Bag of Visual words) may be calculated. Further, the coordinates of the feature points whose feature values have been calculated may be compared using the associated feature information. For example, by using RANSAC (Random Sample Consensus) etc., the outliers are excluded while trying to individually match each feature quantity that constitutes the feature information, thereby identifying the best match as a whole. A technique may be used. That is, between the query image (the image obtained by the imaging unit 1) and the reference image (the image from which the feature information stored in the storage unit 6 is extracted), the feature point coordinates associated with each other in the homography matrix Matching may be performed in consideration of the geometric consistency of

  The storage unit 6 preliminarily calculates and stores feature amounts from a plurality of captured images of the imaging target, and provides the feature amounts for reference during the determination process in the determination unit 5. Here, an artificial deformation such as projective transformation may be applied to the imaging target, and the feature amount may be calculated and stored. In particular, in the deformation in which the size changes, the feature point and the feature are obtained by associating the apparent size of the captured image in the imaging unit 1 with the depth information (scale information) obtained when the calculation unit 4 calculates the feature amount. Since the effect of improving the determination accuracy of the amount is obtained, the feature amount may be stored after linking the depth information and used for reference by the determination unit 5. Further, when the feature point coordinates are used in the determination by the determination unit 5 as in the case of the RANSAC described above, the storage unit 6 also calculates and stores the feature information in which the feature point coordinates and the feature amount are linked. You can do it.

  Note that the feature values (and feature points) stored in the storage unit 6 are the same type as the feature values calculated by the calculation unit 4 (and feature points detected by the detection unit 2).

  Details of the sorting process in the sorting unit 3 will be described below. First, as a general description, in the selection unit 3, the relative relationship between the imaging unit 1 and the imaging target obtained for each pixel coordinate of the image captured by the imaging unit 1 with respect to the feature point score of each feature point Each feature point can be selected by considering the positional relationship. The relative positional relationship to be considered is a camera based on the camera center of the camera constituting the imaging unit 1 with the target point reflected in each pixel position (u, v) of the image obtained by the imaging unit 1 The relationship is expressed as a three-dimensional space coordinate position (x, y, z) = (x (u, v), y (u, v), z (u, v)) by coordinates.

  As a result of the selection process based on the feature point score and the relative positional relationship in the selection unit 3, the distribution of the feature points suitable for the determination process in the determination unit 5 on the subsequent stage is adjusted. . In particular, in the conventional method, only the feature point score is considered, but in the present invention, the feature point distribution adjustment suitable for the determination processing is realized by further considering the relative positional relationship. It becomes. For example, feature points can be obtained by concentrating only on the regions R10 and R20 in the images P1 and P2 of FIG. 1 described above, or feature points can be obtained by concentrating only on the region R13 in the image P10 of FIG. The accuracy of the determination process is improved by preventing the occurrence of such a situation.

  As a first embodiment that also specifically considers the relative positional relationship, after calculating the depth (depth) of each pixel position of the captured image, the weight given by the predetermined relationship according to the depth is used as the feature point score. Further, additional weights may be used. That is, for each feature point i (i = 1, 2, ...), the depth d (i) at the coordinate position (u (i), v (i)) of the feature point is obtained, and the predetermined function F Is used to determine the weight w (i) = F (d (i)) and the weighted score w (i) * score () obtained by multiplying the feature point score score (i) calculated in the detection unit 2 by the weight. i) is adopted as a score in the present invention for selecting the feature point i, and the feature point i such that the weighted score w (i) * score (i) is in the upper predetermined number The sorting result in 3 may be used. Note that the weight w (i) and the feature point score score (i) may be obtained as non-negative values.

  Here, an arbitrary existing method can be used for the depth calculation in the captured image. For example, an existing method using motion parallax, Markov Random Field (for example, disclosed in Non-Patent Document 2 below), or a neural network (for example, disclosed in Non-Patent Document 3 below) can be used. Alternatively, when there are a plurality of cameras constituting the imaging unit 1, stereo parallax may be used. Further, a configuration in which a depth sensor (for example, a depth sensor based on irradiating a marker with infrared rays and imaging the marker with an infrared camera) is also used on the captured image of the imaging unit 1 is possible.

[Non-Patent Document 2] DA. Saxena and et. Al. `` Make3D: Learning 3D Scene Structure from a Single Still Image, '' IEEE Transactions of Pattern Analysis and Machine Intelligence (PAMI), 2008.
[Non-Patent Document 3] D. Eigen and et. Al. `` Depth Map Prediction from a Single Image using a Multi-Scale Deep Network, '' Advances in Neural Information Processing Systems, No. 27, pp.2366-2374, 2014.

  In addition, regarding the predetermined function F for giving the weight w (i) according to the depth d (i), how the imaging unit 1 performs imaging and how the determination unit 5 performs determination (or is stored in the storage unit 6). It is only necessary to set a predetermined one according to the feature amount selection method).

  For example, if it is assumed that the user captures an image with the imaging target relatively close to the imaging unit 1, the feature point is set so as to be inversely proportional to the depth or decreased as the depth increases. It is desirable to set a calculation weight (a weight as a positive value under the assumption that the feature point score score (i) is calculated as a non-negative value as described above). FIG. 3 is a diagram illustrating an example of a certain captured image and a weight map calculated in inverse proportion to the depth from the captured image separately from [1] and [2]. The captured image of [1] is an example of an indoor landscape where various objects (imaging targets) are arranged. The corresponding weight map of [2] has a small depth (that is, closer to the front camera). The fact that a larger weight is given to an imaging target such as a stand or a drawing image is expressed as a weight that is larger as it is closer to white and vice versa.

As another setting example of the weight, the imaging information may be divided into two foregrounds and backgrounds according to the depth, and the weight information may be set so that feature points are selected only from the foreground area. In other words, for example, the following binary weights may be set in terms of expressions. Here, th is a threshold value for distinguishing between the foreground and the background, and a predetermined value may be used, or may be dynamically obtained from a distribution histogram of the depth of feature points (depth value d (i)). Note that any existing statistical method may be used for the dynamic determination.
w (i) = 1 (d (i) ≤th, ie foreground)
w (i) = 0 (d (i)> th ie background)

  Thus, when the method of selecting feature points from only the foreground is employed, the following effects can be obtained. In other words, on the assumption that the imaging target belongs to the foreground area due to relatively close proximity, the feature points are selected only from the foreground area, eliminating the influence of the background area that does not correspond to the imaging target. Therefore, it is possible to solve the problems of the prior art and to improve the determination accuracy in the determination unit 5.

  As described above, in the first embodiment that also considers the relative positional relationship, the three-dimensional spatial coordinate position (x, y, z) = (x (u, v), y (u, v), The relationship corresponding to z (u, v)) was found and used by depth. As a second embodiment that also considers the relative positional relationship, the spatial coordinate position (x (u, v), y (u, v), z (u, v)) is not directly used, and the captured image Only the pixel position (u, v) may be used to realize sorting that can obtain substantially the same effect as in the first embodiment.

  As a general explanation, in the second embodiment, the position (u (i), v (i)) on the image of the feature point i selected in the second embodiment is made uniform without deviation, and the position ( The feature point i may be selected based on u (i), v (i)) and the feature point score score (i). At this time, the range in which the positions (u (i), v (i)) are made uniform without deviation is not necessarily the entire captured image, but a part of the captured image (for example, in the vicinity) A range in which feature points are detected may be set.

  FIG. 4 is a flowchart showing an example of processing of the selection unit 3 in the second embodiment. In step S1, after setting the shape and arrangement of the divided areas for dividing the captured image, the selection order of the divided areas, and the maximum number of feature points (upper predetermined number of feature points to be selected by the selection unit 3), the process proceeds to step S2. Proceed with

  The divided area of the captured image in step S1 may be divided by a predetermined method. For example, a predetermined number n in the horizontal direction and a predetermined number m in the vertical direction may be equally divided into a total of n × m rectangular areas. In other words, if the size (number of pixels) of the captured image is W horizontal and H vertical, it may be divided into n × m rectangular areas that are W / n vertical and H / m horizontal, respectively. . Further, by applying an existing arbitrary region dividing method based on color information, texture information, and the like to the captured image, the shape of the divided region may not necessarily be a rectangular region.

  The selection order for the divided regions in step S1 may be determined as a predetermined order according to the division method. For example, when divided into n × m rectangular areas, the raster scan order may be used. Even in the case of division that does not become a rectangular area, the selection order of the divided areas may be determined so that, for example, the raster scan order is obtained at the center of gravity of the divided areas.

  Here, in the description of FIG. 4, the divided region is expressed as R (k) (k = 1, 2,...), And the selection order is expressed as O (k). For example, if the selection order O (1) = 3, O (2) = 2, O (3) = 1 for three divided regions R (1), R (2), R (3), then `` R (3 ) (O (3) = 1st) "→" R (2) (O (2) = 2nd) "→" R (1) (O (1) = 3rd) "in this order It means to be done.

  It should be noted that when going from step S1 to (for the first time) step S2, the current value of the selection order is set as the first.

  In step S2, from among the divided regions R (k) specified by the current value O (k) of the selection order, an unselected feature point i having the maximum feature point score score (i) After selecting one, go to step S3. If there is no unselected feature point i remaining in the designated divided region R (k) in step S2, the selection may be skipped (omitted) and proceed to step S3 as it is. Also, if the maximum value of the feature point score score (i) among the unselected feature points i in step S2 is less than a predetermined threshold, the selection is skipped (omitted) and the process proceeds directly to step S3. Good.

  In step S3, it is determined whether or not the number of feature points selected in the series of steps S2 up to that point has reached a predetermined number, that is, the upper limit set in step S1, and the upper limit has been reached. If so, the process proceeds to step S4, and if not reached, the process proceeds to step S5.

  In step S4, the feature points selected in the series of steps S2 up to that point in time are output as the sorting result in the sorting unit 3, and the flow of FIG.

  In step S5, the current value O (k) of the selection order used in step S2 immediately before reaching step S5 is updated to the next value “O (k) +1” (= O (k ′)). After setting the next selection target area R (k ′), the process returns to step S2.

  In this way, each time the loop process of steps S2, S3, and S5 is repeated in FIG. 4, the divided areas, for example, the first, second, and third selection orders set in step S2, are set as feature point selection targets, respectively. In step S2, feature points are selected from the set divided areas. If the current value O (k) of the selection order is the last order in step S5, the first selection order may be set again as the next selection target.

  FIG. 5 is a diagram illustrating an application example of the flow of FIG. In FIG. 5, a certain image (not shown) is obtained when the flow of FIG. 4 is applied with the selection upper limit set to 7, 3 × 3 rectangular area division, and the selection order set to the raster scan order. The position distribution in the image of the feature points selected from is shown in [2]. FIG. 4 also shows a case in which [7] is selected from the same image based on only the feature point score value without applying the method of the present invention as a comparison with [2]. ing.

  That is, in [1] and [2] in FIG. 5, the feature points in the same image are plotted with the feature point score values written together. There are 16 feature points in total, and the feature point score is the highest. An example with each integer value from 100 to a minimum of 85 is shown. In both [1] and [2], the selected feature points are indicated by underlining their score values. In the example of [1] in which the method of the present invention is not applied, the highest value 100 to the top seven are selected up to the feature point score 94, but the image can be seen as seen in [1]. The position distribution inside is biased to the lower left. On the other hand, in the example [2] to which the technique of FIG. 4 of the present invention is applied, the bias of [1] is eliminated, and feature points are selected relatively uniformly in the image. That is, in [2], three feature point scores 88, 89, and 85 are selected in the upper part of the image, two feature point scores 92 and 91 are selected in the middle part, and feature points are selected in the lower part. Two of the scores 100 and 99 are selected, and the spatial bias of the selected feature point distribution as seen in [1] is eliminated.

  FIG. 6 is a diagram showing an application example of the flow of FIG. 4 with respect to a certain image (not shown) under a larger number of feature point upper limits than in the case of FIG. Similar to the case of FIG. 5, FIG. 6 also shows the distribution of selected feature points when [1] is the proportional upper limit number of the same feature points in the same image and the present invention is not applied. However, in the application example of the present invention shown in [2], the feature point bias to the middle and lower stages is eliminated, and the feature points are selected almost uniformly on the upper stage. Has been. In the example of FIG. 6, as shown in [3] of FIG. 1, although a large number of feature points having high feature point scores are extracted, the “band” region R13 that does not correspond to the original determination target is displayed as an image. It is an explanatory example of the effect of the present invention when it exists in the middle and lower stage side portion.

  As described above, according to the present invention, after the imaging target is imaged by the imaging unit 1, the determination unit 5 can determine the captured imaging target. At this time, since the feature points are selected based on the relative positional relationship between the imaging unit 1 and the imaging target, the feature point distribution can be optimized and high-precision determination can be performed. Furthermore, since the feature quantity at the feature point coordinates is calculated from the captured image and the feature information of the same condition is selected from the storage unit 6 and determined, the determination accuracy can be increased.

  Hereinafter, supplementary matters for explanation in the present invention will be described.

(1) As a first modification of the second embodiment of the sorting process in the sorting unit 3, a process of selecting feature points i one by one from the one having the largest feature point score score (i) is repeated Sorting is performed until the upper limit is reached, and the next update process may be performed each time one is sorted. That is, for a feature point j whose position on the image is within a predetermined distance from the already selected feature point i, the feature point score score (j) may be updated to a low value. The update to a low value may be performed, for example, by multiplying by a predetermined coefficient r (0 ≦ r <1), and N feature points are already selected within a predetermined distance with respect to a feature point j that has not been selected. If present, the feature point score may be r ^N * score (j) that is cumulatively updated N times. Note that the setting of the predetermined coefficient r = 0 corresponds to completely excluding feature points that are within a predetermined distance from already selected feature points from the selection target.

  The first modification can be realized by replacing the processing in steps S1, S2, and S5 in FIG. 4 with the following processing. That is, in step S1, the area division and the selection order of the divided areas are not set, but only the maximum feature points (upper predetermined number of feature points to be selected by the selection unit 3) are set. In step S2, an unselected feature point having the maximum feature point score is selected from the entire image instead of the selected region. In step S5, the feature point score score (j) is set to a low value for the feature point j that is within the predetermined distance from the feature point i selected in the immediately preceding step S2, instead of selecting the next divided region. Update to

  (2) As a second modification of the second embodiment regarding the sorting process in the sorting unit 3, the following may be performed. That is, the image area division is performed in the same manner as described in step S1 of FIG. Then, for each divided region R (k) (k = 1, 2,...), The total number of feature points detected by the detection unit 2 in the divided region is Num (k), and each divided region R From (k), the total of the number of feature points corresponding to the total number of feature points Num (k), r * Num (k), sorted in descending order of the feature point score is output as the sorting result from the sorting unit 3. . Here, the coefficient r may be set as a predetermined value (0 <r <1) that gives the number corresponding to the total number of feature points Num (k). In particular, the coefficient r is selected from each divided region R (k). The coefficient r may be determined as a value such that the total number “Σr * Num (k)” of the number coincides with a predetermined feature point upper limit predetermined number. Note that the number r * Num (k) multiplied by the coefficient r is generally not an integer, but may be appropriately converted to an integer by a predetermined rounding process. The total number “Σr * Num (k)” of the number may be any as long as it matches the predetermined upper limit number of feature points in the threshold determination.

  According to the second modified example, it is possible to realize an equal feature point distribution according to the number of feature points detected by the detection unit 2 in the divided region in the selection result output from the selection unit 3. Become. That is, it is possible to output the feature point distribution obtained by thinning out the feature point distribution in the captured image detected by the detection unit 2 almost uniformly in the entire image by the coefficient r as the selection result from the selection unit 3.

  (3) Regarding the sorting process in the sorting unit 3, the first embodiment and the second embodiment may be combined. For example, in the second embodiment, the feature point i that maximizes the feature point score score (i) is selected in step S2 of FIG. 4, but in step S2, the feature point score score (i) is replaced with the first feature point score (i). The feature point i that maximizes the weighting score may be selected using the weighting score w (i) * score (i) in one embodiment.

  (4) The weight w (i) in the first embodiment is not considered in the spatial coordinates such as the depth d (i), and the coordinates (u (i), v ( It may be set only based on i)). For example, a positive value weight w (i) may be set such that the maximum value is obtained at a predetermined position such as the center in the image, and the value decreases as the distance from the predetermined position increases. Also in this case, priority is given to the feature points resulting from the imaging target of the determination target, on the assumption that the imaging target to be determined is in the vicinity of the center or other predetermined position in the image. Therefore, it is possible to sort the images, which contributes to higher accuracy of determination in the determination unit 5.

  (5) The information processing apparatus 10 may be configured by only an arbitrary part of the functional units illustrated in FIG. For example, the imaging unit 1 may be omitted, and the detection unit 2 may acquire an arbitrary image from any other location on the network. However, at this time, by acquiring additional information such as camera information about the read image, the above-described processes such as the sorting process by the sorting unit 3 based on the depth described above can be performed. And it is sufficient. For example, considering the case where the information processing apparatus 10 is configured to include only the detection unit 2 and the selection unit 3, a device that selects and outputs feature points suitable for recognition and other determination processes from the image read by the detection unit 2. As a result, the information processing apparatus 10 functions.

  (6) The information processing apparatus 10 can be realized as a computer having a general configuration. That is, a CPU (Central Processing Unit), a main storage device that provides a work area for the CPU, an auxiliary storage device that can be configured with a hard disk, SSD, etc., an input interface that receives input from the user such as a keyboard, mouse, touch panel, etc., network The information processing apparatus 10 can be configured by a general computer including a communication interface for connecting to and communicating, a display for displaying, a camera, and a bus for connecting them. Furthermore, the processing of each unit of the information processing apparatus 10 shown in FIG. 2 can be realized by a CPU that reads and executes a program for executing the processing, but any part of the processing is performed in a separate dedicated circuit or the like. It may be realized. The imaging unit 1 can be realized by a camera as the hardware.

  DESCRIPTION OF SYMBOLS 10 ... Information processing apparatus, 1 ... Imaging part, 2 ... Detection part, 3 ... Sorting part, 4 ... Calculation part, 5 ... Determination part, 6 ... Storage part

Claims (15)

A detection unit for detecting feature points from the image and obtaining a feature point score of each feature point;
A selection unit that outputs, as feature points of the image, feature points obtained by performing selection based on a feature point score and a position of each feature point from the detected feature points; Information processing device.   The information processing apparatus according to claim 1, wherein the selection unit performs the selection based on the position based on a spatial position of each feature point based on a camera center of a camera that has captured the image. .   The information processing apparatus according to claim 2, wherein the selection unit performs selection based on a spatial position of each feature point as selection based on a depth of each feature point.   The information processing apparatus according to claim 3, wherein the sorting unit performs sorting based on the depth of each feature point with priority as the depth becomes smaller.   The selection unit divides the image into a foreground region and a background region, and performs the selection based on the position by selecting only feature points in the divided foreground region as selection targets. Item 4. The information processing apparatus according to Item 1.   The information processing apparatus according to claim 1, wherein the selection unit performs selection based on the position as selection based on a pixel coordinate position in the image of each feature point.   The selection unit performs the selection based on the position as the selection based on the pixel coordinate position of the feature point in the image, and performs the selection so that the distribution of the pixel coordinate position of the feature point to be selected becomes uniform. The information processing apparatus according to claim 6, wherein the information processing apparatus is performed.   The selecting unit selects each divided region obtained by dividing the image into regions in a predetermined order, and continues to select feature points from the selected divided regions based on the feature point score according to the predetermined order. The information processing apparatus according to claim 6, wherein the feature point of the image is output.   The selection unit selects, based on the feature point score, the number of feature points corresponding to the total number of the detected feature points belonging to the divided region from the divided regions obtained by dividing the image into regions. The information processing apparatus according to claim 6, wherein a feature point of the image is output.   The information according to claim 6, wherein the sorting unit performs sorting based on the position such that the pixel coordinate position of each feature point in the image is preferentially sorted closer to a predetermined coordinate position. Processing equipment.   The sorting unit preferentially sorts the feature points based on the feature point score and position of each feature point as the weighting score obtained by multiplying the feature point score of each feature point by the weight based on the position of each feature point is higher. The information processing apparatus according to claim 1, wherein the information processing apparatus performs the processing.   For each feature point output as a feature point of the image by the selection unit, the image processing apparatus further includes a calculation unit that calculates a feature amount at each feature point by referring to a neighborhood region of the feature point in the image. The information processing apparatus according to claim 1.   By comparing the feature amount calculated for the image by the calculation unit with the feature amount calculated from the captured image for each of a plurality of predetermined targets, the target imaged in the image is the plurality of target images. The information processing apparatus according to claim 12, further comprising a determination unit that determines which one of the predetermined objects corresponds.   The information processing apparatus according to claim 1, further comprising an imaging unit that acquires the image by performing imaging.   A program for causing a computer to function as the information processing apparatus according to any one of claims 1 to 14.