JP7088618B2 - Information processing equipment and programs - Google Patents

Description

The present invention relates to an information processing apparatus and a program capable of calculating feature information suitable for recognizing an object with high accuracy from an image.

A device that recognizes an object from an image can convert analog information described in a medium that is easy to distribute and present into digital information, and can improve user convenience. For example, in Non-Patent Document 1, a feature point is detected from an image, a local image feature amount is calculated from the periphery of the feature point, and then the local image feature amount is collated with a pre-stored local image feature amount to obtain a target type. And recognize relative positional relationships.

Furthermore, the following devices have been published as devices for improving recognition accuracy and devices for acquiring robustness. In Non-Patent Document 2, the SIFT feature amount is extracted after the image is deformed by biaxial rotation, and the correspondence of the feature points is made highly accurate. In Patent Document 1, in order to reduce erroneous correspondence of feature points, the recognition rate is improved by Delaunay triangulation with the feature points as vertices and topology evaluation. Patent Document 2 discloses a method of selecting an effective feature amount by evaluating the feature amount discrimination performance. In Patent Document 3, the degree of similarity between the positional relationship between the feature points and the edges is determined to improve the accuracy of the correspondence between the feature points.

Japanese Unexamined Patent Publication No. 2014-127068 Japanese Unexamined Patent Publication No. 2014-134858 Japanese Unexamined Patent Publication No. 2014-126893

D. G. Lowe, "Object recognition from local scale-invariant features", Proc. Of IEEE International Conference on Computer Vision (ICCV), pp.1150-1157, 1999. J.-M. Morel and G. Yu: "ASIFT: A new framework for fully affine invariant image comparison", SIAM J. Imaging Sciences, 2, 2, pp. 438-469, 2009.

However, the above-mentioned conventional techniques have the following problems in order to perform highly accurate recognition.

Non-Patent Document 1, Patent Document 1, Patent Document 2, and Patent Document 3 have a problem that the accuracy is lowered if the feature points cannot be sufficiently detected. In particular, the accuracy is significantly reduced for objects that originally have few feature points. Further, when there are similar objects, there is a problem that there is a risk of erroneous recognition that the features are not calculated from the regions where there is a difference. In Non-Patent Document 2, since the feature point may be easily detected due to the deformation of rotation, the above problem is partially solved. However, the increase in the processing time related to the deformation of rotation and the increase in the processing time due to the increase in the number of feature points become problems. In addition, there is a problem that it cannot be dealt with when there are a plurality of targets.

In view of the above problems of the prior art, an object of the present invention is to provide an information processing apparatus and a program capable of calculating feature information suitable for recognizing an object from an image with high accuracy.

In order to achieve the above object, the present invention is an information processing apparatus, which is characterized by a processing unit that performs a plurality of processing processes on an image to be imaged to obtain each processed image, and each of the processed images. By collating the calculated unit for calculating the information with the calculated feature information and the feature information stored in advance for the predetermined plurality of recognition targets, the image pickup target in the image is the predetermined plurality of recognition targets. It is characterized by including a recognition unit that recognizes which one it corresponds to. Further, the present invention is characterized in that it is a program that causes a computer to function as the information processing apparatus.

According to the present invention, by using a processed image, it is possible to calculate feature information suitable for recognizing an object with high accuracy from the image.

It is a functional block diagram of the information processing apparatus which concerns on one Embodiment. It is a flowchart of the operation of the information processing apparatus which concerns on one Embodiment. As a preferred example of the second embodiment, it is a figure which shows an example which makes it possible to recognize the same recognition object of a three-dimensional shape at various angles. As an example corresponding to the example of FIG. 3, it is a figure which shows the example which the processing process of a plane projective transformation is hierarchically defined in each kth loop process. It is a flowchart of the embodiment which performs further additional processing on the premise of the embodiment of FIG. A schematic example to which the embodiment of FIG. 5 is applied is shown with respect to an image pickup target configured as a housing. A schematic example to which the embodiment of FIG. 5 is applied is shown with respect to a first recognition target configured as a front wheel and a second recognition target configured as a rear wheel.

Hereinafter, the present invention will be described with reference to the drawings. The present invention is suitable when a portable terminal is used as the information processing apparatus 10 and the image pickup target is an arbitrary object, as in the case of realizing augmented reality display in one example. However, the information processing device of the present invention is not limited to the mobile terminal, and may be any information processing device provided with the image pickup unit 1, for example, a desktop type, a laptop type, or other computer. And so on. Further, a part or all other than the image pickup unit 1 may be installed on the server, and information may be exchanged by communication on the network as appropriate. Similarly, the image pickup unit 1 may be an external configuration, and the present invention may be applied to, for example, an image acquired from a server or the like on a network.

FIG. 1 is a functional block diagram of the information processing apparatus 10 according to the embodiment. The information processing apparatus 10 includes an image pickup unit 1, a processing unit 2, a calculation unit 3, a recognition unit 4, and a storage unit 5. The outline of each part is as follows.

The image pickup unit 1 captures an image pickup target by accepting an image pickup operation or the like by a user, and outputs the captured image to the processing unit 2. The imaging target includes a recognition target known in advance. The recognition target may be, for example, a marker, a printed matter, a three-dimensional object, or the like having a pattern having a known feature or the like. As the hardware for realizing the image pickup unit 1, a digital camera, which is often installed as standard equipment in mobile terminals, can be used.

The processing unit 2 applies one or more types of processing to the image captured by the imaging unit 1, obtains a processed image for each of the applied processing processes, and outputs the processed image to the calculation unit 3. For example, if the embodiment is preset to apply the three types of processing M101, M102, M103, the three types of processing M101, M102, M103 are applied to the captured image P to obtain three processed images. Q101, Q102, and Q103 are obtained and output to the calculation unit 3. It should be noted that the plurality of processing processes may include a process in which nothing is processed, that is, a process in which the captured image is used as a processed image as it is.

Here, as for the type and order of application of the processing to be applied, a predetermined type of processing and a processing in a predetermined order according to a predetermined rule are set in advance according to the embodiment. As an example of the application of the predetermined rule, as shown by the line L2 in FIG. 1, there is one that is applied in the form of an instruction to the processing unit 2 based on the recognition result by the recognition unit 4. Details of each embodiment of the processing unit 2 will be described later.

The calculation unit 3 first detects the feature points to be recognized from each of the processed images obtained by the processing unit 2. As the feature point to be detected, a feature point such as a corner in the recognition target can be used. As a detection method, existing methods for detecting characteristic points such as SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Features) can be used. In the calculation of the integrated feature information described later, the detected feature point coordinates may be shared among a plurality of machining information to suppress the detection omission of the feature point.

Next, the calculation unit 3 calculates the local image feature amount from each processed image centering on the detected feature point coordinates. The plurality of feature points and local image feature quantities calculated as described above by the calculation unit 3 are output to the recognition unit 4 as feature information in each processed image. For example, when three processed images Q101, Q102, and Q103 are obtained, the three corresponding feature information f101, f102, and f103 are output to the recognition unit 4. As a method for calculating the local image feature amount, an existing method for calculating a characteristic amount such as SIFT (Scale-Invariant Feature Transform) or SURF (Speeded Up Robust Features) can be used.

The recognition unit 4 obtains the similarity between the feature information calculated by the calculation unit 3 and the feature information stored in the storage unit 5, so that the image captured by the image pickup unit 1 corresponds to the maximum similarity. Obtain the recognition result of the image pickup target. That is, it is specified which of one or more predetermined imaging targets stored in the storage unit 5 as a reference corresponds to the object in the captured image as a query. Here, when comparing feature information with each other, various well-known methods can be used. For example, RANSAC (Random Sample Consensus) or the like is used to individually match each feature amount constituting the feature information. You may use a technique to identify the best match as a whole by eliminating outliers while trying to do so.

Here, the feature information fi (i = 1,2, ...) in one or more processed images Qi (i = 1,2, ...) obtained by processing the captured image P obtained as a query in the imaging unit 1 ...) and the similarity between the feature information Fj (i = 1,2, ...) in the target Oj (j = 1,2, ...) as a reference to be stored in the storage unit 5. By obtaining the score score (fi, Fj) and then specifying the pair (processed image Qmax_i, target Omax_j) that gives the maximum value of the similarity score, the recognition result of the recognition unit 4 can be obtained. .. The formula is as follows.

That is, in the present invention, the object corresponding to the captured image P is the target Omax_j stored in the storage unit 5 (recognition result in the normal sense), and it is the captured image that gives the maximum similarity. The fact that the processed image Qmax_i is obtained from P (and the fact that the processing processing in the corresponding processing unit 2 is the processing processing Mmax_i) is included in the recognition result.

It should be noted that the above recognition result is based on the embodiment A described later, and the recognition result in another format can be obtained in the embodiment B described later.

The storage unit 5 stores the feature information calculated from each of the recognition targets as a reference in advance, and uses it as a reference for the recognition process in the recognition unit 4 described above. The same type of method as in the calculation unit 3 is used for the calculation of the feature information to be stored.

FIG. 2 is a flowchart of the operation of the information processing apparatus 10 according to the embodiment. As shown in the figure, steps S2 to S5 and S7 are loop (repeated) processes, but when explaining each step, the number of loop processes at that time is set to k (k = 1,2, ...). I will refer to it. The processing contents of each step are as follows.

In step S1, the image pickup unit 1 takes an image of the recognition target, obtains the captured image P, outputs it to the processing unit 2, and then proceeds to step S2. In step S2, the processing unit 2 performs a plurality of predetermined processing processes on the obtained captured image P, obtains a processed image corresponding to each processing process, outputs the processed image to the calculation unit 3, and then steps S3. Proceed to. Here, the machining process in step S2 is the machining process M (k, i) (i = 1, 2, .; .., K (k)), and the processed image Q [M (k, i)] corresponding to each processing process is output. Each embodiment of the processing unit 2 as a specific example of the processing process M (k, i) will be described later.

In step S3, the calculation unit 3 calculates the feature information f [M (k, i)] from each processed image Q [M (k, i)] obtained in the latest step S2, and the recognition unit 4 After outputting to, proceed to step S4. In step S4, the degree of similarity between each feature information f [M (k, i)] obtained in the latest step S3 and the feature information Fj stored in the storage unit 5 for each recognition target Oj. The recognition result is obtained by the recognition unit 4 finding the one with the maximum score score (f [M (k, i)], Fj) according to the above equation (1), and then the process proceeds to step S5. .. Here, the recognition result obtained in step S4 of the kth loop processing is used as the processing image Q (k, imax (k)) of the processing process M (k, imax (k)) and the recognition target Ojmax (of the reference). It is coded as if the similarity score with k)) was the largest.

Further, as another embodiment of steps S3 and S4, the following may be performed. That is, each feature information f [M (k, i)] obtained in step S3 is transferred between a series of processing processes M (k, i) (i = 1, 2, ..., K (k)). As an integrated image, the integrated feature information f [k] of the captured image P is obtained and output to the recognition unit 4, and in step S4, the recognition unit 4 recognizes the integrated feature information f [k] and each recognition. The feature information Fj stored for the target Oj and the one with the maximum similarity score score (f [k], Fj) (j = jmax (k)) are recognized and the recognition result Ojmax (k) is output. You may.

Here, in order to clarify the explanation, the former embodiment of steps S3 and S4 (in which the similarity is evaluated by the individual feature information f [M (k, i)] for each machining process) is implemented in the A embodiment. , The latter embodiment (the one in which the similarity evaluation or the like is performed by the feature information f [k] integrated between a series of processing processes) is referred to as the second embodiment.

In step S5, the recognition unit 4 determines whether or not the final recognition result is obtained at the kth time, and if the positive judgment is obtained, the process proceeds to step S6. In the case of a negative judgment that has not been obtained, the process proceeds to step S7.

The determination in step S5 may be determined based on whether or not the number of times k at the relevant time point has reached a predetermined upper limit value according to the embodiment, or the series obtained in step S4 up to the kth time point. It may be determined whether or not any of the maximum similarity scores of the above exceeds a predetermined threshold value. It should be noted that an embodiment in which "1" is set as the predetermined upper limit value of the number of times k at the relevant time point is also possible, and in this case, the second and subsequent loop processes via step S7 are not performed.

In step S6, the recognition unit 4 outputs the final recognition result by selecting from the recognition results obtained in the series of steps S4 in each of the above k loop processes, and then the flow of FIG. 2 ends. do. The final recognition result selected may be the result in the final step S4 or may be given the maximum similarity score in a series of steps S4. In FIG. 1, the output of the selected final recognition result is shown by line L1.

In step S7, when the embodiment A is applied, the recognition result (processed image M (k, imax (k)) obtained by the recognition unit 4 for the kth loop processing in the latest step S4 and the recognition target Ojmax of the reference are used. Based on the recognition result) that the similarity score was maximized between k)), the processing process M (k + 1, i) (i = 1) in the next loop processing k + 1th time for the processing unit 2. , 2, ..., K (k + 1)), then return to step S2. In this way, in step S2 in the loop processing k + 1th time, the processing unit 2 is subjected to the designated processing processing M (k + 1, i) (i = 1, 2) from the recognition unit 4 in the latest kth step S7. , ..., K (k + 1)) is performed to obtain each processed image, and the same processing according to the k + 1st processed image and feature information is continued in the continuing steps S3 and S4. It will be.

Further, in step S7, when the embodiment B is applied, the recognition result (the target Ojmax (k) is calculated by calculating the similarity score with the integrated feature information f [k], as in the case of the above embodiment A. Based on the recognition result that is the maximum similarity), the machining process M (k + 1, i) (i = 1, 2, ..., K (k + 1)) in the loop process k + 1st time. May be specified. An example of the designation is the embodiment of Supplement (2) described later.

The flowchart of FIG. 2 has been described above. The flow is an embodiment that gives a general form of processing by the processing unit 2 and designation processing by the recognition unit 4 of the processing, but in the following, each specific processing and designation is given. An embodiment will be described.

In the first embodiment, the loop processing from step S5 to step S6 is performed at the first time point of k = 1, and the loop processing after the second time is not performed, and the embodiment B is applied.

In the first embodiment, one or more types of resolution conversions preset as the processing process of the processing unit 2 in step S2 are specifically applied. The first embodiment is based on the following considerations. That is, since the feature point calculation by the calculation unit 3 in step S3 in the subsequent stage has the property that the distribution changes due to a slight difference in the images, it is better to calculate N feature points from only one captured image P. Considering that it is better to calculate N / M feature points from the processing information of M sheets converted to different resolutions, which is more effective for mapping the same N feature points. be.

Further, in the calculation of the integrated feature information by the calculation unit 3 in step S3 of the first embodiment, specifically, the features having overlapping coordinates may be integrated. That is, since the correspondence between the coordinate positions is known between the resolution-converted processed images, the first feature point in the first resolution processed image and the second feature point in the second resolution second processed image are known. However, when it is determined by the threshold determination that they are in the same position by applying the conversion of the positional relationship, one of the averages of the first feature amount of the first feature point and the second feature amount of the second feature point. The feature amount may be obtained as the feature amount of the feature points corresponding to each other. Here, the integration between two different resolution processed images has been described, but even between three or more images, the averaged feature amount can be obtained by determining the threshold value of the correspondence between the coordinate positions in exactly the same way. With this integration, the minimum required features can be selected and the increase in the number of feature points can be suppressed, so the processing time can be expected to increase.

When converting the resolution in the processing unit 2 in step S2 of the first embodiment, the aspect ratio of the captured image should be maintained in the embodiment in which the geometric verification which is an existing technique is also used for the recognition in the recognition unit 4. It is desirable to use a series of resolution conversions. However, since a slight change in the aspect ratio can be tolerated by setting the threshold value for geometric verification, it is possible to promote the change in the feature point distribution by intentionally changing the aspect ratio within a predetermined range. For example, it may be converted to a plurality of resolutions different by one pixel. To give a specific example, when the initially captured image is 640 × 480 pixels, a processed image with resolution conversion such as 641 × 480 pixels, 640 × 481 pixels, and 641 × 481 pixels may be output. A processed image that reduces only one pixel, such as 639 × 480 pixels, may be output.

The second embodiment is to which the A embodiment is applied.

Specifically, in the second embodiment, one or more planar projective transformations (transformation by a homography matrix) preset as a machining process are applied. The second embodiment is based on the following considerations. That is, since the value of the feature amount calculation in the subsequent calculation unit 3 changes depending on the relative positional relationship (positional relationship such as the posture with respect to the camera constituting the image pickup unit 1), the feature amount is calculated as it is from the captured image. It is more effective to calculate the feature amount for mapping by calculating the feature amount from the processed image that has been planarly projected so that the positional relationship is the same as the positional relationship when the feature amount stored in the storage unit 5 is calculated. Is a consideration that can be expected.

For example, the feature amount calculated from the images of the same recognition target of the three-dimensional shape taken from various angles is stored in the storage unit 5 as being in different postures of the same recognition target (assuming that they are formally different recognition targets). When stored, only the feature amount from the facing region can be correctly determined from the captured image of the image pickup target, but the feature amount from the non-facing region is held in the storage unit 5. Does not match. By generating a processed image in which a projective transformation such as imaging from an angle is applied to the imaging information, it can be expected that the feature amount calculation that is effective for the correspondence in the recognition unit 4 can be expected.

FIG. 3 is a diagram showing an example in which the same recognition target of the three-dimensional shape can be recognized at various angles as a preferable example of the second embodiment. That is, it is assumed that the captured image P captures a home appliance having a rectangular parallelepiped housing as a recognition target. It is assumed that the planar projective transformations M1, M2, and M3 are among the plurality of planar projective transformations preset in the second embodiment, and each transformation M1, M2, M3 (transformation by the homography matrix) is captured. It is assumed that the result applied to P is the processed images Q1, Q2, and Q3 as shown in the figure. In this case, the processed images Q1, Q2, and Q3 are converted into a state in which the upper surface, the left side surface, and the right side surface of the rectangular parallelepiped housing imaged in the captured image P are more facing each other than in the case of the captured image P. It is supposed to be done. Therefore, if the storage unit 5 stores in advance the characteristic information calculated from the reference images in the housing in which the upper surface, the left side surface, and the right side surface face each other, each of these surfaces is stored in the second embodiment. Highly accurate recognition is possible.

In particular, the posture of the imaged object is generally unknown in the captured image P, but in the second embodiment, by setting a series of planar projective transformation matrices in advance, at least one of the transformations can be performed as shown in FIG. It is a transformation that is almost directly opposed, and it is possible to increase the probability that feature information similar to the feature information stored in the storage unit 5 can be obtained. The set of plane projection matrices includes at least those that change the direction of the plane (rotation that changes the distance from the center of the camera by including rotation components other than around the optical axis of the camera). It is preferable that the thing) is included.

When the above second embodiment is applied to the loop processing count k ≧ 2 or more in the flow of FIG. 2, by defining a plurality of planar projective transformations to be applied to each k times in a tree shape, it is stepwise and iterative. It can be expected that the recognition accuracy will be improved by the conventional recognition unit 4. That is, in step S7 of FIG. 2, processing in the vicinity of processing processing M (k, imax (k)) as a planar projective transformation giving the maximum similarity score in the kth loop processing is performed. The process should be defined as the one to be applied to the k + 1th time. That is, for each processing M (k, i) (i = 1, 2, ..., K (k)) of the kth time, the processing processing that gives the processing in the vicinity thereof should be applied to the k + 1th time. You just have to define it.

FIG. 4 is a diagram showing an example in which the processing of the planar projective transformation is hierarchically defined in each k-th loop processing as an example corresponding to the example of FIG. For example, if the processed image Q1 (in the conversion M1 that makes the upper surface of the housing almost face-to-face (frontal)) as explained in FIG. 3 in the k = 1st loop process gives the maximum similarity, it continues. In the k = second loop processing, for example, by setting three conversions M11, M12, and M13 to be converted in the vicinity of the conversion M1 as processing application targets, the upper surface is further improved while the upper surface is almost facing each other. It is possible to increase the possibility that a processed image that is close to the facing state can be obtained in any of Q11, Q12, and Q13.

That is, as is clear from the example of FIG. 4, while the loop count k is small, a plurality of coarse planar projective transformations with large changes are applied, and as the loop count k increases, the k-1st time selected with the maximum similarity. The machining process at each k times may be defined in a tree shape so that the planar projective transformation with a small change is applied in the vicinity of the planar projective transformation.

The converted M11, M12, and M13 may be defined as such that the processed images Q11, Q12, and Q13 can be obtained by applying them to the initial captured image P, or the presupposed processed image Q1 may be defined. Further, it may be defined as such that the processed images Q11, Q12, and Q13 can be obtained by converting with the conversion M11, M12, and M13, respectively. The processing by the processing unit 2 may be processed by the same method as defined above.

FIG. 5 is a flowchart of an embodiment in which further additional processing is performed on the premise of the embodiment of FIG. In the embodiment, another part of the image pickup target (the first target object described below) is further processed step by step based on the result of recognizing a part of the image pickup target in the captured image P (the "first object" described below). In particular, the "second object" described below), which is another part that was difficult to recognize in the initial captured image P, can also be recognized. Hereinafter, each step in FIG. 5 will be described.

In step S11, the information processing apparatus 10 performs the entire flow of FIG. 2 to obtain the recognition result of the first object in the captured image P, and then proceeds to step S12. The step S11 is preferably realized by the second embodiment in which the flow of FIG. 2 is processed by a planar projective transformation.

In step S12, the processed image processed by the processing unit 2 so that the captured image P (obtained in step S1 of the corresponding flow of FIG. 2) obtained by the imaging unit 1 is in a state suitable for recognition of the second object. After obtaining, proceed to step S13.

The processing of the processing portion 2 in step S12 may be performed as follows. As a premise, as an additional process in the recognition of the first target in step S11, the plane projection conversion H1 representing the position and orientation of the first target image pickup unit 1 with respect to the camera center is obtained in the recognition unit 4. The planar projective transformation H1 is characterized by being stored in the storage unit 5 by the same method as that used for geometric verification, which is an existing technique, or for position / orientation recognition of a square marker as an existing technique in the field of augmented reality. It can be obtained as giving a conversion relationship between the coordinates in the information and the corresponding coordinates of the recognition target in the captured image P. Alternatively, as another embodiment, the planar projective transformation matrix in the processing process that gives the highest similarity in the recognition of the first object in step S11 may be adopted as approximately corresponding to H1. In the other embodiment, the process of additionally obtaining the coordinate conversion relationship as described above can be omitted.

Further, as a premise, in the storage unit 5, it is assumed that the information of the planar projective transformation matrix H2 that faces the second object is registered in advance in association with the first object recognized in step S11. That is, the matrix H2 so that the second object can be face-to-face (and magnified) by applying the transformation by the product H2 * H1 of the above two planar projective transformations to the image P. Shall be registered. The matrix H2 is given based on the positional relationship between the first object and the second object, and is an image in which the first object has a predetermined size and faces the camera (the center of the camera of the imaging unit 1). By further converting (image H1 (P) obtained by converting the captured image P by the matrix H1) by the matrix H2, the image H2 * in which the second object faces the camera (imaging unit 1) at a predetermined size. H1 (P) can be obtained.

In step S12, the processing unit 2 applies the product H2 * H1 of the registered matrix H2 and the matrix H1 obtained in step S11 to the captured image P, so that the second object is a facing / enlarged processed image. After obtaining H2 * H1 (P), proceed to step S13.

In step S13, the calculation unit 3 calculates the feature information from the processed image H2 * H1 (P), and the recognition unit 4 has the similarity between the calculated feature information and the feature information stored in the storage unit 5. By collating, it recognizes what the second object is. In step S14, the recognition unit 4 outputs the recognition result of the second object obtained as described above, and the flow of FIG. 5 ends.

In the storage unit 5, the predetermined matrix H2 and the feature information of a plurality of targets as the corresponding second target candidates are registered for the first target, and in step S13, the plurality of candidates are the first. (Ii) It is possible to specify which of the two targets is applicable. That is, the second object registers a plurality of objects and obtains a recognition result from these, but the matrix H2 registers a common object for the plurality of registered objects in the storage unit 5.

6 and 7 show schematic examples to which the embodiment of FIG. 5 is applied. In FIG. 6, in the image Q1 in which the processing process M1 is applied to the captured image P in the embodiment of FIG. 5 and the upper surface F1 of the housing (the same housing as in the examples of FIGS. , The upper surface is recognized. The matrix H1 is obtained in the recognition result, and the side surface F2 as the second object is expanded / facing state by using the matrix H2 stored in advance corresponding to the recognized upper surface F1 (first object). The processed image Q20 = H2 * H1 (P) is obtained, and the side surface F2 is recognized with high accuracy as the second object in the processed image Q20.

In the example of FIG. 6 above, the first object and the second object are close to each other as surfaces that share sides with each other in the housing (that is, the upper surface F1 and the side surface F2 that share the sides and are adjacent to each other), but at all. Similarly, it is also possible to recognize the spatially separated first object and the second object by the embodiment of FIG. FIG. 7 is an example of the first object and the second object that are spatially separated, and the front wheel W1 of the vehicle (not shown) is the processed image WQ1 as the first object (note that in FIG. 7, a matrix that converts to a facing state). It is shown that H1 matches the matrix that obtains the processed image WQ1), and it is difficult to recognize because the size of the original captured image P is small and it is distorted to the rear side. This is an example in which the rear wheel W2, which was the above, can be recognized in the enlarged / facing processed image WQ2 = H2 * H1 (P).

For example, in the case of FIG. 7, the matrix H2 that converts the front wheel W1 from the magnified facing state to the rear wheel W2's magnified facing state is registered in the storage unit 5 as a fixed one, and the first Each type of the front wheel W1 as the target and the rear wheel W2 as the second target can be stored in the storage unit 5 as one of a plurality of candidates, and can be recognized by the information terminal device 10. .. Furthermore, it is also possible as an application example to perform high-precision superimposition display on the rear wheel W2 that is slightly distorted in the captured image P by superimposition by augmented reality, which is an existing technique.

As described above, according to the present invention, it is possible to generate a plurality of processed images by processing the captured image and to recognize the image pickup target in the captured image with high accuracy by using the feature information calculated from each. Become. Hereinafter, explanatory supplements in the present invention will be described.

(1) Various combinations are possible for each embodiment of the present invention. For example, the application of the second embodiment (planar projective conversion) is applied by using the processing information M201 when the best recognition result (maximum similarity score) is obtained in the recognition unit 4 in the first embodiment (resolution conversion). You may do it. In this case, if the captured image P output by the image pickup unit 1 and to be recognized in the second embodiment is replaced with the one to which the processing information M201 is applied, and then the second embodiment as described above is applied. good.

(2) Further, in the first embodiment (resolution conversion), it is assumed that the loop processing is not performed from the second time onward from step S5 to step S6 at the first time of k = 1, but k ≧ 2 as a modification. Subsequent iterative processing can also be performed for resolution conversion. In this case, the feature information f [k] is obtained as an integrated image of each resolution, but the integrated information is retained from which resolution conversion the integrated information is generated, and is aligned with RANSAC or the like. A tree structure similar to that described for the planar projective transformation using FIG. 4 so that the processing of the near resolution of the resolution containing the most characteristic information determined to be applied is applied in the next k + 1th iterative process. May be set in advance regarding the resolution conversion. If the feature information f [k] is integrated by averaging when the processed images of each resolution are integrated, by counting the weighting coefficient at the time of averaging, the in-liner is used in RANSAC or the like. The resolution that contains the most determined feature information may be determined.

(3) Further, in the second embodiment in which the planar projective transformation is repeatedly applied hierarchically in a tree shape, the integrated feature information f [k] as in the first embodiment may be used. In this case as well, as in (2) above, which explains that the resolution transformation is repeatedly applied, it is retained from which planar projective transformation each of the integrated feature information is generated (in the case of the average, its weight is used). (Hold), and the processing of the near-plane projective transformation of the planar projective transformation containing the largest amount of feature information determined to be inlier by RANSAC or the like may be applied in the next k + 1th iterative process.

(4) The information processing apparatus 10 can be realized as a computer having a general configuration. That is, a CPU (Central Processing Unit), a main storage device that provides a work area for the CPU, an auxiliary storage device that can be configured with a hard disk, SSD, etc., an input interface that receives input from users such as a keyboard, mouse, touch panel, etc., and a network. The information processing unit 10 can be configured by a general computer including a communication interface for connecting to and communicating with, a display for displaying, a camera, and a bus connecting them. Further, the processing of each part of the information processing apparatus 10 shown in FIG. 1 can be realized by a CPU that reads and executes a program for executing the processing, but any part of the processing can be performed by a separate dedicated circuit or the like ( It may be realized in (including GPU). The image pickup unit 1 can be realized by the camera as the hardware.

10 ... Information processing device, 1 ... Imaging unit, 2 ... Processing unit, 3 ... Calculation unit, 4 ... Recognition unit, 5 ... Storage unit

Claims (6)

A processing unit that obtains each processed image by performing multiple processing processes on the image to be imaged.
A calculation unit that calculates feature information from each of the processed images,
The calculated feature information and the feature information stored by being calculated from a plurality of images taken with the plurality of faces facing each other for the same object having a plurality of faces having different orientations. A recognition unit for recognizing which of a plurality of surfaces of the same object corresponds to the image pickup target in the image by collating with the image is provided.
The information processing unit is characterized in that the processed image is obtained by performing a conversion process by a plurality of planar projective transformations. The information processing apparatus according to claim 1, wherein the plurality of plane projection transformations to be performed include at least one that changes the direction of a plane. The recognition unit collates the feature information of each of the processed images with the feature information stored in advance for each of the predetermined plurality of recognition targets, and is similar between each processed image and each recognition target. The information processing apparatus according to claim 1 or 2, wherein the information processing apparatus according to claim 1 or 2 recognizes which of the predetermined plurality of recognition targets corresponds to the image pickup target in the image. The calculation unit calculates feature points and feature quantities from each of the processed images, and averages the feature quantities at the feature points whose coordinates correspond between the processed images, whereby the image integrated in the plurality of processed images. Calculate the feature information corresponding to
The information processing apparatus according to claim 1 or 2, wherein the recognition unit recognizes which of the above is applicable based on the feature information corresponding to the integrated image. The stored feature information, which is the object to be collated by the recognition unit, is characterized in that it is calculated from a plurality of images captured only when the plurality of surfaces face each other. The information processing apparatus according to any one of claims 1 to 4. A program characterized in that a computer functions as the information processing apparatus according to any one of claims 1 to 5 .

Images