JP2008171147A - Object recognition device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress incorrect recognition of an attitude of a recognition object resulting from discrepancy of the center of a recognition object and the center of a scanning window. <P>SOLUTION: An object recognition device 1 has: a template storage section 16 which stores a template dictionary which describes a feature vector of a plurality of rotation template images for collating with an input image and the feature vector of a plurality of translation template images; a feature vector calculation section 13 which calculates the feature vector of a partial image partially chosen from an input image by the scanning window as a processing target area; and an estimation section (template matching section 15 and result integrated section 17) which estimates a location and an attitude of the recognition object included in the input image based on the template dictionary and the feature vector of the partial image. Further, a plurality of rotation template images are an aggregate of images looked at a recognition object from a plurality of directions, and a plurality of translation template images is an assembly of images which have caught a recognition object partially from a plurality of views along a straight line trajectory. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an object recognition apparatus and an object recognition method for estimating the position and orientation of a recognition target included in an input image by collating the input image with a plurality of template images.

  In computer vision, which is responsible for the vision of a robot that moves autonomously based on its own judgment, it is necessary to be able to recognize the position and orientation of an object from an image taken by a camera. As a technique used for such object recognition, a plurality of template images corresponding to how a recognition target object is viewed from multiple directions are stored in advance, and an input image obtained by photographing is collated with a plurality of template images. There is known a method for estimating the posture of a recognition object depending on which template image an input image is similar to. Hereinafter, data storing a plurality of template images used for object recognition is referred to as a template dictionary.

  The collation between the input image and the template dictionary is performed by collating the feature amount extracted from the input image with the feature amount of the template image included in the template dictionary. Here, the feature amount is obtained by extracting essential components useful for image identification from information included in the input image and the template image. Usually, the feature amount is described as a feature vector having elements of a plurality of feature amount values. The number of dimensions of the feature vector corresponds to the number of feature quantities. Further, each of the input image and the template image is expressed as a point on a multidimensional feature space stretched by feature vectors.

  The simplest feature vector is a feature vector having the values of each pixel constituting the image data and the template image as elements. However, in this case, the number of dimensions of the feature vector reaches the number of pixels constituting the image data, and the number thereof is enormous. Therefore, the storage capacity for storing the template image and the time required for image matching become enormous, which is not realistic.

  Therefore, an eigenspace method is known in which a template dictionary can be stored with a small storage capacity by compressing the number of dimensions of a feature vector, and image matching can be performed at high speed. The eigenspace method generates the eigenspace by principal component analysis (PCA: Principal Components Analysis) of the template image, and generates the feature vector of the template image using the principal components obtained by projecting the template image onto the eigenspace. To do.

  Further, a parametric eigenspace method in which the eigenspace method described above is applied to estimation of the position and orientation of an object has been proposed (see Non-Patent Document 1 and Patent Document 1). In the parametric eigenspace method, a template image captured continuously while rotating a recognition object is learned in advance. Hereinafter, a template image obtained by photographing an object from various directions is referred to as a rotated template image. A set of rotation template images is represented as a manifold on the eigenspace (hereinafter referred to as a rotation manifold). The posture of the recognition target object is estimated by obtaining a projection point of the template image that is closest to the projection point obtained by projecting the input image onto the eigenspace.

Further, the position of the recognition object in the input image is estimated as follows. That is, the above-described parametric eigenspace method is applied to the partial image selected by the scanning window while setting a scanning window for partially selecting a part of the input image and sequentially scanning the input image by the scanning window. Estimate the pose of the object. Then, the position of the recognition object in the input image is estimated based on the position of the scanning window where the posture of the recognition object is estimated.
JP 2005-149167 A Murase Hiroshi, SKNayar, "3D object recognition by 2D collation-parametric eigenspace method", IEICE theory, J77-D-II, No.11, pp.2179-2187, November 1994

  As described above, the conventional object recognition method based on matching with a template image generates a template dictionary composed of a rotated template image obtained by photographing a recognition target object while rotating it at the learning stage. Further, in the object recognition stage, the position and orientation of the object are estimated by collating a partial image selected from a part of the input image by the scanning window with the rotation template image included in the template dictionary.

  However, in such a conventional object recognition method, when a part of the recognition target object is included in the partial image selected by the scanning window, more specifically, the center of the recognized recognition target object is recorded. When the center of the partial image selected by the scanning window is deviated, there is a problem that the posture estimation accuracy is lowered because the posture of the recognition target object is easily misrecognized. In the following, the shift between the center of the recognition object and the center of the scanning window is expressed as “translational shift”.

  The present invention has been made in consideration of the above-described problem, and suppresses misrecognition of the posture of the recognition object due to a shift (translational shift) between the center of the recognition object and the center of the scanning window. Objective.

  An object recognition apparatus according to a first aspect of the present invention is an apparatus that estimates the position and orientation of a recognition object using an input image that is captured and input by an imaging unit, and for collating with the input image. A storage unit for storing a template dictionary described with respect to a feature vector of a plurality of rotation template images and a feature vector of a plurality of translation template images, and a scanning window that scans the input image and designates a processing target region. A feature vector calculation unit that calculates a feature vector indicating a feature of a partial image that is partially selected as the processing target region from the input image, and based on the template dictionary and the feature vector of the partial image, And an estimation unit that estimates the position and orientation of the included recognition target object. Further, the plurality of rotation template images are a set of images obtained by viewing the recognition target object from a plurality of directions, and the plurality of translation template images are obtained by partially imaging the recognition target object from a plurality of viewpoints along a linear trajectory. It is a set of images captured as a target. In the first embodiment of the invention to be described later, the template matching unit 15 and the result integration unit 17 correspond to an estimation unit included in the object recognition apparatus according to the first aspect.

  Here, the rotation template image is a set of images obtained by viewing the recognition target object from a plurality of viewpoints positioned on a circular trajectory centered on the recognition target object, and the translation template image is positioned on a linear trajectory. It is good also as a set of images which caught the above-mentioned recognition subject partially from a plurality of viewpoints.

  Further, the plurality of rotation template images are cut out so as to include the entire recognition target object from images photographed including the recognition target object from a plurality of viewpoints along a circular orbit centered on the recognition target object. The plurality of translation template images may be a set of images cut out so as to include only a part of the recognition target object from the image photographed including the recognition target object.

  With such a configuration, it is possible to collate the variation characteristic that the feature vector of the partial image cut out from the input image shows with the movement of the scanning window and the variation characteristic that the feature vector of the translation template image shows with translational deviation. it can. Therefore, even if a misjudgment of a posture due to translational deviation occurs in the collation with the rotation template image, the translation manifold of the rotation template image misjudged in the collation with the translation template image and the evaluation target Since a mismatch occurs between the translation manifolds of the partial images, the posture of the recognition target object that is erroneously estimated can be rejected. Thereby, the misrecognition of the attitude | position of the recognition target object resulting from a translation shift can be suppressed.

  In the object recognition apparatus according to the first aspect of the present invention described above, the plurality of translation template images are images obtained by photographing the recognition object with the center of the photographed image shifted from each of the plurality of rotation template images. You may comprise so that a group may be included.

  Further, the template matching unit has a distance in a feature space between a rotated manifold created by the feature vectors of the plurality of rotated template images and a feature vector of the partial image smaller than a first threshold, and the first A distance in the feature space between a translation manifold created by a translation template image corresponding to a rotation template image that gives a distance from the rotation manifold smaller than a threshold value of 1 and a feature vector of the partial image is a second threshold value. In the case of being smaller, the posture specified by the rotation template image that gives the distance from the rotating manifold smaller than the first threshold value may be estimated as the posture of the recognition target object. Thereby, the misrecognition of the attitude | position of the recognition target object resulting from translational displacement can be suppressed efficiently.

  An object recognition method according to a second aspect of the present invention includes a process of scanning an input image by a scanning window that defines a processing target area, and the scanning window is partially selected from the input image as the processing target area. A feature vector indicating the feature of the partial image, a plurality of rotation template image feature vectors that are a set of images obtained by viewing the recognition target object from a plurality of directions, and a plurality of viewpoints along a linear trajectory The template image described with respect to the feature vectors of a plurality of translation template images, which is a set of images partially capturing the recognition object, is compared with the feature vectors of the partial images, and photographed in the input image And a process for estimating the position and orientation of the recognized recognition object.

  By such a method, it is possible to collate the variation characteristic that the feature vector of the partial image cut out from the input image shows with the movement of the scanning window and the variation characteristic that the feature vector of the translation template image shows with translational deviation. it can. Therefore, even if a misjudgment of a posture due to translational deviation occurs in the collation with the rotation template image, the translation manifold of the rotation template image misjudged in the collation with the translation template image and the evaluation target Since a mismatch occurs between the translation manifolds of the partial images, the posture of the recognition target object that is erroneously estimated can be rejected. Thereby, the misrecognition of the attitude | position of the recognition target object resulting from a translation shift can be suppressed.

  Further, in the posture estimation of the recognition target object in the object recognition method according to the second aspect, in the feature space between the rotation manifold created by the feature vectors of the plurality of rotation template images and the feature vectors of the partial images. It is determined whether or not the distance is smaller than the first threshold, and corresponds to a rotation template image that gives a distance from the rotating manifold smaller than the first threshold when it is determined that the distance is smaller than the first threshold. Determining whether the distance in the feature space between the translation manifold created by the translation template image and the feature vector of the partial image is less than a second threshold, and is determined to be less than the second threshold. The posture identified by the rotation template image that gives a distance from the rotating manifold that is smaller than the first threshold is the recognition target. Of it may be used as the attitude estimation results. As a result, the amount of calculation required for collation between the partial image and the translation template image can be reduced, and object recognition can be performed efficiently.

  According to the present invention, it is possible to provide an object recognition apparatus and an object recognition method capable of suppressing erroneous recognition of the posture of a recognition target object due to a shift (translational shift) between the center of the recognition target object and the center of the scanning window.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. In the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted as necessary for the sake of clarity.

Embodiment 1 of the Invention
A configuration of an object recognition apparatus 1 according to the present embodiment is shown in FIG. The object recognition device 1 is a device that outputs an estimation result of the position and orientation of a recognition target object included in an input image. More specifically, the object recognition apparatus 1 sequentially scans input images in units of scanning windows, and collates partial images selected by the scanning windows with a plurality of template images representing various postures of the recognition target object. The position and orientation of the recognition object included in the input image are estimated. Below, with reference to FIG. 1, each component which the object recognition apparatus 1 has is demonstrated.

  In FIG. 1, an image pickup unit 11 is an image pickup device (not shown) such as a CCD (Charge Coupled Device), and a signal processing unit (not shown) that converts analog image data obtained by the image pickup device into digital image data and outputs it. ).

  The image feature calculation unit 12 performs image processing on the input image input from the imaging unit 11 and calculates the feature amount of the input image. More specifically, the image feature calculation unit 12 calculates a feature amount in units of partial images selected by the scanning window from the input image. The feature amount can be calculated, for example, by converting input image data into grayscale image data and performing a differentiation process on the converted grayscale image data to obtain edge strength and edge direction as the feature amount. Further, the normalized luminance value of each pixel of the input image data may be used as the feature amount. Note that the feature amount extraction method in the image feature calculation unit 12 is not limited to the above-described method, and various methods including a conventionally known method can be applied. The p feature quantities (p is a positive integer) calculated from the input image (specifically, the partial image selected by the scanning window) is represented as a p-dimensional feature vector D.

  The feature vector calculation unit 13 dimensionally compresses the feature vector D calculated by the image feature calculation unit 12 using the basis vector group E stored in advance in the basis vector storage unit 14. The number of dimensions of the feature vector D dimensionally compressed by the feature vector calculation unit 13 is n (n is a positive integer, n <p). The basis vector group E can be obtained, for example, by applying a conventionally known eigenspace method. Specifically, a set of eigenvectors determined by performing principal component analysis (PCA) on a plurality of template images (specifically, rotation template images and translation template images described later) used for matching with the input image And it is sufficient. At this time, the basis vector group E can be expressed by the following expression (1) as a set of n vectors. In addition, the feature vector D of the dimensionally compressed partial image is shown in Equation (2).

  The template matching unit 15 estimates the position and orientation of the recognition object captured in the input image by matching the partial image selected from the input image with the scanning window and a plurality of template images. Specifically, the template image that minimizes the distance s in the feature space with the feature vector D of the partial image is selected from the feature vectors of the plurality of template images stored in the template storage unit 16, and the selected template is selected. The posture of the recognition object corresponding to the image is estimated. Further, the position of the recognition object in the input image is estimated based on the position of the partial image for which the posture estimation has been performed. If the minimum value of the distance s is larger than the predetermined threshold, the template matching unit 15 does not have a template image close to the partial image to be evaluated, that is, the recognition target object is included in the input image. It is determined that there is no image and the partial image is rejected.

  The template matching unit 15 repeatedly performs the above-described arithmetic processing for estimating the posture of the recognition target object for a plurality of partial images cut out by the scanning window from the input image captured by the imaging unit 11. In this embodiment, as shown in FIG. 2, a method of sequentially scanning the scanning window 22 in the horizontal direction with respect to the input image 21, that is, a so-called raster scan, performs a series of processes from selection of a partial image to template matching. Shall be repeated. Of course, the scanning method of the input image by the scanning window is not limited to the method shown in FIG. For example, scanning using a scanning window may be performed along the vertical direction of the input image 21.

The template storage unit 16 holds a set {T _i ^j } of feature vectors related to the template image of the recognition object. Note that the template storage unit 16 in the present embodiment is characterized by holding a set of feature vectors of a rotation template image and a feature vector set of a translation template image. Here, as described above, the rotation template image is a template image obtained by shooting while rotating the recognition target object. In other words, the rotation template image group is a set of images obtained by viewing the recognition target object from a plurality of directions. On the other hand, the translation template image is a template image obtained by continuously capturing images in a state where the center of the captured image is shifted from the center of the recognition object in accordance with the scanning direction of the scanning window. In other words, the translation template image group is a set of images in which the recognition target is partially captured from a plurality of viewpoints along the straight trajectory. That is, by holding the translation template image in the template storage unit 16, the object recognition apparatus 1 can learn how the recognition target object looks when a “translational shift” occurs during the recognition process.

An example of the rotation template image and translation template image of the recognition object is shown in FIG. FIG. 3A shows an example of the recognition object 23 in which the letter “A” is attached to the surface of the cylinder. FIG. 3B shows four rotation template images T ₁ ⁰ , T ₂ ⁰ , T ₃ ⁰ , and T ₄ ⁰ obtained by photographing the recognition object 23 from four directions that are different from each other by 90 degrees. . To be specific, rotate the template image T ₁ ⁰ is an image taken from a position facing the letter A of the recognition target object 23 is attached portion. Rotation template image T ₂ ⁰ is a photographed image of the recognition target object 23 as compared with the rotating template image T ₁ ⁰ while being rotated by 90 degrees clockwise. Rotation template image T ₃ ⁰ is a photographed image of the recognition target object 23 as compared with the rotating template image T ₂ ⁰ in a state of being rotated by 90 degrees clockwise. The rotation template image T ₄ ⁰ is a photographed image of the recognition target object 23 as compared with the rotating template image T ₃ ⁰ in a state of being rotated by 90 degrees clockwise.

FIG. 3 (c) shows a series of translational template images taken in relation to the rotating template image T ₁ ^0. In the set of translation template images {T ₁ ^-2 , T ₁ ^-1 , T ₁ ⁰ , T ₁ ¹ , T ₁ ² } in FIG. 3 (c), in addition to the rotated template image T ₁ ⁰ , accordance scanning direction, includes images obtained by photographing the recognition subject 23 by shifting the center of the captured image compared to rotate the template image T ₁ ⁰ is.

  As described above, all the rotation template images and all the translation template images may be obtained by directly capturing the recognition target object from a plurality of different viewpoints. It may be generated by image processing. That is, the rotation template image group may be generated as a set of images obtained by viewing the recognition target object from a plurality of viewpoints located on a circular orbit centered on the recognition target object. What is necessary is just to produce | generate as a collection of the image which caught the recognition target object partially from several viewpoints located in.

  In addition, the rotation template image group is an image that is cut out so as to include the entire recognition target object from images taken including the recognition target object from a plurality of viewpoints along a circular orbit centered on the recognition target object. May be generated as a set of The translation template image group may be generated as a set of images cut out so as to include only a portion of the recognition object from an image captured including the recognition object.

  The feature vector of the template image held in the template storage unit 16 is generalized by the following equation (3). In equation (3), m is the total number of rotation templates. q is a parameter that defines the number of translation templates photographed in a state of translational deviation with respect to the rotation template.

  Returning to FIG. 1, the result integration unit 17 integrates the estimation results of the position and orientation of the object by the template matching unit 15 and outputs a final estimation result. Specifically, when there is only one image in which the position and orientation of the recognition target object are estimated by the template matching unit 15 among the plurality of partial images selected by scanning the input image with the scanning window. The position of the partial image may be set as the final estimated position of the recognition target object, and the posture estimated by the template matching unit 15 may be output as the final estimation result. On the other hand, when there are a plurality of images in which the position and orientation of the object are estimated by the template matching unit 15, the plurality of candidates may be integrated by weighting as shown in the following equation (4).

  Equation (4) is obtained by weighted integration of two candidates C1 = (x1, y1, θ1) and C2 = (x2, y2, θ2), and the final object position and orientation estimation result C3 = (x3, y3, In this case, θ3) is obtained. In the equation (4), x1 and y1 represent position coordinates in the input image of the first candidate C1, and θ1 represents a posture parameter representing the posture of the first candidate C1. Similarly, x2 and y2 represent position coordinates in the input image of the second candidate C2, and θ2 represents a posture parameter representing the posture of the second candidate C2. Further, x3 and y3 represent position coordinates in the input image of the estimation result C, and θ3 represents a posture parameter representing the posture of the estimation result C.

  Further, w1 and w2 represent weights when the first candidate C1 and the second candidate C2 are integrated, and these are functions of the distances s1 and s2 from the template image calculated by the template matching unit 15 Determine as. Note that s1 represents the minimum distance from the template image when the first candidate C1 is estimated, and s2 represents the minimum distance from the template image when the second candidate C2 is estimated. The weights w1 and w2 can be variously determined, but the simplest example is a linear approximation.

  Next, object recognition processing performed by the object recognition apparatus 1 according to the present embodiment will be described in detail with reference to the flowchart of FIG. FIG. 4 is a flowchart illustrating an object recognition processing procedure performed by the object recognition apparatus 1. In step S <b> 101, captured image data is input from the imaging unit 11 to the image feature calculation unit 12. In step S102, the image feature calculation unit 12 calculates a feature amount from the partial image selected by the scanning window from the input image. In step S103, the feature vector calculation unit 13 calculates the feature vector of the input image projected on the n-dimensional feature space, specifically, the feature vector D of the partial image selected by the scanning window.

Subsequently, in step S104, the template matching unit 15 sets the minimum distance L _sc between the rotated manifold in the feature space created by the feature vectors {T _i ⁰ } of the plurality of rotated template images and the feature vector D of the input image. calculate. Specifically, the distance s _c between the feature vector of the rotated template image and the feature vector D of the input image may be calculated for each of the rotated template images, and the minimum value of the distance s _c may be set as the minimum distance L _sc. . The distance s _c can be calculated by the following equation (5), and the minimum distance L _{sc at} this time is represented by the equation (6). Here, M is a reliability weight, and Conf represents a rotating manifold.

In step S105, the template matching unit 15 determines whether or not the minimum distance L _sc calculated in step S104 is equal to or smaller than a predetermined first threshold Th1. As a result of the determination, if the minimum distance L _sc is larger than the first threshold Th1, the scanning window is moved to the next pixel, and the processes after step S102 are repeated (step S111). On the other hand, when it is determined that the minimum distance L _sc is equal to or less than the first threshold Th1, the processes after step S106 are performed.

In step S106, the template matching unit 15 calculates the translation manifolds rotating the template image providing a minimum distance L _sc, the minimum distance L _st of the translational manifolds partial image which is currently being evaluated. Here, the translation manifold of the rotation template image refers to a feature vector group {T _i ^j } of a series of translation template images photographed in association with the rotation template image as shown in FIG. It is a manifold obtained by projecting to. The translation manifold of partial images includes a feature vector of a partial image that is a current evaluation target, and a feature vector group of a plurality of partial images selected by a scanning window before and after the partial image that is a current evaluation target. A manifold obtained by projecting onto a feature space.

  FIG. 5 is a conceptual diagram illustrating an example of a feature space related to a rotation template image and a translation template image. The rotation manifold 300 shows a manifold formed by a plurality of rotation template images. The translation manifold 311 indicates a manifold formed by a series of translation template images photographed in association with a certain rotation template image 31. The translation manifold 321 indicates a manifold formed by the partial image 32 that is the current evaluation target and a plurality of partial images selected by the scanning window before and after the partial image 32.

The minimum distance _Lst can be calculated by the following equation (7). Here, M is a reliability weight, and Trans is a translation manifold of the rotated template image.

In step S107, the template matching unit 15 determines whether or not the minimum distance _Lst calculated in step S106 is equal to or smaller than a predetermined second threshold Th2. As a result of the determination, when the minimum distance L _sc is larger than the second threshold Th2, the scanning window is moved to the next pixel, and the processes after step S102 are repeated (step S111). On the other hand, when it is determined that the minimum distance L _sc is equal to or smaller than the second threshold Th2, the processes after step S108 are performed.

  In step S108, the position and orientation of the recognition target specified by the partial image that is the current evaluation target are selected as candidate values. If it is determined in step S109 that scanning by the scanning window has not been completed for the entire input image, the scanning window is moved to the next pixel, and the processing from step S102 onward is repeated (step S111). On the other hand, when the entire input image has been scanned, the result integration unit 17 integrates the position and orientation candidate values estimated by different scanning windows, and outputs the estimation result of the position and orientation of the recognition target object. To do.

  The process flow of the object recognition process illustrated in FIG. 4 is an example. The processing flow of FIG. 4 performs the matching between the partial image and the translation template image only when the partial image and the rotation template image are collated and the feature vector of the rotation template image close to the feature vector of the partial image is searched. It was decided to do. However, the collation step between the partial image and the translation template image may be performed immediately without performing the collation step between the partial image and the rotation template image without relying on such a two-stage determination process. However, since the amount of calculation increases when collation between the partial image and the translation template image is performed in all cases, the translation is performed only when the collation result with the rotation template image is satisfactory by the two-stage search as described above. It is desirable to check against the template.

  As described above, the object recognition apparatus 1 according to the present embodiment includes the center of the captured image and the center of the recognition object in addition to the feature vector of the rotated template image obtained by photographing the recognition object while rotating. Are stored in the template storage unit 16 as feature vectors of translation template images obtained by photographing in a state where a shift occurs in the scanning direction of the scanning window. In other words, the object recognition apparatus 1 learns in advance how the recognition object looks when there is a deviation (translational deviation) between the center of the recognition object and the center of the scanning window by holding the translation template image. .

  Further, as described in steps S106 to S108 in FIG. 4, the object recognition apparatus 1 calculates the distance in the feature space between the translation manifold of the rotated template image and the translation manifold of the partial image that is the current evaluation target. The calculated distance is compared with a predetermined threshold Th2. Thereby, the variation characteristic indicated by the feature vector of the partial image cut out from the input image with the movement of the scanning window is collated with the variation characteristic indicated by the feature vector of the translation template image with the translational deviation. Therefore, even if a misjudgment of a posture due to translational deviation occurs in the collation with the rotation template image, the translation manifold of the rotation template image misjudged in the collation with the translation template image and the evaluation target Since a mismatch occurs between the translation manifolds of the partial images, the posture of the recognition target object that is erroneously estimated can be rejected. Thereby, the misrecognition of the attitude | position of the recognition target object resulting from a translation shift can be suppressed.

  In the object recognition apparatus 1 according to the present embodiment described above, other components other than the imaging unit 11 can be configured using a computer system having a CPU (Central Processing Unit) that can execute a program. It is. Specifically, the image feature calculation unit 12, the feature vector calculation unit 13, the template matching unit 15, and the result integration unit 17 are stored in a storage unit such as a ROM or flash memory that the CPU is provided inside or connected to the outside of the CPU. A program for causing the computer system to execute the processing shown in the flowchart of FIG. 4 is stored, and the program is executed by the CPU. The basis vector storage unit 14 and the template storage unit 16 may be non-volatile storage units such as a hard disk and a flash memory included in the computer system.

  The program for causing the computer system to execute the processing shown in the flowchart of FIG. 4 can be stored in various types of storage media, not limited to ROM or flash memory, and is transmitted via a communication medium. It is possible. Here, the storage medium includes, for example, a flexible disk, a hard disk, a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD, a RAM memory cartridge with battery backup, and the like. The communication medium includes a wired communication medium such as a telephone line, a wireless communication medium such as a microwave line, and the Internet.

Other embodiments.
In the object recognition apparatus 1 according to the first exemplary embodiment of the invention, the result integration unit 17 performs weighted integration of the object position and orientation estimation results obtained by a plurality of scanning windows. However, without performing such weighted integration, a position and orientation with the smallest minimum distance _Lst may be simply selected as a final estimation result among a plurality of position and orientation candidates. Further, the object recognition apparatus 1 does not necessarily include the imaging unit 11, and may input image data stored in the memory after shooting to the image feature calculation unit 12.

Also, the object recognition process performed by the object recognition device 1 according to the first embodiment of the invention, specifically, at step S104 in the flowchart of FIG. 4, the object distance s _c correspond to rotate the template image as the minimum Searching for posture. However, the method of identifying the posture of the recognition target used in step S104 is not limited to the method of selecting the rotation template image with the minimum distance, the so-called nearest neighbor determination rule (NN method). For example, a rotating manifold is analytically expressed by interpolating between projected points in a feature space indicating a plurality of rotated template images by polynomial interpolation or spline interpolation, and the rotation manifold and the feature vector of the partial image are expressed. The shortest distance and the projection point that gives the shortest distance may be calculated analytically.

  Further, the template storage unit 16 may not store the feature vector of the template image itself. For example, the feature vector of the template image may be converted into another parameter for specifying the posture of the recognition target included in the input image, and the converted parameter may be stored in the template storage unit 16. Also, conversion information such as a projection matrix for converting the feature vector of the input image into another parameter for specifying the posture of the recognition target included in the input image may be stored in the template storage unit 16.

  Furthermore, the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention described above.

It is a block diagram of the object recognition apparatus concerning embodiment of this invention. It is a conceptual diagram for demonstrating the scanning of the input image by a scanning window. It is a figure which shows an example of the template image which the object recognition apparatus concerning embodiment of this invention hold | maintains. It is a flowchart which shows the procedure of the recognition process of the recognition target object by the object recognition apparatus concerning embodiment of this invention. It is a figure which shows an example of feature space.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Object recognition apparatus 11 Image pick-up part 12 Image feature calculation part 13 Feature vector calculation part 14 Basis vector calculation part 15 Template collation part 16 Template memory | storage part 17 Result integration part 21 Input image 22 Scan window 23 Recognition object 31 Rotation template image 32 Input Image 300 rotation manifold 311 translation manifold 321 of rotation template image translation manifold of input image

Claims (9)

An object recognition device that estimates the position and orientation of a recognition target object using an input image that is captured and input by an imaging unit,
A storage unit for storing a template dictionary described with respect to a feature vector of a plurality of rotation template images and a feature vector of a plurality of translation template images for matching with the input image;
A feature vector calculation unit that calculates a feature vector indicating a feature of a partial image that is partially selected as the processing target region from the input image by a scanning window that is scanned on the input image and specifies the processing target region;
An estimation unit that estimates the position and orientation of a recognition object included in the input image based on the template dictionary and the feature vector of the partial image;
The plurality of rotation template images are a set of images obtained by viewing the recognition object from a plurality of directions,
The plurality of translation template images is an object recognition device that is a set of images obtained by partially capturing the recognition target object from a plurality of viewpoints along a linear trajectory. The rotation template image is a set of images obtained by viewing the recognition object from a plurality of viewpoints located on a circular orbit centered on the recognition object.
The object recognition apparatus according to claim 1, wherein the translation template image is a set of images obtained by partially capturing the recognition target object from a plurality of viewpoints located on a linear trajectory. The plurality of rotation template images are cut out so as to include the entirety of the recognition target object from images photographed including the recognition target object from a plurality of viewpoints along a circular orbit centered on the recognition target object. A collection of images,
The object recognition apparatus according to claim 1, wherein the plurality of translation template images are a set of images cut out so as to include only a portion of the recognition target object from an image captured including the recognition target object.   The plurality of translation template images include an image group obtained by photographing the recognition object by shifting a center of the photographed image and a center of the recognition object as compared with each of the plurality of rotation template images. 4. The object recognition apparatus according to any one of 3.   The template matching unit is configured such that a distance in a feature space between a rotated manifold created by the feature vectors of the plurality of rotated template images and a feature vector of the partial image is smaller than a first threshold, and the first The distance in the feature space between the translation manifold created by the translation template image corresponding to the rotation template image that gives a distance to the rotation manifold that is less than a threshold and the feature vector of the partial image is less than a second threshold The object recognition apparatus according to claim 4, wherein a posture specified by a rotation template image that gives a distance from the rotation manifold smaller than the first threshold is estimated as a posture of the recognition target object. Scanning the input image by a scanning window that defines a processing target area, and calculating a feature vector indicating characteristics of a partial image partially selected from the input image as the processing target area by the scanning window;
A feature vector of a plurality of rotation template images that are a set of images obtained by viewing the recognition target object from a plurality of directions, and a set of images obtained by partially capturing the recognition target object from a plurality of viewpoints along a linear trajectory. Estimating a position and orientation of a recognition object photographed in the input image by comparing a template dictionary described with respect to a feature vector of a plurality of translation template images and a feature vector of the partial image; Object recognition method.   The object recognition method according to claim 6, wherein the plurality of translation template images include an image group obtained by photographing the recognition target object with a center of the photographed image shifted from each of the plurality of rotation template images.   The rotation variety in which the distance in the feature space between the rotation manifold created by the feature vectors of the plurality of rotation template images and the feature vector of the partial image is smaller than the first threshold and smaller than the first threshold. The distance in the feature space between the translation manifold created by the translation template image corresponding to the rotation template image that gives the distance to the body and the feature vector of the partial image is less than a second threshold. The object recognition method according to claim 7, wherein a posture specified by a rotation template image that gives a distance from the rotation manifold that is smaller than a threshold is a posture estimation result of the recognition target object. Determining whether the distance in the feature space between the rotation manifold created by the feature vectors of the plurality of rotation template images and the feature vector of the partial image is less than a first threshold;
A translation manifold created by a translation template image corresponding to a rotation template image that gives a distance from the rotation manifold that is less than the first threshold when determined to be less than the first threshold; Determining whether a distance in the feature space to a feature vector is less than a second threshold;
The posture specified by the rotation template image that gives a distance from the rotating manifold that is smaller than the first threshold when determined to be smaller than the second threshold is set as the posture estimation result of the recognition object. Item 8. The object recognition method according to Item 7.

Images