JP2003288595A - Apparatus and method for object recognition, and computer-readable recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method of recognizing a three-dimensional object speedily and precisely by simple constitution. <P>SOLUTION: While coordinates of a standard image consisting of images of a plurality of parts of a collation object are converted by using standard pattern information consisting of the standard image and the mutual position relation among the plurality of parts, comparative collation with an input image is performed and the standard images share the same part of the specified object respectively. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object recognition apparatus and method, and a computer-readable recording medium, and in particular, it is obtained by photographing a three-dimensional shape.
The present invention relates to an object recognition apparatus and method for recognizing a three-dimensional shape from a three-dimensional image, and a computer-readable recording medium in which information for causing a computer to execute the method is recorded.

[0002]

2. Description of the Related Art Methods for detecting or recognizing a three-dimensional shape of an object include a method of preparing a three-dimensional shape as a model in advance and a method of preparing a two-dimensional grayscale image as a model. The former method is a method of first recovering a three-dimensional shape from an input image and matching it with a three-dimensional model, and a method of projecting the three-dimensional model on a two-dimensional image and matching it with a two-dimensional input image. There is.

On the other hand, the latter method is a method of approximating a relatively flat three-dimensional shape such as an operation panel as a recognition target object with a two-dimensional plane and performing a normalization limited to the deformation as a plane to perform collation. Is. For example, Japanese Patent Laid-Open No. 2000-901
No. 91 publication, the positional displacement of the object, the inclination, the vertical direction,
A technique is disclosed in which combinations of parameters based on affine transformation of planar deformation factors such as left and right directions are prepared in advance for all possible variations of deformation.

[0004]

The above-mentioned method of preparing a three-dimensional shape as a model in advance includes a method of restoring a three-dimensional shape from an input image, a method of creating a three-dimensional shape in advance from training data, and the like. In any of the methods, it is difficult to create an accurate three-dimensional model when the three-dimensional shape is complicated. On the other hand, the method of preparing the two-dimensional grayscale image as a model has a problem that the collation cannot be performed when the deformation of the target image exceeds the approximate range.

The present invention has been made in view of such circumstances, and does not require the processing of restoring a consistent three-dimensional model as in the above-mentioned conventional method, and has a certain degree of three-dimensional shape. It is an object of the present invention to provide an object recognition device and method capable of flexibly coping with dynamic deformation.

[0006]

In order to achieve the above object, according to the present invention, a standard pattern composed of a standard image composed of a plurality of whole or partial images of an object to be detected and a mutual positional relationship of the plurality of whole or parts. The information is used to compare and collate the standard image with the input image while converting the coordinates of the standard image, and the standard image includes the same portion of the detection target object in common.

As described above, standard images of a plurality of whole or portions which commonly include the same portion of the target object are prepared, and coordinate conversion is performed independently while taking into consideration the relative positional relationship between the plurality of whole or portions. Meanwhile, since the comparison and collation with the input image can be performed, it is possible to realize efficient object recognition by applying the coarse / fine method.

That is, according to the method of the present invention, when recognizing an object to be detected, first, a rough image of the whole is collated,
Next, a detailed comparison is performed with respect to a predetermined portion that represents the shape feature of the object. Further, at this time, when the whole image is collated, a three-dimensional robust collation can be performed by using a two-dimensional image having a low resolution, that is, a large permissible range for the deviation in the thickness / depth / depth direction. Quick and highly flexible matching is possible.

After the whole image is collated as described above, the characteristic portion can be collated in detail. That is, if the entire input image and the standard pattern can be matched by collating the entire image, it can be said that the characteristic portion should exist in which portion of the input image and at which inclination according to the relative positional relationship information with the characteristic portion. Therefore, it is possible to apply a relatively high-resolution two-dimensional image and perform detailed feature matching.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION A recognition method according to an embodiment of the present invention uses a plurality of two-dimensional plane regions in which a part or all of an object surface is considered to exist "approximately" in a virtual three-dimensional space. Is assumed and the position and the normal direction are hypothesized and tested, and the matching between the model grayscale image and the input image is advanced on the plane.

Further, in this embodiment, the attribute of "thickness" (depth) of the two-dimensional plane area is also taken into consideration. This "thickness" represents the "error tolerance for matching",
The lower the resolution of the model grayscale image is, the thicker it is, and the higher the resolution of the same grayscale image is, the thinner it is. That is, the lower the resolution of the model grayscale image, the more robust the matching is to the positional deviation of the feature points caused by the three-dimensional posture change.
That is, a substantially correct matching result can be obtained even if the feature points are displaced due to the three-dimensional posture change.

For example, consider the case of recognizing an object having a three-dimensional shape as shown in FIG. In this case
As shown in (B), when the reference (model) grayscale image has a low resolution, the error tolerance range (that is, the thickness) of the plane area includes the range in the thickness direction of the object, so that the collation is possible. That is, the plane approximation can be performed for the entire object, and the collation can be performed only by planar normalization. In the figure, squares indicated by broken lines indicate pixels of the standard grayscale image. However, in this case, since the grayscale image has a low resolution (that is, the number of pixels per unit area is small),
Only the features in the overall structure are collated, and the fine structure cannot be verified.

On the other hand, when the reference grayscale image has a high resolution as shown in FIG. 1 (C), the range in the thickness direction of the object deviates from the allowable error range of the plane area as it is. The correction processing, which has only a two-dimensional effect, cannot cope with it, and robust collation becomes impossible.

However, as shown in FIG. 1D, by dividing the plane area of the reference grayscale image into two plane areas having different inclinations and applying each to the input image, the reference grayscale image has a high resolution. Also makes it possible to include the range in the thickness direction of the target portion of the object in each plane area.

In the embodiment of the present invention, as shown in FIG. 2, the positional relationship between a plurality of plane areas and their resolutions, and information consisting of respective corresponding grayscale images are defined as a "standard pattern". Note that each grayscale image shown in FIG. 2B is preferably configured as an image viewed from an optimum viewpoint (a viewpoint in which the plane is viewed from the normal direction of the plane area, see FIG. 2A).

By performing the collation with the object of FIG. 1 (A) by using all the information of the standard pattern, it becomes possible to effectively collate both the entire structure and the fine structure of the object.

Here, the method according to FIG. 1B relates to the conventional method of approximating with the plane (two-dimensional model).
The method according to (D) is related to the conventional method of preparing the three-dimensional shape as a model. On the other hand, in the embodiment of the present invention, a method of effectively combining the elements of both methods is adopted after explicitly considering the error allowable range in the depth direction, that is, the “thickness of the approximate plane”.

In particular, by first performing the matching shown in FIG. 1 (B) to narrow down the target object candidates and then performing the matching shown in FIG. 1 (D), a coarse / fine search, that is, a search from a rough element to a search by a fine element. It is possible to apply the method that advances to. Therefore, the method according to the embodiment of the present invention can be said to be an effective method for detecting a specific object.

The feature of the standard pattern in the embodiment of the present invention is that the same part of the surface of the target object is duplicately expressed in a plurality of plane regions of the standard pattern as shown in FIG. 2 (B).

That is, in the embodiment of the present invention, the input image is compared with the standard pattern consisting of the grayscale images of a plurality of plane areas and the positional relationship between the plane areas while performing coordinate conversion of each of the plane areas. . Here, the standard pattern includes a plurality of planar regions in which the same portion of the object surface is expressed in an overlapping manner. Incidentally, the hypothesis testing algorithm in each of the embodiments described below is not limited to a specific projection model, and the central projection, parallel projection, weak central projection, pseudo central projection, etc. Any projective model that projects a planar area onto the image plane is applicable.

Further, the applicable range of the present invention is not limited to the object matching algorithm, but is applied when examining the degree of coincidence between the grayscale image and the input image such as the distance in the feature vector space and the correlation of pixel values. It can be applied to any algorithm.

Further, in the embodiment of the present invention, the method of setting the resolution of the plane area is not limited to a particular method. That is, as a method of lowering the resolution for the purpose of robust matching, not only the number of pixels in the grayscale image is reduced, but also a method using a Gaussian filter having a large spatial constant to reduce the spatial resolution as preprocessing at the time of matching. May be applied.

Further, in the embodiment of the present invention, at the time of the pattern matching, one or several plane regions may be brought into a state at a certain time, and the state transition may be made based on the result of the collation regarding the plane region ( Details later).

In the embodiment of the present invention, the input image may be a multi-viewpoint image.

Further, in the embodiment of the present invention, the camera parameter of the image pickup device for picking up the input image may be controlled based on the processing state of the pattern matching algorithm.

Further, according to an embodiment of the present invention, in a computer-readable recording medium, a plurality of plane areas in which the same portion of the object surface is duplicately expressed in a plurality of plane areas whose normal direction and resolution are different from each other. A mode in which a standard pattern composed of the grayscale image and the positional relationship between the plane areas is recorded is also included.

In the embodiment of the present invention, with the above-mentioned configuration, generally, a consistent process for restoring a complicated three-dimensional model is not required, and the three-dimensional deformation can be flexibly dealt with to some extent. A possible object recognition device and method can be realized. Further, in the embodiment of the present invention, it is possible to apply an operation that is generally considered to be applicable only when a method using a three-dimensional model is performed, such as an operation for subtracting a motion effect or a lighting effect.

Further, if the object recognition apparatus according to the embodiment of the present invention is used, as will be described later, it is possible to carry out integrated object recognition by using input images taken by a plurality of cameras. That is, for example, the embodiment of the present invention can be applied to a technique by binocular stereoscopic vision.

Further, by using the object recognition apparatus according to the embodiment of the present invention, it is possible to more efficiently recognize the object shape by re-photographing a more characteristic surface under better conditions during the matching process. .

The configuration of each embodiment of the present invention will be described in detail below.

First, an object recognition apparatus according to the first embodiment of the present invention will be described.

The present apparatus detects a "mouse" as a recognition target object from the photographed image and further recognizes whether it is "mouse A" or "mouse B". Specific examples of the three-dimensional shapes of the mouse A and the mouse B are shown in the upper part of FIG. The lower part of FIG. 3 shows a schematic view of these mice as seen from the side.

As shown in FIG. 3, mouse A has a relatively small angle between the characteristic region (a plane region including the mouse button) and the entire region, whereas mouse B has a smaller angle between the characteristic region and the entire region. The angle between them is relatively large. In this case, the entire area is a plane area obtained when the mouse is viewed from directly above.

FIG. 4 schematically shows a method of creating a mouse standard pattern according to the first embodiment of the present invention, and FIG. 5 shows a flowchart thereof. First, in step S1, images (a) and (b) of mouse A and mouse B are taken. In this case, all the images are taken from right above the mouse. Next, in step S2, several tens of characteristic points (in the figure, on the images (a) and (b) are visually observed by the operator.
Black circle) is set manually.

Then, assuming that all of these feature points are present in a single plane, registration is performed by geometric transformation so that the entire images of mouse A and mouse B are overlapped as much as possible. Then, the aligned grayscale images are averaged to create a grayscale image (e) on the "average entire area" (step S3).

That is, the average position between the corresponding feature points of both of the figures (c) and (d) is obtained, and based on the average position of the feature points thus obtained, the geometrical transformation (c ) And (d), the average grayscale image (e) between the images is obtained. Here, for example, the projection model of each image is considered to be a weak center projection, and the optimum affine transformation coefficient that makes the feature points on the mouse A and the mouse B as close as possible to the average position of the feature points of the mouse A and the mouse B is the least squares. It is determined by the approximation criterion and is used as a geometrical transformation.

On the other hand, in regions (f) and (g) having many features,
An affine transformation coefficient that brings only the feature points closer to the average feature point position is obtained, and similarly, the grayscale image (h) on the "average feature area" is obtained (step S4). However, this average feature area (h) can be assumed to be a flat area having a smaller thickness than the average overall area (e), that is, it can be assumed that the allowable error range in the thickness direction can be made smaller. Create a grayscale image.

Here, the reason why the affine transformation coefficient is calculated independently for each of the plane regions (e) and (h) is that the thickness of the plane region, which is considered to include each feature point, is different, and the mouse is used. This is because it is considered that the same portion of is included in different plane regions having different normal directions depending on the scale.

Further, one affine transformation coefficient corresponding to the correspondence between the average feature point coordinates on the average overall region (e) and the average feature point coordinates on the average feature region (h) is obtained. Then, at the time of collation processing described later, the affine transformation using this coefficient is used as information indicating the relative positional relationship between both regions. As described above, the “standard pattern” is obtained by combining the grayscale images (e) and (h) and the relative positional relationship between them.

Further, for each of mouse A and mouse B (between (c) and (f), and (d) and (g)), the respective features projected onto the entire region where the alignment is performed. It is desirable to obtain individual affine transformation coefficients from the correspondence between the points and the characteristic points projected onto the aligned characteristic region. The information obtained in this way is used when the detected mouse is mouse A
Can be used to identify whether it is a mouse or a mouse B.

Next, according to the first embodiment of the present invention described above,
A matching algorithm in the object recognition device for actually matching the input image pattern will be described. FIG. 6 is a diagram schematically showing the algorithm, and FIG. 7 is a flowchart showing a procedure according to the algorithm.

Here, the matching by the pattern matching algorithm is performed by the perturbation method. That is, the hypothesis testing process is advanced while gradually changing the position of each plane area and the normal direction. First, from the standard pattern obtained as described above, the three-dimensional position of each surface and the slight change in the inclination of the entire plane area (e) and the feature plane area (h) are independently addressed. While gradually changing the affine parameter (affine transformation coefficient),
The corresponding grayscale image is affine transformed and compared with the whole or a part of the input image obtained by photographing the target object each time. The comparison method in this case is performed by obtaining the correlation value between the density values of both grayscale images.

Then, it is determined that the affine parameter applied when the correlation value obtained here is the largest is a good parameter, and if the correlation value is sufficiently large, the standard pattern of the position and inclination in that case is determined. It is determined that there is a high probability that the plane of the target object represented by the affine parameter that matches the plane region exists. As described above, the candidate position having a high existence probability is obtained for each of the plane regions (e) and (h) (steps S11 and S12).

Finally, the positional relationship validity judgment is performed (step S13). That is, a combination satisfying the relative positional relationship (between (e) and (h), see FIG. 4) of the standard pattern is obtained from the candidates of the respective plane areas (e) and (h), and this is calculated. Output as the collation result.

Specifically, as shown in FIG. 8, for each candidate of the entire area, affine transformation coefficients for all corresponding characteristic area candidates are calculated, and this is the average entire area calculated in step S5 of FIG. If it is sufficiently close to the affine transformation coefficient between the average feature area and the average feature area (between (e) and (h)), the combination is selected as a proper detection result. At this time, all the affine transformation coefficients must be appropriate when the combination is judged to be valid. If only the parallel displacement components of the affine transformation coefficients disagree, it is determined that the position is inappropriate, and if only the components other than the parallel displacement component disagree, it is determined that the inclination is inappropriate, and both are valid. It will be excluded without being determined to be a bad one (see FIG. 8).

When the object identification processing is also carried out after the object detection processing, the affine transformation coefficients obtained at the time of detection are the individual relative positional relationships stored at the time of creating the standard pattern ((c) in FIG. 4). , (F), and (d), (g))
The identification is carried out depending on which of the affine transform coefficients that represents

Next explained is an object recognition device according to the second embodiment of the invention. Similar to the first embodiment in the case of the present embodiment, it is an apparatus for detecting a specific "mouse". However,
The input image to be handled is directly above the target mouse (viewpoint 1)
The detection is not limited to the image captured from, but the image captured from diagonally in front of the mouse (viewpoint 2) is also used for detection. That is, it is configured as a device for robustly detecting a posture change from a certain viewpoint to another viewpoint, that is, a device that can obtain an accurate detection result even if such a posture change occurs.

FIG. 9 shows a method of creating a standard pattern in this embodiment. First, in the same manner as in the case of the first embodiment described with reference to FIG. 4, the standard pattern for viewpoint 1 ((a) and (b) in FIG. 4) and the standard pattern for viewpoint 2 ((c) and (d) in FIG. 4). )) And create separately. Next, the average feature areas (b) and (c), which are flat areas common to both patterns, are integrated into one area (f) by the following method.

First, alignment is performed so that the corresponding feature points of the feature areas (b) and (c) of both viewpoints come as close as possible, and then the average value of the coordinates of the corresponding feature points is calculated, and the average value is obtained. An affine transformation is obtained in which the coordinates of the corresponding feature points are closest to each other. The images obtained by applying the affine transformation thus obtained to the respective grayscale images (b) and (c) are further averaged, and the grayscale images thus obtained are
The grayscale image (f) of the integrated average feature area is used.

Further, each of the average overall region 1 (e) (same as (a)) and the average overall region 2 (g) (same as (d)) to the integrated average feature region (f), respectively. The affine transformation of is calculated and added as relative positional relationship information to the information of the entire standard pattern.

As described above, the grayscale images of the average entire area 1 (e), the average entire area 2 (g) and the average characteristic area (f), respectively,
The affine transformation from the average overall region 1 (e) to the average feature region (f) and the affine transformation coefficient from the average overall region 2 (g) to the average feature region (f) are defined as the information of the entire standard pattern.

Next, FIG. 10 shows a matching algorithm in the object recognition apparatus according to the second embodiment. The difference from the first embodiment is that the perturbation method is sequentially applied to each plane area. That is, when an existence candidate of a certain plane area is found, it is predicted where the other plane area is, assuming that the plane area exists, and it is sequentially perturbed by the conversion parameter in the vicinity of the predicted position while sequentially perturbing the conversion parameter. Match.

That is, in the first embodiment, a method of obtaining candidates for the entire area from the beginning, further obtaining candidates for the characteristic area, and then selecting candidates having an appropriate positional relationship from the candidates. On the other hand, in the second embodiment, as shown in FIG.
As will be described later with reference to 1, since the processing state is transited from the “whole region” state to the “feature region” state, the position of the feature region can be estimated from the collation result regarding the whole region in the “feature region” state. Efficient collation processing can be executed. Thereby, the processing cost can be effectively saved.

A specific processing flow according to the algorithm of the second embodiment will be described with reference to the flowchart of FIG. First, an arbitrary area is cut out from the captured image and set as an inspection area, and specifically, the corresponding parameter is set (step S21). This inspection area is for hypothesis / testing whether or not the area substantially matches the entire area to be collated.

Then, the whole area of the viewpoint 1 (entire area 1,
9 (e)) and the whole area of the viewpoint 2 (whole area 2, FIG.
For both (g)), presence candidates are searched for while perturbing parameters other than the parameters (parameters related to position and scaling) set as the inspection area in step S21 (step S22). That is, the image cut out as the inspection region is slightly affine-transformed under the above conditions, and the average entire regions (e) and (g) are compared and collated each time. In the second embodiment, since the plane area to be collated is considered as the processing state, the state at this time has two plane areas, the whole area 1 (e) and the whole area 2 (g). become.

Then, when one of the candidates ((e) or (g) in FIG. 9) is found (Y in step S23).
es), the position of the characteristic region (f) is estimated from the relative positional relationship between the found entire region (e) or (g) and the characteristic region (f) (preliminarily obtained as the information of the standard pattern). Then, (step S24 or S25), the affine parameter is perturbed in the vicinity of the position to search for the corresponding portion of the characteristic region (step S26). At this time, the processing state has transitioned to the characteristic region. Then, when a portion that matches the characteristic region is certainly found (Yes in step S27)
Outputs the result (step S28) and completes the process. That is, the result output in this case is that "the captured image matches the matching target mouse".

If no candidates for the entire area or the characteristic area are found (No in step S23 or No in step S29), the inspection area in the photographed image is changed to
The whole area is searched for again (from step S22 onward). However, in step S29, if the parameters of the characteristic region can be changed (Yes), that is, if there is room for affine transformation for the characteristic region (f),
The affine parameter is changed (step S30) and the search after step S26 is repeated.

With this configuration, since the search is performed based on the "coarse / dense method" in which the general information having a low resolution is moved to the detailed information having a high resolution, an efficient search can be executed.

Next explained is an object recognition device according to the third embodiment of the invention. In this example, the camera for viewpoint 1 (camera 1) and the camera for viewpoint 2 (camera 2) are used to simultaneously shoot the target object from two directions, and a multi-viewpoint image obtained from both cameras is used as an input image.

FIG. 12 shows a method of creating a standard pattern according to the third embodiment. The difference from the second embodiment is that with respect to the characteristic region that is a common region, only the positional relationship (the position of the characteristic region with respect to the entire region) is integrated, and the grayscale images ((f) and (g) in the same figure) are individualized. It is to leave it as it is.
By doing so, it is possible to apply the effect of the three-dimensional method even when the same region looks like a significantly different pattern depending on the viewpoint.

The matching process for detection according to the third embodiment is performed as shown in FIG. First, the entire area is searched in parallel from the images respectively obtained from both cameras (steps S41 and S42). Specifically, each is shown in FIG.
The same processing as steps S21 and S22 in step S21 is performed.
As a result, when it is determined that the target object exists at the same three-dimensional position in consideration of the positional relationship between both cameras (step S43).
Yes), the process for each image obtained from both cameras transits to the search state of the characteristic region (step S44).

Then, the characteristic region (see FIG. 12) in the three-dimensional space
The planes (f) and (g) in the middle are perturbed while changing their positions and normal vectors, and the existence of the characteristic region is tested while assuming a common plane between the processes for both cameras. (Step S45). That is, it is determined whether or not the image area which is regarded as the characteristic area in the inspection area of the captured image substantially matches the characteristic area ((f) or (g) in FIG. 12) to be collated. Then, if a high correlation value is obtained by the processing for both cameras when a certain plane is assumed, it is determined that the plane of the characteristic region exists at that position, and the object (that is, the mouse to be collated) is surely also present. It is determined that it exists there (Yes in step S46).

Incidentally, steps S47, S48, S49, S
The respective processes of 50 are steps S28 and S in FIG.
This is the same as the processing of S29, S30, and S31, and the duplicated description is omitted.

In normal surface-based stereo vision, first, the corresponding surface is searched between multi-viewpoint images. However, in the case of this method, when the object to be collated is such that the appearance of the surface is considerably different when the viewpoint changes. It is difficult to find a correspondence relationship with. On the other hand, in the method of the present embodiment, since only the photographed input image obtained from each viewpoint and the standard pattern are compared, there is no need to perform collation between input images of multiple viewpoints, and it is relatively easy. It is possible to verify the hypothesis of a plane in a three-dimensional space.

Next explained is an object recognition device according to the fourth embodiment of the invention. FIG. 14 is a diagram for explaining the fourth embodiment, and FIG. 15 is a flowchart showing the flow of the processing. The difference from the third embodiment is that a camera for characteristic region (camera 3) C3 is used in addition to a camera for viewpoint 1 (camera 1) C1 and a camera for viewpoint 2 (camera 2) C2. The characteristic region camera C3 is controlled by a predetermined control signal to determine its three-dimensional position, pan during photographing,
The tilt and zoom parameters can be freely set.

The flow of the matching process for object detection differs from that of the third embodiment in that, when hypothesis-testing the plane of the characteristic region, the assumed surface is photographed in the best state. The photographed image is input again each time while adjusting the above-mentioned imaging parameters of the camera 3 (C3) (steps S65, S66, S67, S69, S7).
0). This best condition is a condition in which the normal line of the feature area plane candidate of the target object and the direction of the optical axis of the camera are matched as much as possible, and the candidate area is zoomed so that it can be captured on the entire screen. Is.

Incidentally, other steps S61, S62, S6
The processes of S3, S64, S68, and S71 are the same as steps S41, S42, S43, S44, S47, and S in FIG.
The processing is the same as that of 50, and duplicate description will be omitted.

By applying the so-called active vision method to the camera C3 as described above, the camera C for viewpoint 1
Since the 1st and viewpoint 2 cameras C2 are used only for photographing the respective entire areas, relatively low resolution cameras can be applied.

Next, a fifth embodiment of the present invention will be described. This example is an example applied to tracking a stationary object in a moving image captured while moving a camera that captures the target object. The process flow of this embodiment is shown in FIG.

Detection of target object (steps S81, S8
2) is performed for each frame in the moving image by the same method as the above-described second embodiment (processing of FIG. 11), but for the third frame and after (step S83 and after), the past 2
Inferring the movement of the object from the frame (step S84),
The collation processing can be effectively performed using the information. That is, based on the past positional changes and rotation-related motions of the target plane area obtained by the matching process, a position where the plane area is likely to exist is predicted (step S85), and a search is made in the vicinity thereof (step S86). Therefore, the processing time can be shortened.

Further, as shown in FIG. 17, when it is better to change the whole area used for matching depending on the viewpoint, the optimum whole area is appropriately determined according to the inclination of the surface predicted at that time to carry out the matching. It is desirable to do. Note that the entire regions 1 and 2 in FIG. 17 correspond to the average entire regions (e) and (h) shown in FIG. 12, respectively.

That is, the photographing camera is momentarily moved with respect to the object to be inspected (mouse in this case), and the photographing parameters (the pan, tilt, and
Shooting the target object continuously while automatically adjusting (zooming parameters) (shooting by tracking method)
The images of the respective frames along the time axis of the moving image taken in this case are (a), (b), (c), and (d) of FIG.
In this case, for each of the frames (a) and (b),
The whole area 1 is suitable for matching because they are images from substantially the front of the target object, and the frames (c), (d)
For each of the above, since the camera is rotated with respect to the target object, and the images are from an oblique direction, the whole region 2 is suitable. These judgments can be performed by predicting the inclination of the reference plane (the plane seen when shooting from directly above) of the target object based on the object recognition results obtained from the image captured at each time and the past image. .

(E), (f), (g), (h) of FIG.
Correspond to (a), (b), (c), and (d) of the same drawing, respectively, and correspond to the optimum entire application area determined from the inclination of the reference plane of the target object predicted as described above. The shapes of the characteristic regions, and further, the regions adapted to the input image by the matching process and appropriately affine transformed are shown.

The object recognition apparatus of each of the above-described embodiments of the present invention can be realized by connecting a digital still camera, a video camera or the like 110 to a general-purpose computer 120 such as a personal computer as shown in FIG. . That is, the information of the respective standard patterns shown in FIGS. 4, 9, and 12 created in advance is stored in a storage device such as a hard disk device of the computer, and further, FIG. 6, FIG. 7, FIG. The matching algorithms shown in FIGS. 11, 13, 15, and 16 are stored as software programs. Then, the matching algorithm of each embodiment can be realized by executing the software program using the standard pattern information. However, the mechanism 130 for automatically controlling the shooting parameters of the camera in the fourth and fifth embodiments can be realized by utilizing the technology of separately known camera vision system and tracking system camera control mechanism. .

The present invention includes the configurations described in the following supplementary notes. (Supplementary Note 1) A device for recognizing a predetermined object,
Comparison with the input image while converting the coordinates of each standard image using standard pattern information consisting of a plurality of whole or partial images of the predetermined object and mutual positional relationship of the plurality of whole or portions An object recognizing device configured to perform collation, and the standard image commonly includes the same portion of the predetermined object.

(Supplementary Note 2) The object recognition apparatus according to Supplementary Note 1, wherein the comparison and collation means has at least one standard image as a state, and the state changes according to the result of the comparison and collation.

(Supplementary note 3) The object recognition apparatus according to supplementary note 1 or 2, wherein the input image is a multi-viewpoint image.

(Supplementary Note 4) The object recognition according to any one of Supplementary Notes 1 to 3, further comprising means for controlling photographing parameters when the input image is obtained by photographing based on the comparison and collation result of the comparison and collation means. apparatus.

(Supplementary Note 5) A method for recognizing a predetermined object, which comprises a standard image composed of a plurality of whole or partial images of the predetermined object and a mutual positional relationship of the plurality of whole or parts. A method comprising the step of comparing and collating the standard image with the input image while converting the coordinates of the standard image using the standard pattern information, and the standard image commonly includes the same portion of the predetermined object.

(Supplementary Note 6) A computer-readable recording medium in which information for causing a computer to execute a process for recognizing a predetermined object is recorded. Consisting of the standard image and the mutual positional relationship of the plurality or all of them,
Standard pattern information for performing comparison and collation with an input image while converting the coordinates of the standard image using the information, and the standard image has a standard configuration including the same portion of the predetermined object in common. A computer-readable recording medium on which pattern information is recorded.

(Supplementary Note 7) A software program for causing a computer to execute processing for recognizing a predetermined object, including a standard image composed of a plurality of whole or partial images of the predetermined object and the plurality of whole images. Alternatively, the computer is configured to perform a step of performing comparison and collation with the input image while converting the coordinates of the standard image using standard pattern information consisting of mutual positional relationships of the parts, and the standard image is the predetermined image. A software program that is configured to include the same parts of an object in common.

[0082]

As described above, according to the present invention, it is possible to provide a method capable of promptly and accurately recognizing a three-dimensional object having a comparatively rich undulation with a relatively simple structure.

[Brief description of drawings]

FIG. 1 is a diagram (No. 1) for explaining the principle of the present invention.

FIG. 2 is a diagram (No. 2) for explaining the principle of the present invention.

FIG. 3 is a diagram for describing an example of a recognition target object used for explaining the function of the embodiment of the present invention.

FIG. 4 is a schematic diagram for explaining a standard pattern creating method according to the first embodiment of the present invention.

FIG. 5 is a flowchart for explaining a standard pattern creating method according to the first embodiment of the present invention.

FIG. 6 is a schematic diagram for explaining an object matching algorithm according to the first embodiment of the present invention.

FIG. 7 is a flowchart illustrating an object matching algorithm according to the first exemplary embodiment of the present invention.

FIG. 8 is a schematic diagram for explaining the validity determination processing of the positional relationship in the object matching algorithm according to the first embodiment of the present invention.

FIG. 9 is a schematic diagram for explaining a standard pattern creating method according to a second embodiment of the present invention.

FIG. 10 is a schematic diagram for explaining an object matching algorithm according to a second embodiment of the present invention.

FIG. 11 is a flowchart illustrating an object matching algorithm according to a second exemplary embodiment of the present invention.

FIG. 12 is a schematic diagram for explaining an object matching algorithm according to a third embodiment of the present invention.

FIG. 13 is a flowchart illustrating an object matching algorithm according to a third exemplary embodiment of the present invention.

FIG. 14 is a diagram for explaining the fourth embodiment of the present invention.

FIG. 15 is a flowchart illustrating an object matching algorithm according to a fourth exemplary embodiment of the present invention.

FIG. 16 is a flowchart illustrating an object matching algorithm according to a fifth exemplary embodiment of the present invention.

FIG. 17 is a diagram for explaining an object matching algorithm to which the tracking method according to the fifth embodiment of the present invention is applied.

FIG. 18 is a diagram showing a configuration example capable of realizing each embodiment of the present invention.

[Explanation of symbols]

100 object recognition device 110 cameras 120 personal computer 130 camera control mechanism

   ─────────────────────────────────────────────────── ─── Continued front page    F term (reference) 2F065 AA53 BB05 CC00 FF04 JJ03                       JJ05 JJ26 QQ31                 5B057 AA20 CA13 CD11 DA12 DB03                       DC33                 5L096 AA09 BA20 FA76 HA07 JA18

Claims (5)

[Claims] 1. A device for recognizing a predetermined object, comprising a standard image consisting of a plurality of whole or partial images of the predetermined object and a standard pattern consisting of a mutual positional relationship between the plurality of whole or parts. An object recognition apparatus comprising means for comparing and collating the standard image with the input image while transforming the standard image using information, and the standard image commonly includes the same portion of the predetermined object. 2. The object recognition device according to claim 1, wherein the input image is a multi-viewpoint image. 3. The object recognizing device according to claim 1, further comprising means for controlling photographing parameters when the input image is obtained by photographing based on the comparison and collation result of the comparison and collation means. 4. A method for recognizing a predetermined object, the standard pattern comprising a standard image composed of a plurality of whole or partial images of the predetermined object and a mutual positional relationship of the plurality of whole or parts. A method comprising the step of comparing and collating the standard image with the input image while transforming the coordinates of the standard image using information, wherein the standard image commonly includes the same portion of the predetermined object. 5. A computer-readable recording medium in which information for causing a computer to execute processing for recognizing a predetermined object is recorded, and a standard image composed of a plurality of whole or partial images of the predetermined object. It consists of the mutual positional relationship of all or a plurality of them, and is information for performing comparison and collation with the input image while converting the coordinates of the standard image using the information, and the standard image is the predetermined object. A computer-readable recording medium in which standard pattern information having a configuration that includes the same portion in common is recorded.