JPH05157518A - Object recognizing apparatus - Google Patents

Abstract

PURPOSE:To improve the recognizing accuracy of the position/posture of an object. CONSTITUTION:There are provided a characteristic part extracting means 20 for extracting a characteristic part of an object in an edge image, a first-dimensional position/posture operating means 30 for obtaining the first-dimensional position/posture of the object in the real space based on the position where the characteristic part is present on the image surface, and retrieving area setting means 50, 60 for movably setting the retrieving area. The setting means 50, 60 imagine an outline of the object on the image surface on the basis of the first-dimensional position/posture, and sets the retrieving area along the outline on the image surface which has a predetermined length along the outline in a small width centering the position where the outline exists. Moreover, there are provided a precise edge extracting means 70 and a position/ posture operating means 80. The precise edge extracting means 70 re-processes the dark-and-light image for every retrieving area set movably along the outline, extracts a correct edge of the image of the object, thereby to extract a precise edge corresponding to the outline of the whole object. The position/posture operating means 80 operates the position/posture of the object in the real space from the existing position of the precise edge on the image surface.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object recognizing device for accurately obtaining the position / orientation of an object in real space from an image obtained by picking up an image of the object.

[0002]

2. Description of the Related Art As a method for recognizing the position / orientation of an object in a real space, a projection method (Japanese Patent Laid-Open No. 62-71806),
Stereoscopic method (Binocular: JP-A-62-284479, motion vision:
JP-A-63-142469). In addition, a monocular method using a single TV camera as an image pickup device (Japanese Patent Laid-Open No. 62-22638).
No. 9 bulletin) exists.

[0003]

However, in the projection method,
Object recognition at a long distance is difficult, and high-speed processing cannot be executed. Further, in the stereoscopic method, it is necessary to associate a plurality of images. However, since it is difficult to associate objects in a complicated background, it takes time to associate them. Furthermore, the floodlight method,
In both the stereoscopic method, there is a problem that the image pickup device and the processing device are complicated.

In the monocular method, the position of the object to be recognized
It can be applied when the change in posture is limited to a two-dimensional plane or when the lighting conditions are limited, but the position and posture of the object changes three-dimensionally, and the size and shape of the object in the captured image are different. However, when the lighting condition changes and the contrast between the object and the background changes, there is a problem that the object cannot be recognized accurately.

The present invention has been made to solve the above problems, and an object thereof is that in monocular vision, it seems that the illumination condition changes or the contrast with respect to the background changes depending on the position of the contour of the recognition object. Even if
By accurately extracting the contour of the entire recognition object, the position / orientation of the object in the real space can be accurately recognized.

[0006]

As shown in FIG. 20, the structure of the present invention for solving the above-mentioned problems is a grayscale image capturing means, an edge image generating means, and a relative relative to the entire contour of an object in an edge image. A characteristic part extracting means for extracting a characteristic part of an object having a known relationship, and a primary position / orientation calculating means for obtaining a primary position / orientation in the real space of the object based on the existing position of the characteristic part on the image plane. , Based on the primary position / orientation of the object, the contour of the object is hypothesized on the image plane, and a search area is imaged with a minute width centered on the position of the contour and having a predetermined length along the contour. A search area setting means for movably setting along a contour on a surface, and an accurate object image by reprocessing a grayscale image or a differential image of a grayscale image for each search area movably set along a contour Can be extracted By this, there are provided a precision edge extracting means for extracting a precision edge corresponding to the contour of the entire object, and a position / orientation computing means for computing the position / orientation of the object in the real space from the existing position of the precision edge on the image plane. That is.

[0007]

A grayscale image is obtained by picking up an object, and an edge image is obtained by binarizing a differential image of the grayscale image. The feature of the object is extracted from the edge image.
Since the relative relationship between the characteristic portion and the contour of the entire object is known, the primary position / orientation of the object in the real space can be calculated based on the existing position of the characteristic portion on the image plane.

Also, the primary position of the object in the real space
From the posture, the existence position of the contour of the object on the image plane can be specified. The existence position of this contour on the image surface has an error due to a conversion error, a feature part extraction error, and the like. Therefore, by considering these errors, the existing position of the contour of the object is specified as the error width region. For one existing position in the area of the error width, a search area having a minute width and a predetermined length centering on the contour is movably set on the screen.

Within the movably set search area, the grayscale image or its differential image is reprocessed. At this time, since image processing can be limited to a narrow search area,
Even if there is a location dependence of the contrast between the contrast and the background with respect to the brightness of the entire object, the amount thereof can be made constant within the search area. Therefore, for example, in the edge extraction processing, it is possible to determine the edge emphasis filter for differentiation and the threshold value for binarization for each search area. As a result, accurate edge detection is possible. It is possible to accurately recognize the position / orientation of the object in the real space from the existing positions of these precision edges on the image.

[0010]

According to the present invention, the existing position of the feature portion of the object on the image plane is obtained, the primary position / orientation of the object in the real space is obtained from the information, and based on the primary position / orientation. , The contour of the object is mapped on the image plane, a search area of a small area that is movably set around the existence position of the contour is set, and a grayscale image or its for each search area that moves along the contour. This is a device that reprocesses the differential image to detect an edge, and recognizes the position / orientation of the object in the real space based on the existing position of the edge on the image plane. Therefore, in the present invention, since it is possible to change the conditions for reprocessing the grayscale image or its differential image for each search region, there is a light-dark distribution and a distribution between the background and the object image depending on the position. Also, the edge detection will be performed accurately. As a result, the position / orientation of the object in the real space is obtained based on the accurate position of the edge on the image plane, and thus the recognized position / orientation becomes more accurate.

[0011]

EXAMPLES The present invention will be described below based on a specific example. FIG. 1 shows the overall configuration of the apparatus according to this embodiment, and FIGS. 2 to 5 show the detailed configuration of each apparatus shown in FIG. 1) Configuration of Example Device 1. Grayscale Image Capturing Device The grayscale image capturing device 10 is a device that captures an image of the object A and generates a grayscale image. The apparatus 10 includes a television camera 11 that captures an image of an object A, an A / D conversion circuit 12 that samples a video signal output from the television camera 11 for each pixel and converts the image signal into a digital value, and an image that stores the digital value for each pixel. And a memory 13.

2. Characteristic Part Extraction Device The characteristic part extraction device 20 obtains a differential image of a grayscale image stored in the image memory 13, generates an edge image obtained by binarizing the image with respect to a predetermined threshold value, and generates an object from the edge image. This is a device for extracting a predetermined characteristic part in the image A.

The apparatus 20 is an edge enhancement circuit 2 for scanning a grayscale image and obtaining a differential image by a Sobel operator.
1. An image memory 22 for storing a differential image, a binarization circuit 23 for binarizing the differential image with respect to a predetermined threshold, an image memory 24 for storing an edge image obtained by the binarization, an edge A labeling circuit 25 that divides the image into continuous lines or portions that are considered to be continuous; and a characteristic portion determination circuit 26 that determines whether or not each portion grouped by labeling is a characteristic portion. It is composed of.

3. Characteristic Position / Posture Calculator The characteristic position / posture calculator 30 calculates the position / posture of the characteristic part of the object on the image plane, and based on the position / posture of this characteristic part, the primary position / posture of the object in the real space. Is a device for calculating. The characteristic part position / orientation calculation device 30 includes a characteristic part position calculation circuit 31 for obtaining the detected position of the detected characteristic part on the image plane, a characteristic part size calculation circuit 32 for calculating the size of the characteristic part, and a characteristic part size calculation circuit 32. A size prediction error calculation circuit 33 that calculates a prediction error, a characteristic part inertial spindle direction calculation circuit 34 that calculates the inertial axis direction of a characteristic part, and a spindle direction prediction error calculation circuit 35 that calculates a prediction error in the spindle direction. There is.

4. Model storage device The model storage device 40 stores a relative position memory 41 that stores the relative position of each vertex of the object with respect to the center position of the feature portion, a three-dimensional model of the recognition target object, that is, whether the line segments connecting the vertices of the object are straight lines The object shape memory 42 stores data that can specify the contour shape such as an arc.

5. Model Mapping Device The model mapping device 50 is a device that maps the model on the image plane using the primary position / orientation of the object and predicts the existing position of the image of the object on the image plane. The model mapping device 50 maps the three-dimensional model onto the image plane from the relative position between the center of gravity of the feature and each vertex of the three-dimensional model and the camera parameters, and calculates the position of each vertex on the image plane. A coordinate calculation circuit 51, a size calibration vertex coordinate changing circuit 52 for changing the position of each vertex on the image plane of the three-dimensional model in consideration of the detected dimensions of the characteristic portion, a three-dimensional model considering the rotation angle of the object. Rotation calibration vertex coordinate changing circuit 53 for changing the position of each vertex on the image plane of the above, a vertex coordinate memory 55 for storing the coordinates of each vertex on the image plane of the three-dimensional model, and a vertex for calculating the error of each vertex coordinate. It comprises a coordinate error calculation circuit 54 and a coordinate error memory 56 for storing the possible range of each vertex coordinate.

6. Search Area Setting Device The search area setting device 60 predicts the existence area of the edge of the contour of the object from the mapping of the three-dimensional model of the object onto the image plane,
This is a device for setting a large number of search areas having a small area in the existing area. The search area setting device 60 is a vertex candidate calculation circuit 61 that calculates a vertex coordinate candidate from the vertex coordinates and coordinate errors on the image plane of the three-dimensional model, and a search area position that determines a position to set a search area between the vertex candidates. A determination circuit 62, a search area position memory 63 that stores the position coordinates thereof, an approximate line segment calculation circuit 64 that obtains an approximate line segment between vertex candidates, and the approximate line segment is moved in parallel in correspondence with the existence area of the vertex coordinates. It is composed of a line segment translation circuit 65 for generating a plurality of line segments, and a search region shape memory 66 for determining the width, length, shape, etc. of the search region.

7. Precision Edge Extraction Device The precision edge extraction device 70 is a device that reprocesses the grayscale image for each search area to detect the exact position of the edge of the contour of the object. The precision edge extraction device 70 is an edge enhancement filter selection circuit 71 that selects an edge enhancement filter for reprocessing the grayscale image in the search area, and an edge enhancement circuit 72 that calculates a differential image from the grayscale image by the selected edge enhancement filter. , An image memory 73 for storing a differential image, a threshold value determining circuit 74 for determining a threshold value for binarizing the differential image, and a binarizing circuit 75 for binarizing the differential image based on the threshold value. The image memory 76 stores a binarized image, that is, a precision edge image.

8. Position / Posture Calculator The position / posture calculator 80 calculates the precise position of each vertex on the image plane of the object from the precision edge image, and the precise position information of each vertex calculates the position of the object in the real space of the object. This is a circuit that calculates the posture. Position / orientation calculator 8
In the precision edge image, 0 is an edge line connection circuit 81 that connects edge lines in order to classify each portion that can be regarded as a continuous line, and the outline of the object from the connected edge lines 1
Edge line approximation circuit 82 for obtaining two approximate straight lines, a vertex coordinate calculation circuit 83 for calculating a precise position on the image plane of the vertex of the object from the coordinates of the intersections of the approximate straight lines, and coordinates in the real space from the coordinates on the image plane. A coordinate conversion table memory 84 storing a conversion matrix for calculating a, a vertex three-dimensional position calculation circuit 85 for calculating the position coordinates of each vertex of the object in the real space, and a posture of the object in the real space from the position coordinates of the vertices And a posture calculation circuit 86 that operates.

2) Operation of the apparatus of this embodiment Next, the operation of the apparatus of this embodiment will be described. 6 and 7 show the processing procedure of this apparatus. 1. Generation of Grayscale Image In step 100, the zoom magnification (lens parameter) of the television camera 11 is controlled according to the distance between the object A and the television camera 11 so that the size of the object A on the image plane is optimized. In step 102, the object A
The position / orientation of the television camera 11 is controlled according to the relative positional relationship between the object A and the television camera 11 so as to be optimal for capturing the image.

In step 104, the television camera 11 is driven and the object A is photographed. Then, the level of the video signal output from the television camera 11 is sampled for each pixel on the image surface and digitized by the A / D conversion circuit 12. This digital value is stored in the image memory 13 in correspondence with the pixel address. In this way, the digitized grayscale image is stored in the image memory 13.

2. In the feature extraction step 106, the grayscale image stored in the image memory 13 is differentiated by the edge enhancement circuit 27 to generate a differential image (also referred to as “edge enhanced image”). This differential image can be obtained by scanning the image plane and applying a Sobel operator to the grayscale image. This differential image is stored in the image memory 22. Step 1
At 08, the binarization circuit 23 binarizes the differential image stored in the image memory 22 with a predetermined threshold value as a reference. This binarized image is stored in the image memory 24 as an edge image.

In step 110, the labeling circuit 25
Thus, in the edge image, edges are collected for each figure that seems to be continuous, and labeling is performed for each set. Such clustering can be performed by a search for edge continuity that is normally performed. In step 112, the characteristic part is extracted from the edge image labeled by the characteristic part determination circuit 26.

The object recognition apparatus of the present invention is, for example, a multi-layered pallet as shown in FIG.
It is used to recognize the existing position of the pallet and the insertion position of the fork when the unstacked forklift truck is used to take out and convey the (stacked). The installation position of the object A in the height direction is known. The characteristic portion is defined by the portions W1 and W2 into which the fork is inserted. This feature W
Since 1 and W2 have a strong contrast in the grayscale image, the edge appears relatively clearly in the edge image as compared with other portions of the object A. Therefore, this characteristic part W
1 and W2 are easy to recognize.

The characteristic portions W1 and W2 are, as shown in FIG.
On the image plane, each surface of the image of the object A appears as a rectangle or a parallelogram depending on the angle between the object A and the television camera 11. Using the ratio of the vertical width and the horizontal width when the feature appears in a rectangle, the ratio of the vertical width and the horizontal width of the parallelogram of the feature that appears on the image surface when the object A is imaged from a typical oblique direction, as a reference value. Calculate in advance. Then, as shown in FIG. 10, a parallelogram is extracted from the labeled figure,
The horizontal width Hi and the vertical width Vi of the parallelogram are detected and the ratio H
i / Vi is calculated, and those having a ratio within a predetermined error range with respect to the reference value are determined as characteristic portions W1 and W2.
It should be noted that various images of the characteristic portion that are deformed according to the shooting distance and angle may be stored as a model, and the characteristic portion may be determined by collating the labeled figure with the model.

3. Calculation of primary position / azimuth In step 114, as shown in FIG. 11, coordinates (U1, V1), (U2, V2) on the image plane of the respective centers of gravity G1, G2 of the detected feature portions W1, W2. Is calculated by the characteristic portion position calculation circuit 31. The position of the center of gravity is obtained as the intersection of the diagonal lines of the parallelogram. The height (Y component) of the center of gravity of this feature in real space is known.

In step 116, as shown in FIG. 11, a spindle unit vector D _M = (D _{M, U,} D _{M, V} ) parallel to the inertial spindle M of the feature portion by the feature portion inertial spindle direction arithmetic circuit 34.
Is calculated. This principal axis of inertia is the same as the axis defined in the mechanics, and the U axis of the feature figure on the image plane,
An inertial ellipse is calculated by calculating a moment of inertia around the V-axis, a product of inertias about the U-axis and the V-axis, and is obtained as a main axis of the ellipse.

In step 118, the size of the characteristic portion is calculated by the characteristic portion size calculation circuit 32. As shown in FIG. 11, parallelograms H1 and H2 circumscribing the characteristic portions W1 and W2 along the principal axis of inertia M on the image plane are obtained, and their width and height are set as the characteristic portion size.

In step 120, the main-axis direction prediction error calculation circuit 35 and the size prediction error calculation circuit 33 calculate the direction prediction error and the size prediction error. In the main-axis direction prediction error calculation circuit 35, the unit vector D _G parallel to the line segment connecting the centers of gravity G1 and G2

[Formula 1] D _G = (U2-U1, V2-V1) / {(U2-U1) ² + (V2-V1) ² } ^1/2 displacement vector ΔD _GM = with respect to the spindle unit vector D _M
The directional prediction error is calculated by D _G -D _M. In the size prediction error calculation circuit 33, the variation in the orientation of the parallelogram is obtained from the direction prediction error ΔD _GM , and the size estimation errors ΔHi and ΔVi are calculated from the variation in the orientation.
The direction prediction error may be obtained from the ratio between the major axis and the minor axis of the inertial ellipsoid.

In step 122, the vertex coordinate calculation circuit 5
1, the coordinates of the centers of gravity G1 and G2 on the image plane (U
1, V1), (U2, V2), using camera parameters (conversion matrix), the coordinates (X1, Y1, Z) of the centers of gravity G1, G2 in the real space
1), (X2, Y2, Z2) is calculated. At this time, TV camera 1
The distance and height between 1 and the object A, the magnification of the image, the imaging angle, etc. are also considered as camera parameters. It is known that the coordinates (X, Y, Z) in the real space and the coordinates (U, V) on the image plane can be mutually converted by using the camera parameter according to the Y coordinate in the real space. At this time, the Y components Y1 and Y2 of the centers of gravity G1 and G2 of the characteristic portions W1 and W2
Is known (Y1 = Y2 in this embodiment), and a camera parameter corresponding to this Y component is used. The rotation calibration vertex coordinate changing circuit 53 performs coordinate transformation of the main-axis unit vector D _M to obtain a posture vector D _R = (D _RX, D _RY, D in real space.
_RZ ) is calculated. The posture of the object A in this embodiment is
Assuming that the object A is placed on a horizontal plane (a plane parallel to the XZ plane), only the rotation angle around the Y axis is unknown. Therefore, D _RY = 0.

As shown in FIG. 12, the relative position memory 41 stores, when the object A takes a normal posture (the posture vector D _R
= (0,0,1)) midpoint G0 = (X
0, Y0, Z0) = ( (X1 + X2) / 2, Y1, (Z1 + Z2) / 2 position vector D ₁ of the a) to each vertex of the object A to be the _{origin, D 2, D 3, D} 4 _, D _5, D
_6, D _7, D ₈ are stored. The coordinates of each vertex in the real space are determined using these position vector, the coordinates of the midpoint G0, and the posture vector D _R. As well known, these coordinate transformations are performed by rotational transformation about an axis parallel to the Y axis passing through the midpoint G0 and translation transformation between the origin O of the real space coordinates and the midpoint G0. In addition, the object shape memory 42 stores the shape of a line segment connecting each vertex of the object A.

Further, the size calibration vertex coordinate changing circuit 52 obtains the size in the real space from the size of the feature on the image plane in consideration of the distance and height, the magnification of the image, and the imaging angle. Since the size of this feature portion in the real space is known, the magnification error and the imaging angle error are predicted from the ratio of this size. The vertex coordinates are corrected according to this error. Next, as shown in FIG. 13, each vertex coordinate of the object A in the real space is calculated by the vertex coordinate calculation circuit 51.
It is transformed into coordinates on the image plane using the camera parameters. The vertex coordinates on these image planes are stored in the vertex coordinate memory 55. In step 124, the vertex coordinate error calculation circuit 54 calculates the error of each vertex coordinate on the image plane based on the direction prediction error ΔD _GM and the size prediction errors ΔHi and ΔVi. That is, the existence position of each vertex coordinate varies depending on the posture prediction error and the magnification prediction error. Existable range E1, of each vertex coordinate at this time
E2 and the like are determined as shown in FIG. 14 and stored in the coordinate error memory 56.

4. In step 126 of setting a search area, the vertex candidate calculation circuit 61 determines a vertex candidate using the vertex coordinates and coordinate error. For example, as shown in FIG. 14, the first vertex T ₁ and the second vertex T _{1
In the} possible range E1 and the possible range E2 related to 2, for example, at the three locations of the upper stage, the middle stage, and the lower stage, the first
Vertex candidate T ₁₁ relative to the apex T _1, the coordinates of the T _12, T ₁₃ is determined, the vertex candidate T ₂₁ to the second vertex T _2, T _22, T
_Twenty-three coordinates are determined.

In step 128, the search area position determining circuit 62 causes the first vertex apex candidates T _11, T _12, T ₁₃ and the second apex vertex candidates T _21, T _{22, as shown in FIG.}
T ₂₃ and T ₂₃ are assumed to be connected by a line segment shape connecting the first vertex and the second vertex stored in the object shape memory 42, respectively, and a position where the search area is set on the line segment is assumed. The coordinates of F _1, F _2, ... F ₆ etc. are determined. The coordinates are stored in the search area position memory 63.

In step 130, the approximate line segment calculation circuit 6
4, an approximate line segment that approximates the line segment connecting the vertices with the inter-vertex line segment shape stored in the object shape memory 42 is calculated. Then, in step 132, as shown in FIG. 16, the approximate line segment L ₀ is determined, and the approximate line segment L ₀ is translated to both sides by the width ΔH determined according to the prediction error of the vertex coordinates. The shape of the search area S is determined. Data regarding the shape of the search area S is stored in the search area shape memory 66.

Next, the edge emphasis filter selection circuit 71 determines the type of line segment from the inter-vertex line segment shape stored in the object shape memory 42, and considers the existing direction and shape of the line segment, The edge enhancement filter is selected so that the edge can be detected most easily. That is, the type of operator differs depending on which direction the line segment faces. This operator type is selected.

Next, at step 134, the center position of the search area stored in the search area position memory 63 is sequentially read out by the edge emphasizing circuit 72 and stored in the search area shape memory 66 for each position. Based on the existing search area shape, the search area is movably set on the image plane as shown in FIG. Then, the edge emphasizing circuit 72 operates to calculate the differential image of the grayscale image stored in the image memory 13 for each search area, and the differential image is stored in the image memory 73.

Next, in step 136, the threshold value determining circuit 74 calculates, for each search area, the frequency distribution of the brightness of the differential image included in the search area, and the threshold value optimum for edge detection is calculated. Is determined. For example, if the frequency distribution of brightness is as shown in FIG. 17, it can be considered that the portion where the number of bright pixels is large is an edge. Therefore, by setting the level before the peak appears as a threshold value, the edges can be separated with high accuracy. Next, the binarization circuit 75 binarizes the differential image of each search area using the threshold value determined for each search area. This image is used as a precision edge image as shown in FIG.
6 is stored.

Next, in step 138, the edge line connecting circuit 81 connects the interrupted edge lines.
In this case, line segments whose directions match in a predetermined range, line segments whose distances between end points are in a predetermined range, and the like are regarded as continuous edges and are connected. Next, at step 140, the edge line approximation circuit 82 approximates the edge line segment as shown in FIG. 19 in consideration of the connected edge line segment and the adjacent connected edge line segments l _1, l _1, etc. The approximate line segment L _{1 and the} like are obtained. This approximate line segment is obtained by least-squares approximation for each edge line segment. In this way, as shown in FIG. 19, all the contour lines L _1, L _2, of the object A on the image plane _.
L ₃ etc. are determined.

In step 142, the vertex coordinate calculation circuit 83
Thus, as shown in FIG. 19, the intersection T ₁ = (U
Coordinates such as _T1, V _T1 ) _and T ₂ = (U _T2, V _T2 ) are calculated. Next, in step 144, the vertex three-dimensional position calculation circuit 85 uses the table in which the coordinates on the image plane and the coordinates in the real space stored in the position conversion table memory 84 are associated with each other to determine the real space of each vertex. Coordinate R at
₁ = (X _T1, Y _T1, Z _T1 ) _and R ₂ = (X _T2, Y _T2, Z _T2 ) are calculated. Next, in step 146, the posture calculation circuit 8
6, the position of the object from each vertex, for example, the first vertex T
The position of the object is determined by the coordinates of _{1 in} the real space, and from the vector directed from the first vertex T ₁ to the second vertex T ₂ , the angle between the vector and the Z axis is obtained. The angle determines the posture of the object A. When the position / orientation of the object A in the real space is determined, the positions of the insertion portions W1 and W2 of the forks, which are the characteristic parts, are determined, so that the object A can be gripped by the forklift or the like.

In the above-described embodiment, the characteristic portions W1 and W2 are the portions in which the contrast is strong in the image of the object A. However, a mark that is easy to recognize may be attached to the object and this mark may be used as the characteristic portion. Also, the precision edge extraction device 7
In the case of 0, in the search area, the edge emphasis image stored in the image memory 22 is used as it is instead of the grayscale image stored in the image memory 13, and the processes after the threshold value determination circuit 74 are performed. Alternatively, a precise edge image may be calculated. In this case, there is an advantage that the processing time can be shortened as compared with the above embodiment. Further, the three-dimensional model may be a plurality of images of the object A with respect to a typical angle at which the object A is imaged by the television camera 11. In this case, since the three-dimensional model is given by the data on the image plane, the existence position of the object on the image plane can be determined at high speed. Further, in this case, the reference mapping model on the image plane is corrected and displayed according to the imaging magnification, azimuth, imaging angle, and the like. In the above embodiment, since the outline of the object is represented by a straight line, the search area is also a rectangle. However, in the case of a cylindrical object, the contour appears as an elliptic curve. In this case, the shape of the search area is also a deformed strip shape along the elliptic curve.

[Brief description of drawings]

FIG. 1 is a block diagram showing the overall configuration of an object recognition device according to a specific embodiment of the present invention.

FIG. 2 is a block diagram showing a detailed configuration of each device of the embodiment device shown in FIG.

FIG. 3 is a block diagram showing a detailed configuration of each device of the embodiment device shown in FIG.

FIG. 4 is a block diagram showing a detailed configuration of each device of the embodiment device shown in FIG.

5 is a block diagram showing a detailed configuration of each device of the embodiment device shown in FIG.

FIG. 6 is a flowchart explaining the operation of the embodiment apparatus shown in FIG.

FIG. 7 is a flow chart for explaining the operation of the embodiment apparatus shown in FIG.

FIG. 8 is an explanatory diagram showing the shape of an object recognized by the device of this embodiment and the arrangement in the real space.

FIG. 9 is an explanatory diagram showing an edge image of an object.

FIG. 10 is an explanatory diagram showing a method of extracting a characteristic portion in an edge image.

FIG. 11 is an explanatory diagram showing a method of extracting a characteristic portion from an edge image.

FIG. 12 is an explanatory diagram showing a method of specifying a three-dimensional model of an object in a real space.

FIG. 13 is an explanatory diagram showing the existing positions of vertices on the image surface of the object obtained from the primary position / orientation of the object obtained by recognizing the characteristic portion on the image surface.

FIG. 14 is an explanatory diagram showing a possible existence range of vertices of an object on an image plane in consideration of a directional prediction error and a size prediction error.

FIG. 15 is an explanatory diagram showing a method of determining a search area position.

FIG. 16 is an explanatory diagram showing a method of determining the shape of a search area.

FIG. 17 is an explanatory diagram showing a method of determining a threshold value when binarizing a differential image in a search area.

FIG. 18 is an explanatory diagram showing precision edge images detected in each search region.

FIG. 19 is an explanatory diagram showing a method of calculating contour lines and vertices in a precision edge image.

FIG. 20 is a block diagram showing the overall configuration of the present invention.

[Explanation of symbols]

A ... object W1, W2 ... features G1, G2 ... centroid G0 ... existence range E2 ... second vertex relative position vector E1 ... first vertex of the vertex at the midpoint D ₁ to D ₈ ... 3 dimensional model of the center of gravity Existence possible range F _{1 to} F ₆ ... Setting position of search area S ... Search area T ₁ ... First vertex T ₂ ... Second vertex T ₁₁ ... First candidate of first vertex T ₁₂ ... Second candidate of first vertex T ₁₃ ... Third candidate of first vertex T ₂₁ ... First candidate of second vertex T ₂₂ ... Second candidate of second vertex T ₂₃ ... Third candidate of second vertex l _{1 to} l ₅ ... Connected edge L ₀ … Approximate line segment L _{1 to} L ₃ … Contour line

Claims (1)

[Claims] 1. An object recognition apparatus for recognizing the position and orientation of an object from an object image formed on an image plane by imaging the object with a camera, wherein a grayscale image of the object is obtained by imaging the object with the camera. A grayscale image capturing means for obtaining on the image plane, an edge image generating means for binarizing an edge image obtained by binarizing a differential image of the grayscale image on the image plane, and a contour of an entire object in the edge image. A characteristic part extracting means for extracting a characteristic part of an object whose relative relation to is known, and a primary position / posture for obtaining a primary position / orientation in the real space of the object based on the existing position of the characteristic part on the image plane. Based on the position / orientation calculation means and the primary position / orientation of the object, the contour of the object is hypothesized on the image plane, and the position where the contour is present is centered with a very small width. Search area setting means for movably setting a search area having a predetermined length along the contour on the image surface along the contour; and for each search area movably set along the contour, Precision edge extraction means for extracting a precise edge corresponding to the contour of the entire object by reprocessing the grayscale image or the differential image to extract an accurate edge of the object image, and the presence of the precise edge on the image surface. An object recognition device, comprising: a position / orientation calculating means for calculating the position / orientation of the object in the real space from the position.